Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
jeffding ce74a2ab6f | 1 year ago | |
---|---|---|
.. | ||
ascend310_infer | 1 year ago | |
infer | 1 year ago | |
modelarts | 1 year ago | |
scripts | 1 year ago | |
src | 1 year ago | |
Dockerfile | 1 year ago | |
README.md | 1 year ago | |
README_CN.md | 1 year ago | |
__init__.py | 1 year ago | |
default_config.yaml | 1 year ago | |
export.py | 1 year ago | |
ma-pre-start.sh | 1 year ago | |
mindspore_hub_conf.py | 1 year ago | |
pip-requirements.txt | 1 year ago | |
postprocess.py | 1 year ago | |
requirements.txt | 1 year ago | |
test.py | 1 year ago | |
train.py | 1 year ago |
With the development of convolutional neural network, scene text detection technology has been developed rapidly. However, there are still two problems in this algorithm, which hinders its application in industry. On the one hand, most of the existing algorithms require quadrilateral bounding boxes to accurately locate arbitrary shape text. On the other hand, two adjacent instances of text can cause error detection overwriting both instances. Traditionally, a segmentation-based approach can solve the first problem, but usually not the second. To solve these two problems, a new PSENet (PSENet) is proposed, which can accurately detect arbitrary shape text instances. More specifically, PSENet generates different scale kernels for each text instance and gradually expands the minimum scale kernel to a text instance with full shape. Because of the large geometric margins between the minimum scale kernels, our method can effectively segment closed text instances, making it easier to detect arbitrary shape text instances. The effectiveness of PSENet has been verified by numerous experiments on CTW1500, full text, ICDAR 2015, and ICDAR 2017 MLT.
Paper: Wenhai Wang, Enze Xie, Xiang Li, Wenbo Hou, Tong Lu, Gang Yu, Shuai Shao; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 9336-9345
Progressive Scale Expansion Network (PSENet) is a text detector which is able to well detect the arbitrary-shape text in natural scene.
Note that you can run the scripts based on the dataset mentioned in original paper or widely used in relevant domain/network architecture. In the following sections, we will introduce how to run the scripts using the related dataset below.
Dataset used: ICDAR2015
A training set of 1000 images containing about 4500 readable words
A testing set containing about 2000 readable words
unzip dataset files and needn't transform to mindrecord.
download pytorch pretrained model: resnet50-19c8e357.pth
transform pytorch model to mindspore model
cd src
python psenet_model_torch2mindspore.py --torch_file=/path_to_model/resnet50-19c8e357.pth --output_path=../
# install pybind11
pip install pybind11
# install opencv3.4.9 or opencv4.5.3
wget https://github.com/opencv/opencv/archive/3.4.9.zip
# or wget https://github.com/opencv/opencv/archive/4.5.3.zip
unzip 3.4.9.zip
cd opencv-3.4.9
mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_WEBP=OFF ..
make -j4 # -j specifies the number of threads, the user can modify the parameters according to the machine configuration
make install
# export environment variables
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/usr/local/include
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib64
After installing MindSpore via the official website, you can start training and evaluation as follows:
# run distributed training example
bash scripts/run_distribute_train.sh [RANK_TABLE_FILE] [PRED_TRAINED PATH] [TRAIN_ROOT_DIR]
#enter the path ,run Makefile
cd ./src/PSENET/pse/;make
#run test.py
python test.py --ckpt pretrained_model.ckpt --TEST_ROOT_DIR [test root path]
#go to Evaluation Process for details
download script.py
# run evaluation example
bash scripts/run_eval_ascend.sh
# Example of using distributed training on modelarts :
# Data set storage method
# ├── ICDAR2015 # dir
# ├── train # train dir
# ├── ic15 # train_dataset dir
# ├── ch4_training_images
# ├── ch4_training_localization_transcription_gt
# ├── train_predtrained # predtrained dir
# ├── eval # eval dir
# ├── ic15 # eval dataset dir
# ├── ch4_test_images
# ├── challenge4_Test_Task1_GT
# ├── checkpoint # ckpt files dir
# (1) Choose either a (modify yaml file parameters) or b (modelArts create training job to modify parameters) 。
# a. set "enable_modelarts=True" 。
# set "run_distribute=True"
# set "TRAIN_MODEL_SAVE_PATH=/cache/train/outputs_imagenet/"
# set "TRAIN_ROOT_DIR=/cache/data/ic15/"
# set "pre_trained=/cache/data/train_predtrained/pred file name" Without pre-training weights train_pretrained=""
# b. add "enable_modelarts=True" Parameters are on the interface of modearts。
# Set the parameters required by method a on the modelarts interface
# Note: The path parameter does not need to be quoted
# (2) Set the path of the network configuration file "_config_path=/The path of config in default_config.yaml/"
# (3) Set the code path on the modelarts interface "/path/psenet"。
# (4) Set the model's startup file on the modelarts interface "train.py" 。
# (5) Set the data path of the model on the modelarts interface ".../ICDAR2015/train"(choices ICDAR2015/train Folder path) ,
# The output path of the model "Output file path" and the log path of the model "Job log path" 。
# (6) start trainning the model。
# Example of using model inference on modelarts
# (1) Place the trained model to the corresponding position of the bucket。
# (2) chocie a or b。
# a. set "enable_modelarts=True" 。
# set "TEST_ROOT_DIR=/cache/data/ic15/"
# set "ckpt=/cache/data/checkpoint/ckpt file"
# b. Add "enable_modelarts=True" parameter on the interface of modearts。
# Set the parameters required by method a on the modelarts interface
# Note: The path parameter does not need to be quoted
# (3) Set the path of the network configuration file "_config_path=/The path of config in default_config.yaml/"
# (4) Set the code path on the modelarts interface "/path/psenet"。
# (5) Set the model's startup file on the modelarts interface "eval.py" 。
# (6) Set the data path of the model on the modelarts interface ".../ICDAR2015/eval"(choices ICDAR2015/eval Folder path) ,
# The output path of the model "Output file path" and the log path of the model "Job log path" 。
# (7) Start model inference。
└── psenet
├── export.py // export mindir file
├── postprocess.py // 310 Inference post-processing script
├── __init__.py
├── mindspore_hub_conf.py // hub config file
├── README_CN.md // descriptions about PSENet in Chinese
├── README.md // descriptions about PSENet in English
├── scripts
├── run_distribute_train.sh // shell script for distributed ascend
├── run_distribute_train_gpu.sh // shell script for distributed gpu
├── run_eval_ascend.sh // shell script for evaluation ascend
├── run_eval_gpu.sh // shell script for evaluation gpu
├── ascend310_infer // application for 310 inference
├── src
├── model_utils
├──config.py // Parameter config
├──moxing_adapter.py // modelarts device configuration
├──device_adapter.py // Device Config
├──local_adapter.py // local device config
├── dataset.py // creating dataset
├── PSENET
├── base.py // convolution and BN operator
├── dice_loss.py // calculate PSENet loss value
├── psenet.py // Subnet in PSENet
├── fpn.py // Subnet in PSENet
├── __init__.py
├── pse // Subnet in PSENet
├── __init__.py
├── adaptor.cpp
├── adaptor.h
├── Makefile
├── resnet50.py // Subnet in PSENet
├── __init__.py
├── lr_schedule.py // define learning rate
├── network_define.py // learning ratio generation
├── test.py // test script
├── train.py // training script
├── default_config.yaml // config file
Major parameters in default_config.yaml are:
--device_target: Ascend or GPU
--pre_trained: Whether training from scratch or training based on the
pre-trained model.Optional values are True, False.
--device_id: Device ID used to train or evaluate the dataset. Ignore it
when you use train.sh for distributed training.
For distributed ascend training, a hccl configuration file with JSON format needs to be created in advance.
Please follow the instructions in the link below: https://gitee.com/mindspore/models/tree/master/utils/hccl_tools.
bash scripts/run_distribute_train.sh [RANK_FILE] [PRETRAINED_PATH] [TRAIN_ROOT_DIR]
rank_table_file which is specified by RANK_TABLE_FILE is needed when you are running a distribute task. You can generate it by using the hccl_tools.
The above shell script will run distribute training in the background. You can view the results through the file
device[X]/test_*.log
. The loss value will be achieved as follows:
# grep "epoch: " device_*/loss.log
device_0/log:epoch: 1, step: 20, loss is 0.80383
device_0/log:epcoh: 2, step: 40, loss is 0.77951
...
device_1/log:epoch: 1, step: 20, loss is 0.78026
device_1/log:epcoh: 2, step: 40, loss is 0.76629
bash scripts/run_distribute_train_gpu.sh [PRED_TRAINED PATH] [TRAIN_ROOT_DIR]
After training begins, log and loss.log file will be in train_parallel directory.
# cat train_parallel/loss.log
time: 2021-07-24 02:08:33, epoch: 10, step: 31, loss is 0.68408
time: 2021-07-24 02:08:33, epoch: 10, step: 31, loss is 0.67984
...
time: 2021-07-24 04:01:07, epoch: 90, step: 31, loss is 0.61662
time: 2021-07-24 04:01:07, epoch: 90, step: 31, loss is 0.58495
python test.py --ckpt [CKPK_PATH] --TEST_ROOT_DIR [TEST_DATA_DIR]
# click [Here](https://rrc.cvc.uab.es/?ch=4&com=tasks#TextLocalization) to download evaluation scripts
# choose My Methods -> Offline evaluation -> Evaluation Scripts
# download data and put it in /path_to_data
mkdir eval_ic15
ln -s /path_to_data/script_test_ch4_t1_e1-1577983151.zip eval_ic15/script_test_ch4_t1_e1-1577983151.zip
cd eval_ic15
unzip script_test_ch4_t1_e1-1577983151.zip
cd ..
sh ./script/run_eval_ascend.sh
python test.py --ckpt [CKPK PATH] --TEST_ROOT_DIR [TEST DATA DIR]
Calculated!{"precision": 0.8147966668299853,"recall":0.8006740491092923,"hmean":0.8076736279747451,"AP":0}
step 1: download eval method from here.
step 2: click "My Methods" button,then download Evaluation Scripts.
step 3: it is recommended to symlink the eval method root to $MINDSPORE/model_zoo/psenet/eval_ic15/. if your folder structure is different,you may need to change the corresponding paths in eval script files.
bash ./script/run_eval_ascend.sh
The two scripts ./script/run_eval_ascend.sh and ./script/run_eval_gpu.sh are the same, you may run either for evaluating on ICDAR2015.
Calculated!{"precision": 0.814796668299853, "recall": 0.8006740491092923, "hmean": 0.8076736279747451, "AP": 0}
python export.py --ckpt [CKPT_PATH] --file_name [FILE_NAME] --file_format [FILE_FORMAT]
The ckpt_file parameter is required,
FILE_FORMAT
should be in ["AIR", "MINDIR"]
Export MindIR example on ModelArts
Data storage method is the same as training
# (1) Choose either a (modify yaml file parameters) or b (modelArts create training job to modify parameters)。
# a. set "enable_modelarts=True"
# set "file_name=psenet"
# set "file_format=MINDIR"
# set "ckpt_file=/cache/data/checkpoint file name"
# b. Add "enable_modelarts=True" parameter on the interface of modearts。
# Set the parameters required by method a on the modelarts interface
# Note: The path parameter does not need to be quoted
# (2)Set the path of the network configuration file "_config_path=/The path of config in default_config.yaml/"
# (3) Set the code path on the modelarts interface "/path/psenet"。
# (4) Set the model's startup file on the modelarts interface "export.py" 。
# (5) Set the data path of the model on the modelarts interface ".../ICDAR2015/eval/checkpoint"(choices ICDAR2015/eval/checkpoint Folder path) ,
# The output path of the model "Output file path" and the log path of the model "Job log path" 。
Before performing inference, the mindir file must be exported by export.py
script. We only provide an example of inference using MINDIR model.
Current batch_Size can only be set to 1. Before running the following process, please configure the environment by following the instructions provided in Quick start.
# Ascend310 inference
bash run_infer_310.sh [MINDIR_PATH] [DATA_PATH] [DEVICE_ID]
DEVICE_ID
is optional, default value is 0.The res
folder is generated in the upper-level directory. For details about the final precision calculation, see Eval Script for ICDAR2015.
Parameters | Ascend |
---|---|
Model Version | PSENet |
Resource | Ascend 910; CPU 2.60GHz, 192cores; Memory 755G; OS Euler2.8 |
uploaded Date | 09/30/2020 (month/day/year) |
MindSpore Version | 1.0.0 |
Dataset | ICDAR2015 |
Training Parameters | start_lr=0.1; lr_scale=0.1 |
Optimizer | SGD |
Loss Function | LossCallBack |
outputs | probability |
Loss | 0.35 |
Parameters | batch_size = 4 |
Speed | 1pc: 444 ms/step(fps: 9.0); 8pcs: 465 ms/step(fps: 68.8) |
Total time | 1pc: 75.48 h; 8pcs: 7.11 h |
Parameters (M) | 27.36 |
Checkpoint for Fine tuning | 109.44M (.ckpt file) |
Scripts | https://gitee.com/mindspore/models/tree/master/official/cv/psenet |
Parameters | GPU |
---|---|
Model Version | PSENet |
Resource | GPU(Tesla V100-PCIE); CPU 2.60 GHz, 26 cores; Memory 790G; OS Euler2.0 |
uploaded Date | 07/24/2021 (month/day/year) |
MindSpore Version | 1.3.0 |
Dataset | ICDAR2015 |
Training Parameters | start_lr=0.1; lr_scale=0.1 |
Optimizer | SGD |
Loss Function | LossCallBack |
outputs | probability |
Loss | 0.40 |
Speed | 1pc: 2726 ms/step; 8pcs: 2726 ms/step |
Total time | 1pc: 335.6 h; 8pcs: 41.95 h |
Parameters (M) | 27.36 |
Checkpoint for Fine tuning | 109.44M (.ckpt file) |
Scripts | https://gitee.com/mindspore/models/tree/master/official/cv/psenet |
Parameters | Ascend |
---|---|
Model Version | PSENet |
Resource | Ascend 910 |
Uploaded Date | 09/30/2020 (month/day/year) |
MindSpore Version | 1.0.0 |
Dataset | ICDAR2015 |
outputs | probability |
Accuracy | 1pc: 81%; 8pcs: 81% |
If you need to use the trained model to perform inference on multiple hardware platforms, such as GPU, Ascend 910 or Ascend 310, you can refer to this Link. Following the steps below, this is a simple example:
# Load unseen dataset for inference
dataset = dataset.create_dataset(cfg.data_path, 1, False)
# Define model
config.INFERENCE = False
net = PSENet(config)
net = net.set_train()
param_dict = load_checkpoint(args.pre_trained)
load_param_into_net(net, param_dict)
print('Load Pretrained parameters done!')
criterion = DiceLoss(batch_size=config.TRAIN_BATCH_SIZE)
lrs = lr_generator(start_lr=1e-3, lr_scale=0.1, total_iters=config.TRAIN_TOTAL_ITER)
opt = nn.SGD(params=net.trainable_params(), learning_rate=lrs, momentum=0.99, weight_decay=5e-4)
# warp model
net = WithLossCell(net, criterion)
net = TrainOneStepCell(net, opt)
time_cb = TimeMonitor(data_size=step_size)
loss_cb = LossCallBack(per_print_times=20)
# set and apply parameters of check point
ckpoint_cf = CheckpointConfig(save_checkpoint_steps=1875, keep_checkpoint_max=2)
ckpoint_cb = ModelCheckpoint(prefix="PSENet", config=ckpoint_cf, directory=config.TRAIN_MODEL_SAVE_PATH)
model = Model(net)
model.train(config.TRAIN_REPEAT_NUM, ds, dataset_sink_mode=False, callbacks=[time_cb, loss_cb, ckpoint_cb])
# Load pre-trained model
param_dict = load_checkpoint(cfg.checkpoint_path)
load_param_into_net(net, param_dict)
net.set_train(False)
# Make predictions on the unseen dataset
acc = model.eval(dataset)
print("accuracy: ", acc)
Huwei Ascend 910 Code
Text Unity3D Asset nesC Python Jupyter Notebook other
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》