MindSpore用户增强模型安全性和可信度的工具箱 关注AI的安全和隐私问题

# Contents - [CNNCTC Description](#CNNCTC-description) - [Model Architecture](#model-architecture) - [Dataset](#dataset) - [Features](#features) - [Mixed Precision](#mixed-precision) - [Environment Requirements](#environment-requirements) - [Quick Start](#quick-start) - [Script Description](#script-description) - [Script and Sample Code](#script-and-sample-code) - [Script Parameters](#script-parameters) - [Training Process](#training-process) - [Training](#training) - [Distributed Training](#distributed-training) - [Evaluation Process](#evaluation-process) - [Evaluation](#evaluation) - [Inference Process](#inference-process) - [Export MindIR](#export-mindir) - [Infer on Ascend310](#infer-on-ascend310) - [result](#result) - [Model Description](#model-description) - [Performance](#performance) - [Training Performance](#training-performance) - [Evaluation Performance](#evaluation-performance) - [Inference Performance](#inference-performance) - [How to use](#how-to-use) - [Inference](#inference) - [Continue Training on the Pretrained Model](#continue-training-on-the-pretrained-model) - [Transfer Learning](#transfer-learning) - [Description of Random Situation](#description-of-random-situation) - [ModelZoo Homepage](#modelzoo-homepage) # [CNNCTC Description](#contents) This paper proposes three major contributions to addresses scene text recognition (STR). First, we examine the inconsistencies of training and evaluation datasets, and the performance gap results from inconsistencies. Second, we introduce a unified four-stage STR framework that most existing STR models fit into. Using this framework allows for the extensive evaluation of previously proposed STR modules and the discovery of previously unexplored module combinations. Third, we analyze the module-wise contributions to performance in terms of accuracy, speed, and memory demand, under one consistent set of training and evaluation datasets. Such analyses clean up the hindrance on the current comparisons to understand the performance gain of the existing modules. [Paper](https://arxiv.org/abs/1904.01906): J. Baek, G. Kim, J. Lee, S. Park, D. Han, S. Yun, S. J. Oh, and H. Lee, “What is wrong with scene text recognition model comparisons? dataset and model analysis,” ArXiv, vol. abs/1904.01906, 2019. # [Model Architecture](#contents) This is an example of training CNN+CTC model for text recognition on MJSynth and SynthText dataset with MindSpore. # [Dataset](#contents) Note that you can run the scripts based on the dataset mentioned in original paper or widely used in relevant domain/network architecture. In the following sections, we will introduce how to run the scripts using the related dataset below. The [MJSynth](https://www.robots.ox.ac.uk/~vgg/data/text/) and [SynthText](https://github.com/ankush-me/SynthText) dataset are used for model training. The [The IIIT 5K-word dataset](https://cvit.iiit.ac.in/research/projects/cvit-projects/the-iiit-5k-word-dataset) dataset is used for evaluation. - step 1: All the datasets have been preprocessed and stored in .lmdb format and can be downloaded [**HERE**](https://drive.google.com/drive/folders/192UfE9agQUMNq6AgU3_E05_FcPZK4hyt). - step 2: Uncompress the downloaded file, rename the MJSynth dataset as MJ, the SynthText dataset as ST and the IIIT dataset as IIIT. - step 3: Move above mentioned three datasets into `cnnctc_data` folder, and the structure should be as below: ```text |--- CNNCTC/ |--- cnnctc_data/ |--- ST/ data.mdb lock.mdb |--- MJ/ data.mdb lock.mdb |--- IIIT/ data.mdb lock.mdb ...... ``` - step 4: Preprocess the dataset by running: ```bash python src/preprocess_dataset.py ``` This takes around 75 minutes. # [Features](#contents) ## Mixed Precision The [mixed precision](https://mindspore.cn/tutorials/en/master/advanced/mixed_precision.html) training method accelerates the deep learning neural network training process by using both the single-precision and half-precision data formats, and maintains the network precision achieved by the single-precision training at the same time. Mixed precision training can accelerate the computation process, reduce memory usage, and enable a larger model or batch size to be trained on specific hardware. For FP16 operators, if the input data type is FP32, the backend of MindSpore will automatically handle it with reduced precision. Users could check the reduced-precision operators by enabling INFO log and then searching ‘reduce precision’. # [Environment Requirements](#contents) - Hardware（Ascend/GPU） - Prepare hardware environment with Ascend or GPU processor. - Framework - [MindSpore](https://www.mindspore.cn/install/en) - For more information, please check the resources below： - [MindSpore tutorials](https://www.mindspore.cn/tutorials/en/master/index.html) - [MindSpore Python API](https://www.mindspore.cn/docs/en/master/index.html) # [Quick Start](#contents) - Install dependencies: ```bash pip install lmdb pip install Pillow pip install tqdm pip install six ``` ```default_config.yaml TRAIN_DATASET_PATH: /home/DataSet/MJ-ST-IIIT/ST-MJ/ TRAIN_DATASET_INDEX_PATH: /home/DataSet/MJ-ST-IIIT/st_mj_fixed_length_index_list.pkl TEST_DATASET_PATH: /home/DataSet/MJ-ST-IIIT/IIIT5K_3000 Modify the parameters according to the actual path ``` - Standalone Ascend Training: ```bash bash scripts/run_standalone_train_ascend.sh $DEVICE_ID $PRETRAINED_CKPT(options) # example: bash scripts/run_standalone_train_ascend.sh 0 ``` - Standalone GPU Training: ```bash bash scripts/run_standalone_train_gpu.sh $PRETRAINED_CKPT(options) ``` - Distributed Ascend Training: ```bash bash scripts/run_distribute_train_ascend.sh $RANK_TABLE_FILE $PRETRAINED_CKPT(options) # example: bash scripts/run_distribute_train_ascend.sh ~/hccl_8p.json ``` - Distributed GPU Training: ```bash bash scripts/run_distribute_train_gpu.sh $PRETRAINED_CKPT(options) ``` - Ascend Evaluation: ```bash bash scripts/run_eval_ascend.sh $DEVICE_ID $TRAINED_CKPT # example: scripts/run_eval_ascend.sh 0 /home/model/cnnctc/ckpt/CNNCTC-1_8000.ckpt ``` - GPU Evaluation: ```bash bash scripts/run_eval_gpu.sh $TRAINED_CKPT ``` # [Script Description](#contents) ## [Script and Sample Code](#contents) The entire code structure is as following: ```text |--- CNNCTC/ |---README.md // descriptions about cnnctc |---README_cn.md // descriptions about cnnctc |---default_config.yaml // config file |---train.py // train scripts |---eval.py // eval scripts |---export.py // export scripts |---preprocess.py // preprocess scripts |---postprocess.py // postprocess scripts |---ascend310_infer // application for 310 inference |---scripts |---run_infer_310.sh // shell script for infer on ascend310 |---run_standalone_train_ascend.sh // shell script for standalone on ascend |---run_standalone_train_gpu.sh // shell script for standalone on gpu |---run_distribute_train_ascend.sh // shell script for distributed on ascend |---run_distribute_train_gpu.sh // shell script for distributed on gpu |---run_eval_ascend.sh // shell script for eval on ascend |---src |---__init__.py // init file |---cnn_ctc.py // cnn_ctc network |---callback.py // loss callback file |---dataset.py // process dataset |---util.py // routine operation |---preprocess_dataset.py // preprocess dataset |--- model_utils |---config.py // Parameter config |---moxing_adapter.py // modelarts device configuration |---device_adapter.py // Device Config |---local_ada