基于PaddleOCR文字识别实现视频中关键字替换或遮盖项目源码+使用说明.zip

English | [简体中文](README_ch.md) # Layout analysis - [1. Introduction](#1-Introduction) - [2. Quick start](#2-Quick-start) - [3. Install](#3-Install) - [3.1 Install PaddlePaddle](#31-Install-paddlepaddle) - [3.2 Install PaddleDetection](#32-Install-paddledetection) - [4. Data preparation](#4-Data-preparation) - [4.1 English data set](#41-English-data-set) - [4.2 More datasets](#42-More-datasets) - [5. Start training](#5-Start-training) - [5.1 Train](#51-Train) - [5.2 FGD Distillation training](#52-Fgd-distillation-training) - [6. Model evaluation and prediction](#6-Model-evaluation-and-prediction) - [6.1 Indicator evaluation](#61-Indicator-evaluation) - [6.2 Test layout analysis results](#62-Test-layout-analysis-results) - [7. Model export and inference](#7-Model-export-and-inference) - [7.1 Model export](#71-Model-export) - [7.2 Model inference](#72-Model-inference) ## 1. Introduction Layout analysis refers to the regional division of documents in the form of pictures and the positioning of key areas, such as text, title, table, picture, etc. The layout analysis algorithm is based on the lightweight model PP-picodet of [PaddleDetection]( https://github.com/PaddlePaddle/PaddleDetection ) <div align="center"> <img src="../docs/layout/layout.png" width="800"> </div> ## 2. Quick start PP-Structure currently provides layout analysis models in Chinese, English and table documents. For the model link, see [models_list](../docs/models_list_en.md). The whl package is also provided for quick use, see [quickstart](../docs/quickstart_en.md) for details. ## 3. Install ### 3.1. Install PaddlePaddle - **（1) Install PaddlePaddle** ```bash python3 -m pip install --upgrade pip # GPU Install python3 -m pip install "paddlepaddle-gpu>=2.3" -i https://mirror.baidu.com/pypi/simple # CPU Install python3 -m pip install "paddlepaddle>=2.3" -i https://mirror.baidu.com/pypi/simple ``` For more requirements, please refer to the instructions in the [Install file](https://www.paddlepaddle.org.cn/install/quick)。 ### 3.2. Install PaddleDetection - **（1）Download PaddleDetection Source code** ```bash git clone https://github.com/PaddlePaddle/PaddleDetection.git ``` - **（2）Install third-party libraries** ```bash cd PaddleDetection python3 -m pip install -r requirements.txt ``` ## 4. Data preparation If you want to experience the prediction process directly, you can skip data preparation and download the pre-training model. ### 4.1. English data set Download document analysis data set [PubLayNet](https://developer.ibm.com/exchanges/data/all/publaynet/)（Dataset 96G），contains 5 classes：`{0: "Text", 1: "Title", 2: "List", 3:"Table", 4:"Figure"}` ``` # Download data wget https://dax-cdn.cdn.appdomain.cloud/dax-publaynet/1.0.0/publaynet.tar.gz # Decompress data tar -xvf publaynet.tar.gz ``` Uncompressed **directory structure：** ``` |-publaynet |- test |- PMC1277013_00004.jpg |- PMC1291385_00002.jpg | ... |- train.json |- train |- PMC1291385_00002.jpg |- PMC1277013_00004.jpg | ... |- val.json |- val |- PMC538274_00004.jpg |- PMC539300_00004.jpg | ... ``` **data distribution：** | File or Folder | Description | num | | :------------- | :------------- | ------- | | `train/` | Training set pictures | 335,703 | | `val/` | Verification set pictures | 11,245 | | `test/` | Test set pictures | 11,405 | | `train.json` | Training set annotation files | - | | `val.json` | Validation set dimension files | - | **Data Annotation** The JSON file contains the annotations of all images, and the data is stored in a dictionary nested manner.Contains the following keys： - info，represents the dimension file info。 - licenses，represents the dimension file licenses。 - images，represents the list of image information in the annotation file，each element is the information of an image。The information of one of the images is as follows: ``` { 'file_name': 'PMC4055390_00006.jpg', # file_name 'height': 601, # image height 'width': 792, # image width 'id': 341427 # image id } ``` - annotations， represents the list of annotation information of the target object in the annotation file，each element is the annotation information of a target object。The following is the annotation information of one of the target objects: ``` { 'segmentation': # Segmentation annotation of objects 'area': 60518.099043117836, # Area of object 'iscrowd': 0, # iscrowd 'image_id': 341427, # image id 'bbox': [50.58, 490.86, 240.15, 252.16], # bbox [x1,y1,w,h] 'category_id': 1, # category_id 'id': 3322348 # image id } ``` ### 4.2. More datasets We provide CDLA(Chinese layout analysis), TableBank(Table layout analysis)etc. data set download links，process to the JSON format of the above annotation file，that is, the training can be conducted in the same way。 | dataset | 简介 | | ------------------------------------------------------------ | ------------------------------------------------------------ | | [cTDaR2019_cTDaR](https://cndplab-founder.github.io/cTDaR2019/) | For form detection (TRACKA) and form identification (TRACKB).Image types include historical data sets (beginning with cTDaR_t0, such as CTDAR_T00872.jpg) and modern data sets (beginning with cTDaR_t1, CTDAR_T10482.jpg). | | [IIIT-AR-13K](http://cvit.iiit.ac.in/usodi/iiitar13k.php) | Data sets constructed by manually annotating figures or pages from publicly available annual reports, containing 5 categories:table, figure, natural image, logo, and signature. | | [TableBank](https://github.com/doc-analysis/TableBank) | For table detection and recognition of large datasets, including Word and Latex document formats | | [CDLA](https://github.com/buptlihang/CDLA) | Chinese document layout analysis data set, for Chinese literature (paper) scenarios, including 10 categories:Table, Figure, Figure caption, Table, Table caption, Header, Footer, Reference, Equation | | [DocBank](https://github.com/doc-analysis/DocBank) | Large-scale dataset (500K document pages) constructed using weakly supervised methods for document layout analysis, containing 12 categories:Author, Caption, Date, Equation, Figure, Footer, List, Paragraph, Reference, Section, Table, Title | ## 5. Start training Training scripts, evaluation scripts, and prediction scripts are provided, and the PubLayNet pre-training model is used as an example in this section. If you do not want training and directly experience the following process of model evaluation, prediction, motion to static, and inference, you can download the provided pre-trained model (PubLayNet dataset) and skip this part. ``` mkdir pretrained_model cd pretrained_model # Download PubLayNet pre-training model（Direct experience model evaluates, predicts, and turns static） wget https://paddleocr.bj.bcebos.com/ppstructure/models/layout/picodet_lcnet_x1_0_fgd_layout.pdparams # Download the PubLaynet inference model（Direct experience model reasoning） wget https://paddleocr.bj.bcebos.com/ppstructure/models/layout/picodet_lcnet_x1_0_fgd_layout_infer.tar ``` If the test image is Chinese, the pre-trained model of Chinese CDLA dataset can be downloaded to identify 10 types of document regions：Table, Figure, Figure caption, Table, Table caption, Header, Footer, Reference, Equation，Download the training model and inference model of Model 'picodet_lcnet_x1_0_fgd_layout_cdla' in [layout analysis model](https://github.com/PaddlePaddle/PaddleOCR/blob/dygraph/ppstructure/docs/models_list.md)�