DETR-基于Pytorch实现DN-DETR算法-附详细流程教程+模型下载-优质项目实战.zip

## Model We build upon DAB-DETR and add a denoising part to accelerate training convergence. It only adds minimal computation and will be removed during inference time.  We conduct extensive experiments to validate the effectiveness of our denoising training, for example, the convergnece curve comparison.  ## Model Zoo We provide our models under **DAB-DETR**, **DAB-Deformable-DETR(deformable encoder only)**, and **DAB-Deformable-DETR** You can also refer to our [[model zoo in google drive]](https://drive.google.com/drive/folders/1wmiZKfOg_DIGIwyQedV4rzNqUwHVTkUv?usp=sharing) [[model zoo in 百度网盘]](https://pan.baidu.com/s/13Dc_DgliVps4bHVlNujA4Q?pwd=niet)（提取码niet）. ### 50 epoch setting <table> <thead> <tr style="text-align: right;"> <th></th> <th>name</th> <th>backbone</th> <th>box AP</th> <th>Log/Config/Checkpoint</th> <th>Where in <a href="https://arxiv.org/pdf/2203.01305.pdf">Our Paper</a></th> </tr> </thead> <tbody> <tr> <th>0</th> <td>DN-DETR-R50</td> <td>R50</td> <td>44.4^{<a id="sup3c" herf="#sup1">1</a>}</td> <td><a href="https://drive.google.com/drive/folders/1kuwScU8PhN61qQOl5bbiPhKAYbzDHsWs?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1TqvnjsbAjARZp1i8cB2w8A?pwd=niet">BaiDu</a>&nbsp</td> <td>Table 1</td> </tr> <tr> <th>2</th> <td>DN-DETR-R50-DC5</td> <td>R50</td> <td>46.3</td> <td><a href="https://drive.google.com/drive/folders/1jr8BdDdMu8esABXdU3lNY7fpWVxAJtWa?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1lWrLUkxNfrncRTM-zmpbeA?pwd=niet">BaiDu</a>&nbsp;</td> <td>Table 1</td> </tr> <tr> <th>5</th> <td>DN-DAB-Deformbale-DETR<br>(Deformbale Encoder Only)^{<a id="sup3c" herf="#sup3">3</a>}</td> <td>R50</td> <td>48.6</td> <td><a href="https://drive.google.com/drive/folders/1TLIuvMw6F9lBv77gWQ3Qcn5tdfG7kqdU?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1emOumSadTJbCcoqxhKnllQ?pwd=niet">BaiDu</a>&nbsp</td> <td>Table 3</td> </tr> <tr> <th>6</th> <td>DN-DAB-Deformable-DETR-R50-v2^{<a id="sup4c" herf="#sup4">4</a>}</td> <td>R50</td> <td>49.5 (48.4 in 24 epochs)</td> <td><a href="https://drive.google.com/drive/folders/1pIllR0VfSIqX8TmQy0PFNiPdp87j-78j?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1ugoXlpr3x72qcXPKQ669sA?pwd=niet">BaiDu</a>&nbsp</td> <td>Optimized implementation with deformable attention in both encoder and decoder. See <a href="https://github.com/IDEA-opensource/DAB-DETR">DAB-DETR</a> for more details.</td> </tr> </tbody> </table> ### 12 epoch setting <table> <thead> <tr style="text-align: right;"> <th></th> <th>name</th> <th>backbone</th> <th>box AP</th> <th>Log/Config/Checkpoint</th> <th>Where in <a href="https://arxiv.org/pdf/2203.01305.pdf">Our Paper</a></th> </tr> </thead> <tbody> <tr> <th>1</th> <td>DN-DAB-DETR-R50-DC5(3 pat)^{<a id="sup2c" herf="#sup1">2</a>}</td> <td>R50</td> <td>41.7</td> <td><a href="https://drive.google.com/drive/folders/1jWSIWTWgoiIvyA7w2xIkdk-B1pS2atPA?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1fgCIGpMf0cgO6ToIb0xFuA?pwd=niet">BaiDu</a>&nbsp</td> <td>Table 2</td> </tr> <tr> <th>4</th> <td>DN-DAB-DETR-R101-DC5(3 pat)^{<a id="sup2c" herf="#sup1">2</a>}</td> <td>R101</td> <td>42.8</td> <td><a href="https://drive.google.com/drive/folders/1elPn06gs8mNxR3jtE53zi4cK5qLGH0AV?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1fgCIGpMf0cgO6ToIb0xFuA?pwd=niet">BaiDu</a>&nbsp</td> <td>Table 2</td> </tr> <tr> <th>5</th> <td>DN-DAB-Deformbale-DETR<br>(Deformble Encoder Only)^{<a id="sup3c" herf="#sup3">3</a>}</td> <td>R50</td> <td>43.4</td> <td><a href="https://drive.google.com/drive/folders/1T-qiHrvDF38PyqLhMU3QFC_n-ZkVgMJh?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1XygS_bjhe8Gg9tFhs9Z6uQ?pwd=niet">BaiDu</a>&nbsp</td> <td>Table 2</td> </tr> <tr> <th>5</th> <td>DN-DAB-Deformbale-DETR<br>(Deformble Encoder Only)^{<a id="sup3c" herf="#sup3">3</a>}</td> <td>R101</td> <td>44.1</td> <td><a href="https://drive.google.com/drive/folders/1V8CH7AMf9HGUYNNgEYaGK_2g2T2LNNWD?usp=sharing">Google Drive</a>&nbsp/&nbsp<a href="https://pan.baidu.com/s/1Q_C7FMXAlyPcWkIhbf5M6g?pwd=niet">BaiDu</a>&nbsp</td> <td>Table 2</td> </tr> </tbody> </table> Notes: - ^{<a id="sup1" herf="#sup1c">1</a>}: The result increases compared with the reported one in our paper (from ```44.1```to ```44.4```) since we optimized the code. We did not rerun other models, so <font color=red>**you are expected to get better performance than reported ones in our paper**.</font> - ^{<a id="sup2" herf="#sup2c">2</a>}: The models with marks (3 pat) are trained with multiple pattern embeds (refer to Anchor DETR or [DAB-DETR](https://arxiv.org/pdf/2203.01305.pdf) for more details.). - ^{<a id="sup3" herf="#sup3c">3</a>}: This model is based on DAB-Deformbale-DETR(Deformbale Encoder Only), which is a multiscale version of DAB-DETR. It requires 16 GPUs to train as it only use deformable attention in the encoder. - ^{<a id="sup4" herf="#sup4c">4</a>}: This model is based on DAB-Deformbale-DETR which is an optimized implementation with deformable DETR. See <a href="https://github.com/IDEA-opensource/DAB-DETR">DAB-DETR</a> for more details. <font color=red>**You are encouraged to use this deformable version**</font> as it uses deformable attention in both encoder and deocder, which is more lightweight (i.e, train with 4/8 A100 GPUs) and converges faster (i.e, achieves ```48.4``` in 24 epochs, comparable to the 50-epoch DAB-Deformable-DETR). # Usage ## How to use denoising training in your own model Our code largely follows DAB-DETR and adds additional components for denoising training, which are warped in a file [dn_components.py](models/DAB_DETR/dn_components.py). There are mainly 3 functions including **prepare_for_dn**, **dn_post_proces** (the first two are used in your detection forward function to process the dn part), and **compute_dn_loss**(this one is used to calculate dn loss). You can import these functions and add them to your own detection model. You may also compare DN-DETR and DAB-DETR to see how these functions are added if you would like to use it in your own detection models. You are also encouraged to apply it to some other DETR-like models or even traditional detection models and update results in this repo. ## Installation We use the DAB-DETR project as our codebase, hence no extra dependency is needed for our **DN-DETR**. For the **DN-Deformable-DETR**, you need to compile the deformable attention operator manually. We test our models under ```python=3.7.3,pytorch=1.9.0,cuda=11.1```. Other versions might be available as well. 1. Download the project 2. Install Pytorch and torchvision Follow the instruction on https://pytorch.org/get-started/locally/. ```sh # an example: conda install -c pytorch pytorch torchvision ``` 3. Install other needed packages ```sh pip install -r requirements.txt ``` 4. Compiling CUDA operators ```sh cd models/dn_dab_deformable_detr/ops python setup.py build install # unit test (should see all checking is True) python test.py cd ../../.. ``` ## Data Please download [COCO 2017](https://cocodataset.org/) dataset and organize them as following: ``` COCODIR/ ├── train2017/ ├── val2017/ └── annotations/ ├── instances_train2017.json └── instances_val2017.json