# DETR : End-to-End Object Detection with Transformers (Tensorflow)
Tensorflow implementation of DETR : Object Detection with Transformers, including code for inference, training, and finetuning. DETR is a promising model that brings widely adopted transformers to vision models. We believe that models based on convolution and transformers will soon become the default choice for most practitioners because of the simplicity of the training procedure: NMS and anchors free! Therefore this repository is a step toward making this type of architecture widely available.
<img src="images/detr-figure.png"></img>
<b>About this implementation:</b> This repository includes codes to run an inference with the original model's weights (based on the PyTorch weights), to train the model from scratch (multi-GPU training support coming soon) as well as examples to finetune the model on your dataset. Unlike the PyTorch implementation, the training uses fixed image sizes and a standard Adam optimizer with gradient norm clipping.
<img src="images/wandb_logging.png"></img>
## Install
The code has been currently tested with Tensorflow 2.3.0 and python 3.7. The following dependencies are required.
```
wandb
matplotlib
numpy
pycocotools
scikit-image
imageio
pandas
```
```
pip install -r requirements.txt
```
## Datasets
This repository currently supports three dataset formats: **COCO**, **VOC**, and **Tensorflow Object detection csv**. The easiest way to get started is to set up your dataset based on one of these formats. Along with the datasets, we provide a code example to finetune your model.
Finally, we provide a jupyter notebook to help you understand how to load a dataset, set up a custom dataset, and finetune your model.
<img src="images/datasetsupport.png"></img>
## Tutorials
To get started with the repository you can check the following Jupyter notebooks:
- â [DETR Tensorflow - How to load a dataset.ipynb](./notebooks/How%20to%20load%20a%20dataset.ipynb)
- â [DETR Tensorflow - Finetuning tutorial.ipynb](./notebooks/DETR%20Tensorflow%20-%20%20Finetuning%20tutorial.ipynb)
- â [DETR Tensorflow - How to setup a custom dataset.ipynb](./notebooks/DETR%20Tensorflow%20-%20%20How%20to%20setup%20a%20custom%20dataset.ipynb)
As well as the logging board on wandb https://wandb.ai/thibault-neveu/detr-tensorflow-log and this report:
- ð [Finetuning DETR on Tensorflow - A step by step guide](https://wandb.ai/thibault-neveu/detr-tensorflow-log/reports/Finetuning-DETR-on-Tensorflow-A-step-by-step-tutorial--VmlldzozOTYyNzQ)
## Evaluation
Run the following to evaluate the model using the pre-trained weights.
- **data_dir** is your coco dataset folder
- **img_dir** is the image folder relative to the data_dir
- **ann_file** is the validation annotation file relative to the data_dir
Checkout â [DETR Tensorflow - How to load a dataset.ipynb](./notebooks/How%20to%20load%20a%20dataset.ipynb) for more information about the supported dataset ans their usage.
```
python eval.py --data_dir /path/to/coco/dataset --img_dir val2017 --ann_file annotations/instances_val2017.json
```
Outputs:
```
| all | .50 | .55 | .60 | .65 | .70 | .75 | .80 | .85 | .90 | .95 |
-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
box | 36.53 | 55.38 | 53.13 | 50.46 | 47.11 | 43.07 | 38.11 | 32.10 | 25.01 | 16.20 | 4.77 |
mask | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 | 0.00 |
-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+-------+
```
The result is not the same as reported in the paper because the evaluation is run on the <b>original image size</b> and not on the larger images. The actual implementation resizes the image so that the shorter side is at least 800pixels and the longer side at most 1333.
## Finetuning
To fine-tune the model on a new dataset we siply need to set the number of class to detect in our new dataset (**nb_class**). The method will remove the last layers that predict the box class&positions and add new layers to finetune.
```python
# Load the pretrained model
detr = get_detr_model(config, include_top=False, nb_class=3, weights="detr", num_decoder_layers=6, num_encoder_layers=6)
detr.summary()
# Load your dataset
train_dt, class_names = load_tfcsv_dataset(config, config.batch_size, augmentation=True)
# Setup the optimziers and the trainable variables
optimzers = setup_optimizers(detr, config)
# Train the model
training.fit(detr, train_dt, optimzers, config, epoch_nb, class_names)
```
The following commands gives some examples to finetune the model on new datasets: (Pacal VOC) and (The Hard hat dataset), with a real ```batch_size``` of 8 and a virtual ```target_batch``` size (gradient aggregate) of 32. ```--log``` is used for logging the training into wandb.
- **data_dir** is your voc dataset folder
- **img_dir** is the image folder relative to the data_dir
- **ann_file** is the validation annotation file relative to the data_dir
```
python finetune_voc.py --data_dir /home/thibault/data/VOCdevkit/VOC2012 --img_dir JPEGImages --ann_dir Annotations --batch_size 8 --target_batch 32 --log
```
- **data_dir** is the hardhatcsv dataset folder
- **img_dir** and **ann_file** set in the training file to load the training and validation differently
Checkout â [DETR Tensorflow - How to load a dataset.ipynb](./notebooks/How%20to%20load%20a%20dataset.ipynb) for more information about the supported dataset ans their usage.
```
python finetune_hardhat.py --data_dir /home/thibault/data/hardhat --batch_size 8 --target_batch 32 --log
```
## Training
(Multi GPU training comming soon)
- **data_dir** is the coco dataset folder
- **img_dir** and **ann_file** set in the training file to load the training and validation differently.
```
python train_coco.py --data_dir /path/to/COCO --batch_size 8 --target_batch 32 --log
```
## Inference
Here is an example of running an inference with the model on your webcam.
```
python webcam_inference.py
```
<img src="images/webcam_detr.png" width="400"></img>
没有合适的资源?快使用搜索试试~ 我知道了~
DETR-基于TensorFlow实现DETR目标检测算法-支持训练和微调+附详细流程教程+项目源码-优质项目实战.zip
共40个文件
py:28个
png:6个
ipynb:3个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 110 浏览量
2024-05-11
17:01:24
上传
评论
收藏 2.96MB ZIP 举报
温馨提示
DETR_基于TensorFlow实现DETR目标检测算法_支持训练和微调+附详细流程教程+项目源码_优质项目实战
资源推荐
资源详情
资源评论
收起资源包目录
DETR_基于TensorFlow实现DETR目标检测算法_支持训练和微调+附详细流程教程+项目源码_优质项目实战.zip (40个子文件)
DETR_基于TensorFlow实现DETR目标检测算法_支持训练和微调+附详细流程教程+项目源码_优质项目实战
train_coco.py 3KB
eval.py 3KB
finetune_voc.py 4KB
finetune_coco.py 2KB
detr_tf
loss
loss.py 6KB
hungarian_matching.py 7KB
compute_map.py 11KB
networks
weights.py 1KB
custom_layers.py 3KB
position_embeddings.py 2KB
detr.py 8KB
transformer.py 14KB
resnet_backbone.py 5KB
optimizers.py 6KB
training.py 4KB
data
__init__.py 166B
processing.py 2KB
coco.py 5KB
voc.py 5KB
tfcsv.py 3KB
transformation.py 5KB
inference.py 3KB
logger
wandb_logging.py 14KB
training_logging.py 4KB
training_config.py 5KB
bbox.py 10KB
finetune_hardhat.py 3KB
requirements.txt 62B
images
datasetsupport.png 384KB
webcam_detr.png 393KB
detr-figure.png 322KB
hardhatdataset.jpg 144KB
wandb_logging.png 518KB
tutorials
data-pipeline.png 22KB
download_hardhat_dataset.png 76KB
webcam_inference.py 2KB
README.md 6KB
notebooks
DETR Tensorflow - Finetuning tutorial.ipynb 397KB
DETR Tensorflow - How to setup a custom dataset.ipynb 665KB
How to load a dataset.ipynb 485KB
共 40 条
- 1
资源评论
__AtYou__
- 粉丝: 1727
- 资源: 572
下载权益
C知道特权
VIP文章
课程特权
开通VIP
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功