# DETR tensorRT 部署
detr_onnx:测试图像、测试结果、测试demo脚本
detr_tensorrt:测试图像、测试结果、测试tensorrt脚本、onnx2tensorRT脚本(tensorRT-7.2.3.4)
由于模型较大无法直接上传,onnx和tensorrt [模型文件](https://github.com/cqu20160901/DETR_onnx_tensorRT/releases)。
说明:
(1)本示例提供的模型只检测行人,由于训练的时类别写成了3,因此模型输出结果只有第二类是有效的。
(2)本示例不涉及模型训练,训练自己数据可以参考网上教程。我第一次训练没有使用预训练权重,导致模型不收敛最终的AP全为0;第二次加载预训练模型才收敛,加载预训练权重参考网上提供的将模型输出适配成自己的类别。
(3)解决转tensorrt 输出全为 0 的问题。
解决转tensorrt 输出全为 0 问题的[终极方法](https://github.com/cqu20160901/DETR_onnx_tensorRT_V2)
## detr C++ 部署
detr C++ 部署[参考链接](https://github.com/cqu20160901/DETR_tensorRT_Cplusplus)。
## 转 tensorrt 可能会遇到的问题
(1)导出onnx后转tensorrt 加载不了,建议用onnxsim处理一下。
(2)导出的tensorrt推理输出全为0,这个问题让我费解很久,网上查到也有遇到这个问题的但没有给出解决方案,几度想过放弃。
## tensorrt 推理输出全为 0
(1)修改onnx模型输出层Gather的参数(在网上看到的修改方法):
```python
graph = gs.import_onnx(onnx.load("./detr_r50_person_sim.onnx"))
for node in graph.nodes:
# print(node)
if node.name == "Gather_2711":
print(node)
print(node.inputs[1])
node.inputs[1].values = np.int64(5)
print(node.inputs[1])
if node.name == "Gather_2713":
print(node)
print(node.inputs[1])
node.inputs[1].values = np.int64(5)
print(node.inputs[1])
onnx.save(gs.export_onnx(graph), 'detr_r50_person_sim_change.onnx')
```
按照上述修改输出结果还全是0,这下让人崩溃了。
(2)解决输出全为0的问题,转 tensorrt 不使用任何量化,使用 fp32_mode 模式
代码里tensorrt 默认量化是 fp16_mode,将量化方式注释掉输出结果正常。
```python
def get_engine(onnx_model_name, trt_model_name):
explicit_batch = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH)
with trt.Builder(G_LOGGER) as builder, builder.create_network(explicit_batch) as network, trt.OnnxParser(network,
G_LOGGER) as parser:
builder.max_batch_size = batch_size
builder.max_workspace_size = 2 << 30
print('Loading ONNX file from path {}...'.format(onnx_model_name))
with open(onnx_model_name, 'rb') as model:
print('Beginning ONNX file parsing')
if not parser.parse(model.read()):
for error in range(parser.num_errors):
print(parser.get_error(error))
print('Completed parsing of ONNX file')
print('Building an engine from file {}; this may take a while...'.format(onnx_model_name))
####
# builder.int8_mode = True
# builder.int8_calibrator = calib
# builder.fp16_mode = True
####
print("num layers:", network.num_layers)
# last_layer = network.get_layer(network.num_layers - 1)
# if not last_layer.get_output(0):
# network.mark_output(network.get_layer(network.num_layers - 1).get_output(0))//有的模型需要,有的模型在转onnx的之后已经指定了,就不需要这行
network.get_input(0).shape = [batch_size, 3, imput_h, imput_w]
engine = builder.build_cuda_engine(network)
print("engine:", engine)
print("Completed creating Engine")
with open(trt_model_name, "wb") as f:
f.write(engine.serialize())
return engine
```
onnx 测试结果
![image](https://github.com/cqu20160901/DETR_onnx_tensorRT/blob/main/detr_onnx/test_onnx_result.jpg)
tensorrt 测试结果
![image](https://github.com/cqu20160901/DETR_onnx_tensorRT/blob/main/detr_tensorrt/test_result_tensorRT.jpg)
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![pdf](https://img-home.csdnimg.cn/images/20210720083512.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![py](https://img-home.csdnimg.cn/images/20210720083646.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
![rar](https://img-home.csdnimg.cn/images/20210720083606.png)
![zip](https://img-home.csdnimg.cn/images/20210720083736.png)
收起资源包目录
![package](https://csdnimg.cn/release/downloadcmsfe/public/img/package.f3fc750b.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/JPG.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/JPG.png)
![folder](https://csdnimg.cn/release/downloadcmsfe/public/img/folder.005fa2e5.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/JPG.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/JPG.png)
![file-type](https://csdnimg.cn/release/download/static_files/pc/images/minetype/UNKNOWN.png)
共 8 条
- 1
资源评论
![avatar-default](https://csdnimg.cn/release/downloadcmsfe/public/img/lazyLogo2.1882d7f4.png)
![avatar](https://profile-avatar.csdnimg.cn/30cca166a8ba4bd9b70a7e3128b56485_lwx666sl.jpg!1)
熬夜写代码的平头哥
- 粉丝: 3788
- 资源: 7360
上传资源 快速赚钱
我的内容管理 展开
我的资源 快来上传第一个资源
我的收益
登录查看自己的收益我的积分 登录查看自己的积分
我的C币 登录后查看C币余额
我的收藏
我的下载
下载帮助
![voice](https://csdnimg.cn/release/downloadcmsfe/public/img/voice.245cc511.png)
![center-task](https://csdnimg.cn/release/downloadcmsfe/public/img/center-task.c2eda91a.png)
安全验证
文档复制为VIP权益,开通VIP直接复制
![dialog-icon](https://csdnimg.cn/release/downloadcmsfe/public/img/green-success.6a4acb44.png)