# Object Detection With A TensorFlow SSD Network
**Table Of Contents**
- [Description](#description)
- [How does this sample work?](#how-does-this-sample-work)
* [Processing the input graph](#processing-the-input-graph)
* [Preparing the data](#preparing-the-data)
* [sampleUffSSD plugins](#sampleuffssd-plugins)
* [Verifying the output](#verifying-the-output)
* [TensorRT API layers and ops](#tensorrt-api-layers-and-ops)
- [Prerequisites](#prerequisites)
- [Running the sample](#running-the-sample)
* [Sample `--help` options](#sample-help-options)
- [Additional resources](#additional-resources)
- [License](#license)
- [Changelog](#changelog)
- [Known issues](#known-issues)
## Description
This sample, sampleUffSSD, preprocesses a TensorFlow SSD network, performs inference on the SSD network in TensorRT, using TensorRT plugins to speed up inference.
This sample is based on the [SSD: Single Shot MultiBox Detector](https://arxiv.org/abs/1512.02325) paper. The SSD network performs the task of object detection and localization in a single forward pass of the network.
The SSD network used in this sample is based on the TensorFlow implementation of SSD, which actually differs from the original paper, in that it has an inception_v2 backbone. For more information about the actual model, download [ssd_inception_v2_coco](http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2017_11_17.tar.gz). The TensorFlow SSD network was trained on the InceptionV2 architecture using the [MSCOCO dataset](http://cocodataset.org/#home) which has 91 classes (including the background class). The config details of the network can be found [here](https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_inception_v2_coco.config).
## How does this sample work?
The SSD network performs the task of object detection and localization in a single forward pass of the network. The TensorFlow SSD network was trained on the InceptionV2 architecture using the MSCOCO dataset.
The sample makes use of TensorRT plugins to run the SSD network. To use these plugins, the TensorFlow graph needs to be preprocessed, and we use the GraphSurgeon utility to do this.
The main components of this network are the Image Preprocessor, FeatureExtractor, BoxPredictor, GridAnchorGenerator and Postprocessor.
**Image Preprocessor**
The image preprocessor step of the graph is responsible for resizing the image. The image is resized to a 300x300x3 size tensor. This step also performs normalization of the image so all pixel values lie between the range [-1, 1].
**FeatureExtractor**
The FeatureExtractor portion of the graph runs the InceptionV2 network on the preprocessed image. The feature maps generated are used by the anchor generation step to generate default bounding boxes for each feature map.
In this network, the size of feature maps that are used for anchor generation are [(19x19), (10x10), (5x5), (3x3), (2x2), (1x1)].
**BoxPredictor**
The BoxPredictor step takes in a high level feature map as input and produces a list of box encodings (x-y coordinates) and a list of class scores for each of these encodings per feature map. This information is passed to the postprocessor.
**GridAnchorGenerator**
The goal of this step is to generate a set of default bounding boxes (given the scale and aspect ratios mentioned in the config) for each feature map cell. This is implemented as a plugin layer in TensorRT called the `gridAnchorGenerator` plugin. The registered plugin name is `GridAnchor_TRT`.
**Postprocessor**
The postprocessor step performs the final steps to generate the network output. The bounding box data and confidence scores for all feature maps are fed to the step along with the pre-computed default bounding boxes (generated in the `GridAnchorGenerator` namespace). It then performs NMS (non-maximum suppression) which prunes away most of the bounding boxes based on a confidence threshold and IoU (Intersection over Union) overlap, thus storing only the top `N` boxes per class. This is implemented as a plugin layer in TensorRT called the NMS plugin. The registered plugin name is `NMS_TRT`.
**Note:** This sample also implements another plugin called `FlattenConcat` which is used to flatten each input and then concatenate the results. This is applied to the location and confidence data before it is fed to the post processor step since the NMS plugin requires the data to be in this format.
For details on how a plugin is implemented, see the implementation of `FlattenConcat` plugin and `FlattenConcatPluginCreator` in the `sampleUffSSD.cpp` file in the `tensorrt/samples/sampleUffSSD` directory.
Specifically, this sample performs the following steps:
- [Processing the input graph](#processing-the-input-graph)
- [Preparing the data](#preparing-the-data)
- [sampleUffSSD plugins](#sampleuffssd-plugins)
- [Verifying the output](#verifying-the-output)
### Processing the input graph
The TensorFlow SSD graph has some operations that are currently not supported in TensorRT. Using a preprocessor on the graph, we can combine multiple operations in the graph into a single custom operation which can be implemented as a plugin layer in TensorRT. Currently, the preprocessor provides the ability to stitch all nodes within a namespace into one custom node.
To use the preprocessor, the `convert-to-uff` utility should be called with a `-p` flag and a config file. The config script should also include attributes for all custom plugins which will be embedded in the generated `.uff` file. Current sample script for SSD is located in `/usr/src/tensorrt/samples/sampleUffSSD/config.py`.
Using the preprocessor on the graph, we were able to remove the `Preprocessor` namespace from the graph, stitch the `GridAnchorGenerator` namespace together to create the `GridAnchorGenerator` plugin, stitch the `postprocessor` namespace together to get the NMS plugin and mark the concat operations in the BoxPredictor as `FlattenConcat` plugins.
The TensorFlow graph has some operations like `Assert` and `Identity` which can be removed for the inferencing. Operations like `Assert` are removed and leftover nodes (with no outputs once assert is deleted) are then recursively removed.
`Identity` operations are deleted and the input is forwarded to all the connected outputs. Additional documentation on the graph preprocessor can be found in the [TensorRT API](https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/graphsurgeon/graphsurgeon.html).
### Preparing the data
The generated network has an input node called `Input`, and the output node is given the name `MarkOutput_0` by the UFF converter. These nodes are registered by the UFF Parser in the sample.
```
parser->registerInput("Input", DimsCHW(3, 300, 300),
UffInputOrder::kNCHW);
parser->registerOutput("MarkOutput_0");
```
The input to the SSD network in this sample is 3 channel 300x300 images. In the sample, we normalize the image so the pixel values lie in the range [-1,1]. This is equivalent to the image preprocessing stage of the network.
Since TensorRT does not depend on any computer vision libraries, the images are represented in binary `R`, `G`, and `B` values for each pixel. The format is Portable PixMap (PPM), which is a netpbm color image format. In this format, the `R`, `G`, and `B` values for each pixel are represented by a byte of integer (0-255) and they are stored together, pixel by pixel.
There is a simple PPM reading function called `readPPMFile`.
### sampleUffSSD plugins
Details about how to create TensorRT plugins can be found in [Extending TensorRT With Custom Layers](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#extending).
The `config.py` defined for the `convert-to-uff` command should have the custom layers mapped to the plugin names in TensorRT by modifying the `op` field. The names of the plugin parameters should also exactly match those expected
没有合适的资源?快使用搜索试试~ 我知道了~
TensorRT-7.2.3.4.Ubuntu-18.04.x86-64-gnu.cuda-11.1.cudnn8.1
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 166 浏览量
2023-02-08
17:11:21
上传
评论
收藏 973.35MB GZ 举报
温馨提示
共1605个文件
html:461个
js:259个
h:136个
该资源为TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.1.cudnn8.1.tar.gz,欢迎下载使用哦!
资源推荐
资源详情
资源评论
收起资源包目录
TensorRT-7.2.3.4.Ubuntu-18.04.x86-64-gnu.cuda-11.1.cudnn8.1 (1605个子文件)
libmyelin.so.1 6.41MB
libmyelin.so.1 6.41MB
libmyelin.so.1.1.116 6.41MB
libmyelin.so.1.1.116 6.41MB
libnvinfer.so.7.2.3 599.35MB
libnvinfer.so.7.2.3 599.35MB
libnvinfer_plugin.so.7.2.3 16.35MB
libnvinfer_plugin.so.7.2.3 16.35MB
libnvcaffe_parser.so.7.2.3 3.19MB
libnvparsers.so.7.2.3 3.19MB
libnvcaffe_parser.so.7.2.3 3.19MB
libnvparsers.so.7.2.3 3.19MB
libnvonnxparser.so.7.2.3 2.46MB
libnvonnxparser.so.7.2.3 2.46MB
libnvinfer.so.7 599.35MB
libnvinfer.so.7 599.35MB
libnvinfer_plugin.so.7 16.35MB
libnvinfer_plugin.so.7 16.35MB
libnvparsers.so.7 3.19MB
libnvcaffe_parser.so.7 3.19MB
libnvparsers.so.7 3.19MB
libnvcaffe_parser.so.7 3.19MB
libnvonnxparser.so.7 2.46MB
libnvonnxparser.so.7 2.46MB
libnvinfer_static.a 741.39MB
libnvinfer_static.a 741.39MB
libnvinfer_plugin_static.a 18.37MB
libnvinfer_plugin_static.a 18.37MB
libprotobuf.a 15.22MB
libprotobuf.a 15.22MB
libmyelin_compiler_static.a 11.44MB
libmyelin_compiler_static.a 11.44MB
libmyelin_executor_static.a 8.35MB
libmyelin_executor_static.a 8.35MB
libnvcaffe_parser.a 4.18MB
libnvparsers_static.a 4.18MB
libnvcaffe_parser.a 4.18MB
libnvparsers_static.a 4.18MB
libnvonnxparser_static.a 1.4MB
libnvonnxparser_static.a 1.4MB
libprotobuf-lite.a 1.28MB
libprotobuf-lite.a 1.28MB
libonnx_proto.a 567KB
libonnx_proto.a 567KB
libmyelin_pattern_library_static.a 331KB
libmyelin_pattern_library_static.a 331KB
libmyelin_pattern_runtime_static.a 245KB
libmyelin_pattern_runtime_static.a 245KB
libnvrtc_static.a 3KB
libnvrtc_static.a 3KB
batch_calibration34.batch 1.03MB
batch_calibration7.batch 1.03MB
batch_calibration0.batch 1.03MB
batch_calibration1.batch 1.03MB
batch_calibration5.batch 1.03MB
batch_calibration41.batch 1.03MB
batch_calibration32.batch 1.03MB
batch_calibration29.batch 1.03MB
batch_calibration43.batch 1.03MB
batch_calibration8.batch 1.03MB
batch_calibration4.batch 1.03MB
batch_calibration25.batch 1.03MB
batch_calibration18.batch 1.03MB
batch_calibration17.batch 1.03MB
batch_calibration30.batch 1.03MB
batch_calibration22.batch 1.03MB
batch_calibration39.batch 1.03MB
batch_calibration11.batch 1.03MB
batch_calibration6.batch 1.03MB
batch_calibration21.batch 1.03MB
batch_calibration15.batch 1.03MB
batch_calibration3.batch 1.03MB
batch_calibration10.batch 1.03MB
batch_calibration44.batch 1.03MB
batch_calibration45.batch 1.03MB
batch_calibration16.batch 1.03MB
batch_calibration13.batch 1.03MB
batch_calibration40.batch 1.03MB
batch_calibration42.batch 1.03MB
batch_calibration28.batch 1.03MB
batch_calibration49.batch 1.03MB
batch_calibration36.batch 1.03MB
batch_calibration24.batch 1.03MB
batch_calibration46.batch 1.03MB
batch_calibration35.batch 1.03MB
batch_calibration38.batch 1.03MB
batch_calibration2.batch 1.03MB
batch_calibration33.batch 1.03MB
batch_calibration12.batch 1.03MB
batch_calibration31.batch 1.03MB
batch_calibration26.batch 1.03MB
batch_calibration23.batch 1.03MB
batch_calibration20.batch 1.03MB
batch_calibration47.batch 1.03MB
batch_calibration37.batch 1.03MB
batch_calibration27.batch 1.03MB
batch_calibration14.batch 1.03MB
batch_calibration19.batch 1.03MB
batch_calibration48.batch 1.03MB
batch_calibration9.batch 1.03MB
共 1605 条
- 1
- 2
- 3
- 4
- 5
- 6
- 17
资源评论
假技术po主
- 粉丝: 515
- 资源: 4209
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- pta题库答案c语言之复杂度1最大子列和问题.zip
- 三维装箱问题(Three-Dimensional Bin Packing Problem,3D-BPP)是一个经典的组合优化问题
- 以下是一些关于Linux线程同步的基本概念和方法.txt
- 以下是一个简化的示例,它使用pygame库来模拟烟花动画的框架.txt
- Linux线程同步机制深度解析与实用指南.zip
- PTA题库C语言解题策略与实战.rar
- SVPWM控制技术的simulink建模与仿真【包括simulink模型,参考文献,操作步骤】
- AI高清修复图片画质易语言易语言源码易语言填表
- 映射窗口.ec易语言易语言模块CPU占用0%游戏监控窗口监控
- 易语言 361窗口模块高效、便捷、自封装、自用
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功