# Object Detection With A TensorFlow SSD Network
**Table Of Contents**
- [Description](#description)
- [How does this sample work?](#how-does-this-sample-work)
* [Processing the input graph](#processing-the-input-graph)
* [Preparing the data](#preparing-the-data)
* [sampleUffSSD plugins](#sampleuffssd-plugins)
* [Verifying the output](#verifying-the-output)
* [TensorRT API layers and ops](#tensorrt-api-layers-and-ops)
- [Prerequisites](#prerequisites)
- [Running the sample](#running-the-sample)
* [Sample `--help` options](#sample-help-options)
- [Additional resources](#additional-resources)
- [License](#license)
- [Changelog](#changelog)
- [Known issues](#known-issues)
## Description
This sample, sampleUffSSD, preprocesses a TensorFlow SSD network, performs inference on the SSD network in TensorRT, using TensorRT plugins to speed up inference.
This sample is based on the [SSD: Single Shot MultiBox Detector](https://arxiv.org/abs/1512.02325) paper. The SSD network performs the task of object detection and localization in a single forward pass of the network.
The SSD network used in this sample is based on the TensorFlow implementation of SSD, which actually differs from the original paper, in that it has an inception_v2 backbone. For more information about the actual model, download [ssd_inception_v2_coco](http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2017_11_17.tar.gz). The TensorFlow SSD network was trained on the InceptionV2 architecture using the [MSCOCO dataset](http://cocodataset.org/#home) which has 91 classes (including the background class). The config details of the network can be found [here](https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_inception_v2_coco.config).
## How does this sample work?
The SSD network performs the task of object detection and localization in a single forward pass of the network. The TensorFlow SSD network was trained on the InceptionV2 architecture using the MSCOCO dataset.
The sample makes use of TensorRT plugins to run the SSD network. To use these plugins, the TensorFlow graph needs to be preprocessed, and we use the GraphSurgeon utility to do this.
The main components of this network are the Image Preprocessor, FeatureExtractor, BoxPredictor, GridAnchorGenerator and Postprocessor.
**Image Preprocessor**
The image preprocessor step of the graph is responsible for resizing the image. The image is resized to a 300x300x3 size tensor. This step also performs normalization of the image so all pixel values lie between the range [-1, 1].
**FeatureExtractor**
The FeatureExtractor portion of the graph runs the InceptionV2 network on the preprocessed image. The feature maps generated are used by the anchor generation step to generate default bounding boxes for each feature map.
In this network, the size of feature maps that are used for anchor generation are [(19x19), (10x10), (5x5), (3x3), (2x2), (1x1)].
**BoxPredictor**
The BoxPredictor step takes in a high level feature map as input and produces a list of box encodings (x-y coordinates) and a list of class scores for each of these encodings per feature map. This information is passed to the postprocessor.
**GridAnchorGenerator**
The goal of this step is to generate a set of default bounding boxes (given the scale and aspect ratios mentioned in the config) for each feature map cell. This is implemented as a plugin layer in TensorRT called the `gridAnchorGenerator` plugin. The registered plugin name is `GridAnchor_TRT`.
**Postprocessor**
The postprocessor step performs the final steps to generate the network output. The bounding box data and confidence scores for all feature maps are fed to the step along with the pre-computed default bounding boxes (generated in the `GridAnchorGenerator` namespace). It then performs NMS (non-maximum suppression) which prunes away most of the bounding boxes based on a confidence threshold and IoU (Intersection over Union) overlap, thus storing only the top `N` boxes per class. This is implemented as a plugin layer in TensorRT called the NMS plugin. The registered plugin name is `NMS_TRT`.
**Note:** This sample also implements another plugin called `FlattenConcat` which is used to flatten each input and then concatenate the results. This is applied to the location and confidence data before it is fed to the post processor step since the NMS plugin requires the data to be in this format.
For details on how a plugin is implemented, see the implementation of `FlattenConcat` plugin and `FlattenConcatPluginCreator` in the `sampleUffSSD.cpp` file in the `tensorrt/samples/sampleUffSSD` directory.
Specifically, this sample performs the following steps:
- [Processing the input graph](#processing-the-input-graph)
- [Preparing the data](#preparing-the-data)
- [sampleUffSSD plugins](#sampleuffssd-plugins)
- [Verifying the output](#verifying-the-output)
### Processing the input graph
The TensorFlow SSD graph has some operations that are currently not supported in TensorRT. Using a preprocessor on the graph, we can combine multiple operations in the graph into a single custom operation which can be implemented as a plugin layer in TensorRT. Currently, the preprocessor provides the ability to stitch all nodes within a namespace into one custom node.
To use the preprocessor, the `convert-to-uff` utility should be called with a `-p` flag and a config file. The config script should also include attributes for all custom plugins which will be embedded in the generated `.uff` file. Current sample script for SSD is located in `/usr/src/tensorrt/samples/sampleUffSSD/config.py`.
Using the preprocessor on the graph, we were able to remove the `Preprocessor` namespace from the graph, stitch the `GridAnchorGenerator` namespace together to create the `GridAnchorGenerator` plugin, stitch the `postprocessor` namespace together to get the NMS plugin and mark the concat operations in the BoxPredictor as `FlattenConcat` plugins.
The TensorFlow graph has some operations like `Assert` and `Identity` which can be removed for the inferencing. Operations like `Assert` are removed and leftover nodes (with no outputs once assert is deleted) are then recursively removed.
`Identity` operations are deleted and the input is forwarded to all the connected outputs. Additional documentation on the graph preprocessor can be found in the [TensorRT API](https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/graphsurgeon/graphsurgeon.html).
### Preparing the data
The generated network has an input node called `Input`, and the output node is given the name `MarkOutput_0` by the UFF converter. These nodes are registered by the UFF Parser in the sample.
```
parser->registerInput("Input", DimsCHW(3, 300, 300),
UffInputOrder::kNCHW);
parser->registerOutput("MarkOutput_0");
```
The input to the SSD network in this sample is 3 channel 300x300 images. In the sample, we normalize the image so the pixel values lie in the range [-1,1]. This is equivalent to the image preprocessing stage of the network.
Since TensorRT does not depend on any computer vision libraries, the images are represented in binary `R`, `G`, and `B` values for each pixel. The format is Portable PixMap (PPM), which is a netpbm color image format. In this format, the `R`, `G`, and `B` values for each pixel are represented by a byte of integer (0-255) and they are stored together, pixel by pixel.
There is a simple PPM reading function called `readPPMFile`.
### sampleUffSSD plugins
Details about how to create TensorRT plugins can be found in [Extending TensorRT With Custom Layers](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#extending).
The `config.py` defined for the `convert-to-uff` command should have the custom layers mapped to the plugin names in TensorRT by modifying the `op` field. The names of the plugin parameters should also exactly match those expected
没有合适的资源?快使用搜索试试~ 我知道了~
TensorRT-7.1.3.4.Windows10.x86_64.cuda-11.0.cudnn8.0.zip
共364个文件
h:67个
list:50个
batch:50个
需积分: 5 20 下载量 163 浏览量
2021-04-06
15:42:35
上传
评论
收藏 650.53MB ZIP 举报
温馨提示
也可以到官网下载:https://developer.nvidia.com/nvidia-tensorrt-7x-download
资源推荐
资源详情
资源评论
收起资源包目录
TensorRT-7.1.3.4.Windows10.x86_64.cuda-11.0.cudnn8.0.zip (364个子文件)
batch_calibration31.batch 1.03MB
batch_calibration1.batch 1.03MB
batch_calibration46.batch 1.03MB
batch_calibration47.batch 1.03MB
batch_calibration49.batch 1.03MB
batch_calibration34.batch 1.03MB
batch_calibration23.batch 1.03MB
batch_calibration10.batch 1.03MB
batch_calibration39.batch 1.03MB
batch_calibration26.batch 1.03MB
batch_calibration21.batch 1.03MB
batch_calibration41.batch 1.03MB
batch_calibration24.batch 1.03MB
batch_calibration43.batch 1.03MB
batch_calibration45.batch 1.03MB
batch_calibration44.batch 1.03MB
batch_calibration28.batch 1.03MB
batch_calibration3.batch 1.03MB
batch_calibration35.batch 1.03MB
batch_calibration8.batch 1.03MB
batch_calibration6.batch 1.03MB
batch_calibration20.batch 1.03MB
batch_calibration4.batch 1.03MB
batch_calibration15.batch 1.03MB
batch_calibration7.batch 1.03MB
batch_calibration14.batch 1.03MB
batch_calibration27.batch 1.03MB
batch_calibration48.batch 1.03MB
batch_calibration16.batch 1.03MB
batch_calibration42.batch 1.03MB
batch_calibration2.batch 1.03MB
batch_calibration11.batch 1.03MB
batch_calibration30.batch 1.03MB
batch_calibration22.batch 1.03MB
batch_calibration5.batch 1.03MB
batch_calibration17.batch 1.03MB
batch_calibration29.batch 1.03MB
batch_calibration19.batch 1.03MB
batch_calibration40.batch 1.03MB
batch_calibration25.batch 1.03MB
batch_calibration18.batch 1.03MB
batch_calibration36.batch 1.03MB
batch_calibration13.batch 1.03MB
batch_calibration9.batch 1.03MB
batch_calibration0.batch 1.03MB
batch_calibration38.batch 1.03MB
batch_calibration12.batch 1.03MB
batch_calibration32.batch 1.03MB
batch_calibration37.batch 1.03MB
batch_calibration33.batch 1.03MB
mnist_mean.binaryproto 3KB
getopt.c 17KB
googlenet.caffemodel 51.05MB
mnist.caffemodel 1.65MB
mnist_lenet.caffemodel 1.65MB
checkpoint 247B
sampleNMT.cpp 60KB
sampleOptions.cpp 48KB
sampleCharRNN.cpp 42KB
sampleINT8API.cpp 32KB
sampleAlgorithmSelector.cpp 28KB
sampleUffFasterRCNN.cpp 26KB
sampleUffPluginV2Ext.cpp 25KB
sampleEngines.cpp 23KB
sampleMovieLens.cpp 23KB
sampleUffMaskRCNN.cpp 23KB
sampleReformatFreeIO.cpp 22KB
sampleFasterRCNN.cpp 19KB
sampleDynamicReshape.cpp 19KB
sampleMLP.cpp 19KB
sampleINT8.cpp 18KB
sampleMNISTAPI.cpp 16KB
sampleUffSSD.cpp 16KB
sampleInference.cpp 15KB
sampleSSD.cpp 15KB
samplePlugin.cpp 15KB
sampleReporting.cpp 14KB
sampleMNIST.cpp 14KB
sampleUffMNIST.cpp 12KB
sampleOnnxMNIST.cpp 11KB
sampleGoogleNet.cpp 9KB
getOptions.cpp 8KB
beamSearchPolicy.cpp 7KB
bleuScoreWriter.cpp 7KB
lstmDecoder.cpp 6KB
trtexec.cpp 5KB
lstmEncoder.cpp 5KB
softmaxLikelihood.cpp 4KB
debugUtil.cpp 4KB
multiplicativeAlignment.cpp 3KB
slpAttention.cpp 3KB
slpProjection.cpp 3KB
slpEmbedder.cpp 3KB
vocabulary.cpp 2KB
textReader.cpp 2KB
benchmarkWriter.cpp 2KB
dataWriter.cpp 2KB
limitedSamplesDataReader.cpp 2KB
componentWeights.cpp 2KB
contextNMT.cpp 2KB
共 364 条
- 1
- 2
- 3
- 4
资源评论
Stars-Chan
- 粉丝: 33
- 资源: 9
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功