# Object Detection With A TensorFlow SSD Network
**Table Of Contents**
- [Description](#description)
- [How does this sample work?](#how-does-this-sample-work)
* [Processing the input graph](#processing-the-input-graph)
* [Data preparation](#data-preparation)
* [sampleUffSSD plugins](#sampleuffssd-plugins)
* [Verifying the output](#verifying-the-output)
* [TensorRT API layers and ops](#tensorrt-api-layers-and-ops)
- [Preparing sample data](#preparing-sample-data)
- [Running the sample](#running-the-sample)
* [Sample `--help` options](#sample-help-options)
- [Additional resources](#additional-resources)
- [License](#license)
- [Changelog](#changelog)
- [Known issues](#known-issues)
## Description
This sample, sampleUffSSD, preprocesses a TensorFlow SSD network, performs inference on the SSD network in TensorRT, using TensorRT plugins to speed up inference.
This sample is based on the [SSD: Single Shot MultiBox Detector](https://arxiv.org/abs/1512.02325) paper. The SSD network performs the task of object detection and localization in a single forward pass of the network.
The SSD network used in this sample is based on the TensorFlow implementation of SSD, which actually differs from the original paper, in that it has an inception_v2 backbone. For more information about the actual model, download [ssd_inception_v2_coco](http://download.tensorflow.org/models/object_detection/ssd_inception_v2_coco_2017_11_17.tar.gz). The TensorFlow SSD network was trained on the InceptionV2 architecture using the [MSCOCO dataset](http://cocodataset.org/#home) which has 91 classes (including the background class). The config details of the network can be found [here](https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_inception_v2_coco.config).
## How does this sample work?
The SSD network performs the task of object detection and localization in a single forward pass of the network. The TensorFlow SSD network was trained on the InceptionV2 architecture using the MSCOCO dataset.
The sample makes use of TensorRT plugins to run the SSD network. To use these plugins, the TensorFlow graph needs to be preprocessed, and we use the GraphSurgeon utility to do this.
The main components of this network are the Image Preprocessor, FeatureExtractor, BoxPredictor, GridAnchorGenerator and Postprocessor.
**Image Preprocessor**
The image preprocessor step of the graph is responsible for resizing the image. The image is resized to a 300x300x3 size tensor. This step also performs normalization of the image so all pixel values lie between the range [-1, 1].
**FeatureExtractor**
The FeatureExtractor portion of the graph runs the InceptionV2 network on the preprocessed image. The feature maps generated are used by the anchor generation step to generate default bounding boxes for each feature map.
In this network, the size of feature maps that are used for anchor generation are [(19x19), (10x10), (5x5), (3x3), (2x2), (1x1)].
**BoxPredictor**
The BoxPredictor step takes in a high level feature map as input and produces a list of box encodings (x-y coordinates) and a list of class scores for each of these encodings per feature map. This information is passed to the postprocessor.
**GridAnchorGenerator**
The goal of this step is to generate a set of default bounding boxes (given the scale and aspect ratios mentioned in the config) for each feature map cell. This is implemented as a plugin layer in TensorRT called the `gridAnchorGenerator` plugin. The registered plugin name is `GridAnchor_TRT`.
**Postprocessor**
The postprocessor step performs the final steps to generate the network output. The bounding box data and confidence scores for all feature maps are fed to the step along with the pre-computed default bounding boxes (generated in the `GridAnchorGenerator` namespace). It then performs NMS (non-maximum suppression) which prunes away most of the bounding boxes based on a confidence threshold and IoU (Intersection over Union) overlap, thus storing only the top `N` boxes per class. This is implemented as a plugin layer in TensorRT called the NMS plugin. The registered plugin name is `NMS_TRT`.
**Note:** This sample also implements another plugin called `FlattenConcat` which is used to flatten each input and then concatenate the results. This is applied to the location and confidence data before it is fed to the post processor step since the NMS plugin requires the data to be in this format.
For details on how a plugin is implemented, see the implementation of `FlattenConcat` plugin and `FlattenConcatPluginCreator` in the `sampleUffSSD.cpp` file in the `tensorrt/samples/sampleUffSSD` directory.
Specifically, this sample performs the following steps:
- [Processing the input graph](#processing-the-input-graph)
- [Preparing the data](#preparing-the-data)
- [sampleUffSSD plugins](#sampleuffssd-plugins)
- [Verifying the output](#verifying-the-output)
### Processing the input graph
The TensorFlow SSD graph has some operations that are currently not supported in TensorRT. Using a preprocessor on the graph, we can combine multiple operations in the graph into a single custom operation which can be implemented as a plugin layer in TensorRT. Currently, the preprocessor provides the ability to stitch all nodes within a namespace into one custom node.
To use the preprocessor, the `convert-to-uff` utility should be called with a `-p` flag and a config file. The config script should also include attributes for all custom plugins which will be embedded in the generated `.uff` file. Current sample script for SSD is located in `/usr/src/tensorrt/samples/sampleUffSSD/config.py`.
Using the preprocessor on the graph, we were able to remove the `Preprocessor` namespace from the graph, stitch the `GridAnchorGenerator` namespace together to create the `GridAnchorGenerator` plugin, stitch the `postprocessor` namespace together to get the NMS plugin and mark the concat operations in the BoxPredictor as `FlattenConcat` plugins.
The TensorFlow graph has some operations like `Assert` and `Identity` which can be removed for the inferencing. Operations like `Assert` are removed and leftover nodes (with no outputs once assert is deleted) are then recursively removed.
`Identity` operations are deleted and the input is forwarded to all the connected outputs. Additional documentation on the graph preprocessor can be found in the [TensorRT API](https://docs.nvidia.com/deeplearning/sdk/tensorrt-api/python_api/graphsurgeon/graphsurgeon.html).
### Data preparation
The generated network has an input node called `Input`, and the output node is given the name `MarkOutput_0` by the UFF converter. These nodes are registered by the UFF Parser in the sample.
```
parser->registerInput("Input", Dims3(3, 300, 300),
UffInputOrder::kNCHW);
parser->registerOutput("MarkOutput_0");
```
The input to the SSD network in this sample is 3 channel 300x300 images. In the sample, we normalize the image so the pixel values lie in the range [-1,1]. This is equivalent to the image preprocessing stage of the network.
Since TensorRT does not depend on any computer vision libraries, the images are represented in binary `R`, `G`, and `B` values for each pixel. The format is Portable PixMap (PPM), which is a netpbm color image format. In this format, the `R`, `G`, and `B` values for each pixel are represented by a byte of integer (0-255) and they are stored together, pixel by pixel.
There is a simple PPM reading function called `readPPMFile`.
### sampleUffSSD plugins
Details about how to create TensorRT plugins can be found in [Extending TensorRT With Custom Layers](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#extending).
The `config.py` defined for the `convert-to-uff` command should have the custom layers mapped to the plugin names in TensorRT by
没有合适的资源?快使用搜索试试~ 我知道了~
资源详情
资源评论
资源推荐
收起资源包目录
TensorRT-8.2.1.8.Windows10.x86_64.cuda-11.4.cudnn8.2.zip (489个子文件)
batch_calibration46.batch 1.03MB
batch_calibration6.batch 1.03MB
batch_calibration23.batch 1.03MB
batch_calibration21.batch 1.03MB
batch_calibration48.batch 1.03MB
batch_calibration45.batch 1.03MB
batch_calibration10.batch 1.03MB
batch_calibration41.batch 1.03MB
batch_calibration30.batch 1.03MB
batch_calibration31.batch 1.03MB
batch_calibration8.batch 1.03MB
batch_calibration47.batch 1.03MB
batch_calibration34.batch 1.03MB
batch_calibration24.batch 1.03MB
batch_calibration27.batch 1.03MB
batch_calibration1.batch 1.03MB
batch_calibration43.batch 1.03MB
batch_calibration11.batch 1.03MB
batch_calibration7.batch 1.03MB
batch_calibration20.batch 1.03MB
batch_calibration42.batch 1.03MB
batch_calibration39.batch 1.03MB
batch_calibration26.batch 1.03MB
batch_calibration4.batch 1.03MB
batch_calibration2.batch 1.03MB
batch_calibration35.batch 1.03MB
batch_calibration49.batch 1.03MB
batch_calibration15.batch 1.03MB
batch_calibration16.batch 1.03MB
batch_calibration3.batch 1.03MB
batch_calibration14.batch 1.03MB
batch_calibration22.batch 1.03MB
batch_calibration44.batch 1.03MB
batch_calibration28.batch 1.03MB
batch_calibration13.batch 1.03MB
batch_calibration32.batch 1.03MB
batch_calibration17.batch 1.03MB
batch_calibration12.batch 1.03MB
batch_calibration19.batch 1.03MB
batch_calibration29.batch 1.03MB
batch_calibration5.batch 1.03MB
batch_calibration0.batch 1.03MB
batch_calibration33.batch 1.03MB
batch_calibration25.batch 1.03MB
batch_calibration40.batch 1.03MB
batch_calibration38.batch 1.03MB
batch_calibration9.batch 1.03MB
batch_calibration37.batch 1.03MB
batch_calibration36.batch 1.03MB
batch_calibration18.batch 1.03MB
mnist_mean.binaryproto 3KB
getopt.c 18KB
ResNet50_fp32.caffemodel 97.72MB
googlenet.caffemodel 51.05MB
mnist.caffemodel 1.65MB
mnist_lenet.caffemodel 1.65MB
checkpoint 247B
sampleOptions.cpp 72KB
sampleNMT.cpp 61KB
sampleEngines.cpp 52KB
sampleCharRNN.cpp 44KB
sampleINT8API.cpp 33KB
sampleInference.cpp 33KB
sampleUffFasterRCNN.cpp 28KB
sampleAlgorithmSelector.cpp 27KB
sampleUffPluginV2Ext.cpp 26KB
sampleFasterRCNN.cpp 26KB
sampleIOFormats.cpp 24KB
sampleUffMaskRCNN.cpp 24KB
sampleDynamicReshape.cpp 22KB
sampleINT8.cpp 19KB
sampleMNISTAPI.cpp 18KB
sampleReporting.cpp 18KB
sampleUffSSD.cpp 17KB
sampleSSD.cpp 16KB
sampleMNIST.cpp 15KB
sampleUffMNIST.cpp 13KB
sampleOnnxMNIST.cpp 12KB
trtexec.cpp 11KB
sampleGoogleNet.cpp 10KB
getOptions.cpp 8KB
bleuScoreWriter.cpp 7KB
beamSearchPolicy.cpp 7KB
customClipPlugin.cpp 7KB
lstmDecoder.cpp 6KB
lstmEncoder.cpp 5KB
softmaxLikelihood.cpp 4KB
multiplicativeAlignment.cpp 3KB
slpAttention.cpp 3KB
slpProjection.cpp 3KB
slpEmbedder.cpp 3KB
vocabulary.cpp 3KB
textReader.cpp 2KB
benchmarkWriter.cpp 2KB
dataWriter.cpp 2KB
limitedSamplesDataReader.cpp 2KB
componentWeights.cpp 2KB
contextNMT.cpp 2KB
trtUtil.cpp 2KB
textWriter.cpp 1KB
共 489 条
- 1
- 2
- 3
- 4
- 5
FL1623863129
- 粉丝: 9063
- 资源: 1万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0