# “Hello World” For TensorRT From ONNX
**Table Of Contents**
- [Description](#description)
- [How does this sample work?](#how-does-this-sample-work)
* [Converting the ONNX model to a TensorRT network](#converting-the-onnx-model-to-a-tensorrt-network)
* [Building the engine](#building-the-engine)
* [Running inference](#running-inference)
* [TensorRT API layers and ops](#tensorrt-api-layers-and-ops)
- [Running the sample](#running-the-sample)
* [Sample `--help` options](#sample---help-options)
- [Additional resources](#additional-resources)
- [License](#license)
- [Changelog](#changelog)
- [Known issues](#known-issues)
## Description
This sample, sampleOnnxMNIST, converts a model trained on the [MNIST dataset](https://github.com/onnx/models/tree/master/mnist) in Open Neural Network Exchange (ONNX) format to a TensorRT network and runs inference on the network.
ONNX is a standard for representing deep learning models that enables models to be transferred between frameworks.
## How does this sample work?
This sample creates and runs the TensorRT engine from an ONNX model of the MNIST network. It demonstrates how TensorRT can consume an ONNX model as input to create a network.
Specifically, this sample:
- [Converts the ONNX model to a TensorRT network](#converting-the-onnx-model-to-a-tensorrt-network)
- [Builds an engine](#building-an-engine)
- [Runs inference using the generated TensorRT network](#running-inference)
### Converting the ONNX model to a TensorRT network
The model file can be converted to a TensorRT network using the ONNX parser. The parser can be initialized with the
network definition that the parser will write to and the logger object.
`auto parser = nvonnxparser::createParser(*network, gLogger.getTRTLogger());`
The ONNX model file is then passed onto the parser along with the logging level
```
if (!parser->parseFromFile(model_file, static_cast<int>(gLogger.getReportableSeverity())))
{
string msg("failed to parse onnx file");
gLogger->log(nvinfer1::ILogger::Severity::kERROR, msg.c_str());
exit(EXIT_FAILURE);
}
```
To view additional information about the network, including layer information and individual layer dimensions, issue the following call:
```
parser->reportParsingInfo();
```
After the TensorRT network is constructed by parsing the model, the TensorRT engine can be built to run inference.
### Building the engine
To build the engine, create the builder and pass a logger created for TensorRT which is used for reporting errors, warnings and informational messages in the network:
`IBuilder* builder = createInferBuilder(gLogger);`
To build the engine from the generated TensorRT network, issue the following call:
`nvinfer1::ICudaEngine* engine = builder->buildCudaEngine(*network);`
After you build the engine, verify that the engine is running properly by confirming the output is what you expected. The output format of this sample should be the same as the output of sampleMNIST.
### Running inference
To run inference using the created engine, see [Performing Inference In C++](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#perform_inference_c).
**Note:** It’s important to preprocess the data and convert it to the format accepted by the network. In this example, the sample input is in PGM (portable graymap) format. The model expects an input of image `1x28x28` scaled to between `[0,1]`.
### TensorRT API layers and ops
In this sample, the following layers are used. For more information about these layers, see the [TensorRT Developer Guide: Layers](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#layers) documentation.
[Activation layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#activation-layer)
The Activation layer implements element-wise activation functions. Specifically, this sample uses the Activation layer with the type `kRELU`.
[Convolution layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#convolution-layer)
The Convolution layer computes a 2D (channel, height, and width) convolution, with or without bias.
[FullyConnected layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#fullyconnected-layer)
The FullyConnected layer implements a matrix-vector product, with or without bias.
[Pooling layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#pooling-layer)
The Pooling layer implements pooling within a channel. Supported pooling types are `maximum`, `average` and `maximum-average blend`.
[Scale layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#scale-layer)
The Scale layer implements a per-tensor, per-channel, or per-element affine transformation and/or exponentiation by constant values.
[Shuffle layer](https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#shuffle-layer)
The Shuffle layer implements a reshape and transpose operator for tensors.
## Running the sample
1. Compile this sample by running `make` in the `<TensorRT root directory>/samples/sampleOnnxMNIST` directory. The binary named `sample_onnx_mnist` will be created in the `<TensorRT root directory>/bin` directory.
```
cd <TensorRT root directory>/samples/sampleOnnxMNIST
make
```
Where `<TensorRT root directory>` is where you installed TensorRT.
2. Run the sample to build and run the MNIST engine from the ONNX model.
```
./sample_onnx_mnist [-h or --help] [-d or --datadir=<path to data directory>] [--useDLACore=<int>] [--int8 or --fp16]
```
3. Verify that the sample ran successfully. If the sample runs successfully you should see output similar to the following:
```
&&&& RUNNING TensorRT.sample_onnx_mnist # ./sample_onnx_mnist
----------------------------------------------------------------
Input filename: ../../../../../../data/samples/mnist/mnist.onnx
ONNX IR version: 0.0.3
Opset version: 1
Producer name: CNTK
Producer version: 2.4
Domain:
Model version: 1
Doc string:
----------------------------------------------------------------
[I] Input:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@*. .*@@@@@@@@@@@
@@@@@@@@@@*. +@@@@@@@@@@
@@@@@@@@@@. :#+ %@@@@@@@@@
@@@@@@@@@@.:@@@+ +@@@@@@@@@
@@@@@@@@@@.:@@@@: +@@@@@@@@
@@@@@@@@@@=%@@@@: +@@@@@@@@
@@@@@@@@@@@@@@@@# +@@@@@@@@
@@@@@@@@@@@@@@@@* +@@@@@@@@
@@@@@@@@@@@@@@@@: +@@@@@@@@
@@@@@@@@@@@@@@@@: +@@@@@@@@
@@@@@@@@@@@@@@@* .@@@@@@@@@
@@@@@@@@@@%**%@. *@@@@@@@@@
@@@@@@@@%+. .: .@@@@@@@@@@
@@@@@@@@= .. :@@@@@@@@@@
@@@@@@@@: *@@: :@@@@@@@@@@
@@@@@@@% %@* *@@@@@@@@@
@@@@@@@% ++ ++ .%@@@@@@@@
@@@@@@@@- +@@- +@@@@@@@@
@@@@@@@@= :*@@@# .%@@@@@@@
@@@@@@@@@+*@@@@@%. %@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[I] Output:
Prob 0 0.0000 Class 0:
Prob 1 0.0000 Class 1:
Prob 2 1.0000 Class 2: **********
Prob 3 0.0000 Class 3:
Prob 4 0.0000 Class 4:
Prob 5 0.0000 Class 5:
Prob 6 0.0000 Class 6:
Prob 7 0.0000 Class 7:
Prob 8 0.0000 Class 8:
Prob 9 0.0000 Class 9:
&&&& PASSED TensorRT.sample_onnx_mnist # ./sample_onnx_mnist
```
This output shows that the sample ran successfully; PASSED.
### Sample --help options
To see the full list of available options and their descriptions, use the `-h` or `--help` command line option. For example:
```
Usage: ./sample_onnx_mnist [-h or --help] [-d or --datadir=<path to data directory>] [--useDLACore=<int>]
--help Display help information
--datadir Specify path to a data directory, overriding the default. This option can be used multiple times to add multiple directories. If no data directories are given, the default is to use (data/samples/mnist/, data/mnist/)
--useDLACore=N Specify a DLA engine for layers that support DLA. Value can range from 0 to
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
算法部署_使用TensorRT部署SGDN算法_优质算法部署项目实战.zip (8个子文件)
算法部署_使用TensorRT部署SGDN算法_优质算法部署项目实战
sampleOnnxSGDN
Makefile 157B
data
img.txt 1.13MB
sampleOnnxSGDN.cpp 13KB
README.md 9KB
create_txt
_23_rgb.png 226KB
img.txt 1.13MB
create_txt.py 856B
README.md 7KB
共 8 条
- 1
资源评论
极智视界
- 粉丝: 2w+
- 资源: 1468
下载权益
C知道特权
VIP文章
课程特权
开通VIP
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功