# YOLOv8-TensorRT
`YOLOv8` using TensorRT accelerate !
---
[![Build Status](https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fatrox%2Fsync-dotenv%2Fbadge&style=flat)](https://github.com/triple-Mu/YOLOv8-TensorRT)
[![Python Version](https://img.shields.io/badge/Python-3.8--3.10-FFD43B?logo=python)](https://github.com/triple-Mu/YOLOv8-TensorRT)
[![img](https://badgen.net/badge/icon/tensorrt?icon=azurepipelines&label)](https://developer.nvidia.com/tensorrt)
[![C++](https://img.shields.io/badge/CPP-11%2F14-yellow)](https://github.com/triple-Mu/YOLOv8-TensorRT)
[![img](https://badgen.net/github/license/triple-Mu/YOLOv8-TensorRT)](https://github.com/triple-Mu/YOLOv8-TensorRT/blob/main/LICENSE)
[![img](https://badgen.net/github/prs/triple-Mu/YOLOv8-TensorRT)](https://github.com/triple-Mu/YOLOv8-TensorRT/pulls)
[![img](https://img.shields.io/github/stars/triple-Mu/YOLOv8-TensorRT?color=ccf)](https://github.com/triple-Mu/YOLOv8-TensorRT)
---
# Prepare the environment
1. Install `CUDA` follow [`CUDA official website`](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#download-the-nvidia-cuda-toolkit).
ð RECOMMENDED `CUDA` >= 11.4
2. Install `TensorRT` follow [`TensorRT official website`](https://developer.nvidia.com/nvidia-tensorrt-8x-download).
ð RECOMMENDED `TensorRT` >= 8.4
2. Install python requirements.
``` shell
pip install -r requirements.txt
```
3. Install [`ultralytics`](https://github.com/ultralytics/ultralytics) package for ONNX export or TensorRT API building.
``` shell
pip install ultralytics
```
5. Prepare your own PyTorch weight such as `yolov8s.pt` or `yolov8s-seg.pt`.
***NOTICE:***
Please use the latest `CUDA` and `TensorRT`, so that you can achieve the fastest speed !
If you have to use a lower version of `CUDA` and `TensorRT`, please read the relevant issues carefully !
# Normal Usage
If you get ONNX from origin [`ultralytics`](https://github.com/ultralytics/ultralytics) repo, you should build engine by yourself.
You can only use the `c++` inference code to deserialize the engine and do inference.
You can get more information in [`Normal.md`](docs/Normal.md) !
Besides, other scripts won't work.
# Export End2End ONNX with NMS
You can export your onnx model by `ultralytics` API and add postprocess such as bbox decoder and `NMS` into ONNX model at the same time.
``` shell
python3 export-det.py \
--weights yolov8s.pt \
--iou-thres 0.65 \
--conf-thres 0.25 \
--topk 100 \
--opset 11 \
--sim \
--input-shape 1 3 640 640 \
--device cuda:0
```
#### Description of all arguments
- `--weights` : The PyTorch model you trained.
- `--iou-thres` : IOU threshold for NMS plugin.
- `--conf-thres` : Confidence threshold for NMS plugin.
- `--topk` : Max number of detection bboxes.
- `--opset` : ONNX opset version, default is 11.
- `--sim` : Whether to simplify your onnx model.
- `--input-shape` : Input shape for you model, should be 4 dimensions.
- `--device` : The CUDA deivce you export engine .
You will get an onnx model whose prefix is the same as input weights.
### Just Taste First
If you just want to taste first, you can download the onnx model which are exported by `YOLOv8` package and modified by me.
[**YOLOv8-n**](https://triplemu-shared.oss-cn-beijing.aliyuncs.com/models/yolov8n.onnx?OSSAccessKeyId=LTAI5tNk9iiMqhFC64jCcgpv&Expires=2690974569&Signature=3ct9pnRygBduWdgAtfKOQAt4PeU%3D)
[**YOLOv8-s**](https://triplemu-shared.oss-cn-beijing.aliyuncs.com/models/yolov8s.onnx?OSSAccessKeyId=LTAI5tNk9iiMqhFC64jCcgpv&Expires=10000000001690974000&Signature=cbHjUwmRsYdvilcirzjBI6%2BzmvI%3D)
[**YOLOv8-m**](https://triplemu-shared.oss-cn-beijing.aliyuncs.com/models/yolov8m.onnx?OSSAccessKeyId=LTAI5tNk9iiMqhFC64jCcgpv&Expires=101690974603&Signature=XnJnQqbKsnJSKSgqVQ41kxoeETU%3D)
[**YOLOv8-l**](https://triplemu-shared.oss-cn-beijing.aliyuncs.com/models/yolov8l.onnx?OSSAccessKeyId=LTAI5tNk9iiMqhFC64jCcgpv&Expires=2690974619&Signature=djxvNzcaFosHrMS5ylWh1R0%2Ff8E%3D)
[**YOLOv8-x**](https://triplemu-shared.oss-cn-beijing.aliyuncs.com/models/yolov8x.onnx?OSSAccessKeyId=LTAI5tNk9iiMqhFC64jCcgpv&Expires=2690974637&Signature=DMmuT2wlfBzai%2BBpYJFcmNbkMKU%3D)
# Build End2End Engine from ONNX
### 1. Build Engine by TensorRT ONNX Python api
You can export TensorRT engine from ONNX by [`build.py` ](build.py).
Usage:
``` shell
python3 build.py \
--weights yolov8s.onnx \
--iou-thres 0.65 \
--conf-thres 0.25 \
--topk 100 \
--fp16 \
--device cuda:0
```
#### Description of all arguments
- `--weights` : The ONNX model you download.
- `--iou-thres` : IOU threshold for NMS plugin.
- `--conf-thres` : Confidence threshold for NMS plugin.
- `--topk` : Max number of detection bboxes.
- `--fp16` : Whether to export half-precision engine.
- `--device` : The CUDA deivce you export engine .
You can modify `iou-thres` `conf-thres` `topk` by yourself.
### 2. Export Engine by Trtexec Tools
You can export TensorRT engine by [`trtexec`](https://github.com/NVIDIA/TensorRT/tree/main/samples/trtexec) tools.
Usage:
``` shell
/usr/src/tensorrt/bin/trtexec \
--onnx=yolov8s.onnx \
--saveEngine=yolov8s.engine \
--fp16
```
**If you installed TensorRT by a debian package, then the installation path of `trtexec`
is `/usr/src/tensorrt/bin/trtexec`**
**If you installed TensorRT by a tar package, then the installation path of `trtexec` is under the `bin` folder in the path you decompressed**
# Build TensorRT Engine by TensorRT API
Please see more information in [`API-Build.md`](docs/API-Build.md)
***Notice !!!*** We don't support YOLOv8-seg model now !!!
# Inference
## 1. Infer with python script
You can infer images with the engine by [`infer-det.py`](infer-det.py) .
Usage:
``` shell
python3 infer-det.py \
--engine yolov8s.engine \
--imgs data \
--show \
--out-dir outputs \
--device cuda:0
```
#### Description of all arguments
- `--engine` : The Engine you export.
- `--imgs` : The images path you want to detect.
- `--show` : Whether to show detection results.
- `--out-dir` : Where to save detection results images. It will not work when use `--show` flag.
- `--device` : The CUDA deivce you use.
- `--profile` : Profile the TensorRT engine.
## 2. Infer with C++
You can infer with c++ in [`csrc/detect/end2end`](csrc/detect/end2end) .
### Build:
Please set you own librarys in [`CMakeLists.txt`](csrc/detect/end2end/CMakeLists.txt) and modify `CLASS_NAMES` and `COLORS` in [`main.cpp`](csrc/detect/end2end/main.cpp).
``` shell
export root=${PWD}
cd csrc/detect/end2end
mkdir -p build && cd build
cmake ..
make
mv yolov8 ${root}
cd ${root}
```
Usage:
``` shell
# infer image
./yolov8 yolov8s.engine data/bus.jpg
# infer images
./yolov8 yolov8s.engine data
# infer video
./yolov8 yolov8s.engine data/test.mp4 # the video path
```
# TensorRT Segment Deploy
Please see more information in [`Segment.md`](docs/Segment.md)
# TensorRT Pose Deploy
Please see more information in [`Pose.md`](docs/Pose.md)
# DeepStream Detection Deploy
See more in [`README.md`](csrc/deepstream/README.md)
# Jetson Deploy
Only test on `Jetson-NX 4GB`.
See more in [`Jetson.md`](docs/Jetson.md)
# Profile you engine
If you want to profile the TensorRT engine:
Usage:
``` shell
python3 trt-profile.py --engine yolov8s.engine --device cuda:0
```
# Refuse To Use PyTorch for Model Inference !!!
If you need to break away from pytorch and use tensorrt inference,
you can get more information in [`infer-det-without-torch.py`](infer-det-without-torch.py),
the usage is the same as the pytorch version, but its performance is much worse.
You can use `cuda-python` or `pycuda` for inference.
Please install by such command:
```shell
pip install cuda-python
# or
pip install pycuda
```
Usage:
``` shell
python3 infer-det-without-torch.py \
--engine yolov8s.engine \
--imgs data \
--show \
--out-dir outputs \
--method cudart
```
#### Description of all arguments
- `--engine` : The Engine you export.
- `--imgs` : The i
没有合适的资源?快使用搜索试试~ 我知道了~
TensorRT-YOLOV8 python代码
共89个文件
py:22个
txt:17个
hpp:16个
2 下载量 197 浏览量
2024-04-22
17:18:02
上传
评论
收藏 135KB ZIP 举报
温馨提示
包含YOLOv8的TensorRT加速的相关代码 1、适用于TensorRT模型的导出代码 2、模型调用,结果图保存 3、POST接口、Get接口和接口调用代码 接口代码是先加载模型,再进行推理,推理速度RTX3090速度3.6ms左右 相关教程,请看我博客内容。
资源推荐
资源详情
资源评论
收起资源包目录
YOLOv8_TensorRT.zip (89个子文件)
infer-post.py 429B
export.txt 151B
build.txt 113B
export-seg.py 2KB
csrc
segment
normal
include
common.hpp 4KB
yolov8-seg.hpp 14KB
CMakeLists.txt 2KB
main.cpp 6KB
simple
include
common.hpp 4KB
yolov8-seg.hpp 13KB
CMakeLists.txt 2KB
main.cpp 6KB
jetson
segment
include
common.hpp 4KB
yolov8-seg.hpp 13KB
CMakeLists.txt 2KB
main.cpp 6KB
detect
include
yolov8.hpp 10KB
common.hpp 4KB
CMakeLists.txt 2KB
main.cpp 5KB
pose
include
common.hpp 4KB
yolov8-pose.hpp 13KB
CMakeLists.txt 2KB
main.cpp 7KB
deepstream
deepstream_app_config.txt 3KB
CMakeLists.txt 2KB
config_yoloV8.txt 3KB
custom_bbox_parser
nvdsparsebbox_yoloV8.cpp 5KB
labels.txt 625B
README.md 2KB
detect
normal
include
yolov8.hpp 11KB
common.hpp 4KB
CMakeLists.txt 2KB
main.cpp 6KB
end2end
include
yolov8.hpp 10KB
common.hpp 4KB
CMakeLists.txt 2KB
main.cpp 5KB
build
CMakeFiles
cmake.check_cache 86B
CMakeCache.txt 1KB
pose
normal
include
common.hpp 4KB
yolov8-pose.hpp 13KB
CMakeLists.txt 2KB
main.cpp 7KB
weights
data
LICENSE 1KB
output
.pre-commit-config.yaml 646B
.idea
YOLOv8-TensorRT-main.iml 332B
workspace.xml 11KB
misc.xml 302B
inspectionProfiles
Project_Default.xml 1KB
profiles_settings.xml 174B
modules.xml 299B
.gitignore 184B
infer-det.txt 101B
docs
star.md 172B
Jetson.md 5KB
Pose.md 3KB
Normal.md 2KB
Segment.md 6KB
API-Build.md 719B
infer-det.py 3KB
api.py 3KB
infer-seg.py 4KB
infer-det-without-torch.py 3KB
requirements.txt 105B
models
utils.py 10KB
__init__.py 556B
cudart_api.py 6KB
pycuda_api.py 5KB
api.py 13KB
common.py 6KB
engine.py 14KB
__pycache__
torch_utils.cpython-38.pyc 3KB
utils.cpython-38.pyc 7KB
common.cpython-38.pyc 7KB
engine.cpython-38.pyc 12KB
__init__.cpython-38.pyc 552B
torch_utils.py 3KB
infer-seg-without-torch.py 4KB
.gitignore 2KB
export-det.py 3KB
build.py 2KB
gen_pkl.py 1KB
__pycache__
config.cpython-38.pyc 2KB
outputs
README.md 8KB
infer-pose.py 4KB
infer-pose-without-torch.py 4KB
trt-profile.py 767B
config.py 2KB
共 89 条
- 1
资源评论
相识已是上上签
- 粉丝: 4223
- 资源: 6
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功