# SuperPoint SuperGlue TensorRT
SuperPoint and SuperGlue with TensorRT. Deploy with C++.
## Demo
<img src="image/superpoint_superglue_tensorrt.gif" width = "640" height = "240" alt="match_image" border="10" />
* This demo was tested on the Quadro P620 GPU.
## Baseline
| Image Size: 320 x 240 | RTX3080 | Quadro P620 |
|:----------------------:|:-------:|:-----------:|
| SuperPoint (250 points)| 1.04 MS | 13.61 MS |
| SuperPoint (257 points)| 1.03 MS | 13.32 MS |
| SuperGlue (256 dims) | 7.47 MS | 58.83 MS |
| FPS (superpoint one time + superglue one time) | $\approx$ 117 FPS | $\approx$ 13 FPS |
## Docker(Recommand)
```bash
docker pull yuefan2022/tensorrt-ubuntu20.04-cuda11.6:latest
docker run -it --env DISPLAY=$DISPLAY --volume /tmp/.X11-unix:/tmp/.X11-unix --privileged --runtime nvidia --gpus all --volume ${PWD}:/workspace --workdir /workspace --name tensorrt yuefan2022/tensorrt-ubuntu20.04-cuda11.6:latest /bin/bash
```
## Environment required
* CUDA==11.6
* TensorRT==8.4.1.5
* OpenCV>=4.0
* EIGEN
* yaml-cpp
## Convert model(Optional)
The converted model is already provided in the [weights](./weights) folder, if you are using the pretrained model officially provided by [SuperPoint and SuperGlue](https://github.com/magicleap/SuperGluePretrainedNetwork), you do not need to go through this step.
```bash
python convert2onnx/convert_superpoint_to_onnx.py --weight_file superpoint_pth_file_path --output_dir superpoint_onnx_file_dir
python convert2onnx/convert_superglue_to_onnx.py --weight_file superglue_pth_file_path --output_dir superglue_onnx_file_dir
```
## Build and run
```bash
git clone https://github.com/yuefanhao/SuperPoint-SuperGlue-TensorRT.git
cd SuperPoint-SuperGlue-TensorRT
mkdir build
cd build
cmake ..
make
# test on image pairs 100 times, the output image will be saved in the build dir
./superpointglue_image ../config/config.yaml ../weights/ ${PWD}/../image/image0.png ${PWD}/../image/image1.png
# test on the folder with image sequence, output images will be saved in the param assigned dir
./superpointglue_sequence ../config/config.yaml ../weights/ ${PWD}/../image/freiburg_sequence/ ${PWD}/../image/freiburg_sequence/match_images/
```
The default image size param is 320x240, if you need to modify the image size in the config file, you should delete the old .engine file in the weights dir.
## Samples
```c++
#include "super_point.h"
#include "super_glue.h"
// read image
cv::Mat image0 = cv::imread("../image/image0.png", cv::IMREAD_GRAYSCALE);
cv::Mat image1 = cv::imread("../image/image1.png", cv::IMREAD_GRAYSCALE);
// read config from file
Configs configs("../config/config.yaml", "../weights/");
// create superpoint detector and superglue matcher
auto superpoint = std::make_shared<SuperPoint>(configs.superpoint_config);
auto superglue = std::make_shared<SuperGlue>(configs.superglue_config);
// build engine
superpoint->build();
superglue->build();
// infer superpoint
Eigen::Matrix<double, 259, Eigen::Dynamic> feature_points0, feature_points1;
superpoint->infer(image0, feature_points0);
superpoint->infer(image1, feature_points1)
// infer superglue
std::vector<cv::DMatch> superglue_matches;
superglue->matching_points(feature_points0, feature_points1, superglue_matches);
```
## Acknowledgements
[SuperPoint](https://github.com/magicleap/SuperPointPretrainedNetwork) [SuperGlue](https://github.com/magicleap/SuperGluePretrainedNetwork) [TensorRT](https://github.com/NVIDIA/TensorRT) [AirVO](https://github.com/xukuanHIT/AirVO)
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
【毕业设计】使用TensorRT+Cpp部署SuperPoint+SuperGlue算法模型源码+运行环境说明.zip (50个子文件)
code
inference_image.cpp 5KB
include
super_point.h 2KB
super_glue.h 3KB
read_config.h 4KB
utils.h 4KB
CMakeLists.txt 1KB
weights
superglue_indoor_sim_int32.engine 62.54MB
superglue_outdoor_sim_int32.onnx 45.92MB
superpoint_v1_sim_int32.onnx 4.97MB
superglue_indoor_sim_int32.onnx 45.92MB
superpoint_v1_sim_int32.engine 8.86MB
superglue_outdoor_sim_int32.engine 62.53MB
image
freiburg_sequence
1341847987.758741.png 490KB
1341847985.746954.png 476KB
1341847980.722988.png 464KB
1341847996.874766.png 472KB
1341847994.866828.png 467KB
1341847984.743352.png 455KB
1341847995.870641.png 476KB
1341847989.802890.png 507KB
1341847982.730674.png 460KB
1341847983.738736.png 460KB
1341847981.726650.png 463KB
1341847986.762616.png 493KB
1341847992.818723.png 487KB
1341847988.769740.png 503KB
1341847993.826735.png 467KB
1341847991.814748.png 476KB
1341847990.810771.png 486KB
superpoint_superglue_tensorrt.gif 1.7MB
image1.png 463KB
image0.png 464KB
match_image.png 218KB
src
super_point.cpp 14KB
super_glue.cpp 22KB
inference_sequence.cpp 4KB
3rdparty
tensorrtbuffer
include
buffers.h 16KB
logger.h 1KB
logging.h 18KB
common.h 31KB
error_recorder.h 5KB
half.h 201KB
CMakeLists.txt 343B
src
logger.cpp 1KB
convert2onnx
superglue.py 12KB
convert_superpoint_to_onnx.py 3KB
convert_superglue_to_onnx.py 4KB
superpoint.py 9KB
README.md 4KB
config
config.yaml 654B
共 50 条
- 1
资源评论
熬夜写代码的平头哥
- 粉丝: 3789
- 资源: 7360
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功