# 3D Bounding Box Estimation Using Deep Learning and Geometry
If interested, join the slack workspace where the paper is discussed, issues are worked through, and more! Click this [link](https://join.slack.com/t/3dboundingbox-oun9186/shared_invite/enQtNDk4Njg2NzYyNzY5LWVlZWRlMjNhZmZlYjVmNGY3NWVlNDA4MmY2ZWQ3ZmUyY2Q4OWIxMmY4NzU4YmViM2ViZWI5YjgxOTIyOTI4ZjI) to join.
## Introduction
PyTorch implementation for this [paper](https://arxiv.org/abs/1612.00496).
![example-image](http://soroushkhadem.com/img/2d-top-3d-bottom1.png)
At the moment, it takes approximately 0.4s per frame, depending on the number of objects
detected. An improvement will be speed upgrades soon. Here is the current fastest possible:
![example-video](eval/example/3d-bbox-vid.gif)
## Requirements
- PyTorch
- Cuda
- OpenCV >= 3.4.3
## Usage
In order to download the weights:
```
cd weights/
./get_weights.sh
```
This will download pre-trained weights for the 3D BoundingBox net and also YOLOv3 weights from the
official yolo [source](https://pjreddie.com/darknet/yolo/).
>If script is not working: [pre trained weights](https://drive.google.com/open?id=1yEiquJg9inIFgR3F-N5Z3DbFnXJ0aXmA) and
[YOLO weights](https://pjreddie.com/media/files/yolov3.weights)
To see all the options:
```
python Run.py --help
```
Run through all images in default directory (eval/image_2/), optionally with the 2D
bounding boxes also drawn. Press SPACE to proceed to next image, and any other key to exit.
```
python Run.py [--show-yolo]
```
>Note: See [training](#training) for where to download the data from
There is also a script provided to download the default video from Kitti in ./eval/video. Or,
download any Kitti video and corresponding calibration and use `--image-dir` and `--cal-dir` to
specify where to get the frames from.
```
python Run.py --video [--hide-debug]
```
## Training
First, the data must be downloaded from [Kitti](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=2d).
Download the left color images, the training labels, and the camera calibration matrices. Total is ~13GB.
Unzip the downloads into the Kitti/ directory.
```
python Train.py
```
By default, the model is saved every 10 epochs in weights/.
The loss is printed every 10 batches. The loss should not converge to 0! The loss function for
the orientation is driven to -1, so a negative loss is expected. The hyper-parameters to tune
are alpha and w (see paper). I obtained good results after just 10 epochs, but the training
script will run until 100.
## How it works
The PyTorch neural net takes in images of size 224x224 and predicts the orientation and
relative dimension of that object to the class average. Thus, another neural net must give
the 2D bounding box and object class. I chose to use YOLOv3 through OpenCV.
Using the orientation, dimension, and 2D bounding box, the 3D location is calculated, and then
back projected onto the image.
There are 2 key assumptions made:
1. The 2D bounding box fits very tightly around the object
2. The object has ~0 pitch and ~0 roll (valid for cars on the road)
## Future Goals
- Train custom YOLO net on the Kitti dataset
- Some type of Pose visualization (ROS?)
## Credit
I originally started from a fork of this [repo](https://github.com/fuenwang/3D-BoundingBox), and some of the original code still exists in the training script.
没有合适的资源?快使用搜索试试~ 我知道了~
3dbounding box代码
共112个文件
txt:33个
png:29个
py:13个
需积分: 0 5 下载量 139 浏览量
2022-07-29
09:18:28
上传
评论
收藏 563.93MB RAR 举报
温馨提示
3dbounding box代码
资源详情
资源评论
资源推荐
收起资源包目录
3dbounding box代码 (112个子文件)
yolov3.cfg 9KB
evaluate_object.cpp 38KB
3d-bbox-vid.gif 6.52MB
.gitignore 112B
mail.h 5KB
000.jpg 178KB
LICENSE 1KB
visualization.m 3KB
writeLabels.m 3KB
run_demo.m 3KB
run_statistics.m 1KB
readLabels.m 1KB
computeBox3D.m 1KB
drawBox2D.m 1KB
drawBox3D.m 1KB
computeOrientation3D.m 978B
run_readWriteDemo.m 796B
projectToImage.m 598B
readCalibration.m 208B
README.md 3KB
README.md 459B
README.md 68B
coco.names 705B
epoch_10.pkl 351.97MB
007091.png 910KB
000000.png 873KB
000008.png 866KB
000004.png 826KB
000003.png 815KB
000010.png 806KB
000007.png 795KB
000005.png 761KB
000002.png 749KB
000009.png 734KB
000006.png 731KB
000001.png 719KB
000036.png 664KB
000008.png 275KB
000006.png 275KB
000009.png 274KB
000007.png 274KB
000005.png 274KB
000004.png 274KB
000000.png 274KB
000010.png 272KB
000003.png 272KB
000001.png 272KB
000011.png 269KB
000002.png 269KB
000012.png 268KB
000013.png 265KB
000014.png 264KB
000015.png 258KB
Dataset.py 10KB
Run.py 7KB
Math.py 6KB
Plotting.py 4KB
Run_no_yolo.py 4KB
Train.py 4KB
yolo.py 3KB
Model.py 2KB
ClassAverages.py 2KB
File.py 2KB
__init__.py 0B
__init__.py 0B
__init__.py 0B
Dataset.cpython-36.pyc 7KB
Math.cpython-36.pyc 4KB
Plotting.cpython-36.pyc 4KB
yolo.cpython-36.pyc 3KB
File.cpython-36.pyc 2KB
ClassAverages.cpython-36.pyc 2KB
Model.cpython-36.pyc 2KB
__init__.cpython-36.pyc 166B
__init__.cpython-36.pyc 164B
__init__.cpython-36.pyc 161B
get_weights.sh 545B
get_video.sh 529B
train_mapping.txt 369KB
train_rand.txt 39KB
readme.txt 16KB
CMakeLists.txt 4KB
calib_cam_to_cam.txt 3KB
calib_cam_to_cam.txt 3KB
000000.txt 2KB
000006.txt 2KB
000010.txt 2KB
000003.txt 2KB
000005.txt 2KB
000008.txt 2KB
000007.txt 2KB
000002.txt 2KB
000009.txt 2KB
000036.txt 2KB
000004.txt 2KB
000001.txt 2KB
007091.txt 2KB
000010.txt 1KB
000008.txt 798B
class_averages.txt 763B
共 112 条
- 1
- 2
一眼半年
- 粉丝: 39
- 资源: 3
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功
评论0