# Objects as Points
Object detection, 3D detection, and pose estimation using center point detection:
![](readme/fig2.png)
> [**Objects as Points**](http://arxiv.org/abs/1904.07850),
> Xingyi Zhou, Dequan Wang, Philipp Krähenbühl,
> *arXiv technical report ([arXiv 1904.07850](http://arxiv.org/abs/1904.07850))*
Contact: [[email protected]](mailto:[email protected]). Any questions or discussions are welcomed!
## Updates
- (June, 2020) We released a state-of-the-art Lidar-based 3D detection and tracking framework [CenterPoint](https://github.com/tianweiy/CenterPoint).
- (April, 2020) We released a state-of-the-art (multi-category-/ pose-/ 3d-) tracking extension [CenterTrack](https://github.com/xingyizhou/CenterTrack).
## Abstract
Detection identifies objects as axis-aligned boxes in an image. Most successful object detectors enumerate a nearly exhaustive list of potential object locations and classify each. This is wasteful, inefficient, and requires additional post-processing. In this paper, we take a different approach. We model an object as a single point -- the center point of its bounding box. Our detector uses keypoint estimation to find center points and regresses to all other object properties, such as size, 3D location, orientation, and even pose. Our center point based approach, CenterNet, is end-to-end differentiable, simpler, faster, and more accurate than corresponding bounding box based detectors. CenterNet achieves the best speed-accuracy trade-off on the MS COCO dataset, with 28.1% AP at 142 FPS, 37.4% AP at 52 FPS, and 45.1% AP with multi-scale testing at 1.4 FPS. We use the same approach to estimate 3D bounding box in the KITTI benchmark and human pose on the COCO keypoint dataset. Our method performs competitively with sophisticated multi-stage methods and runs in real-time.
## Highlights
- **Simple:** One-sentence method summary: use keypoint detection technic to detect the bounding box center point and regress to all other object properties like bounding box size, 3d information, and pose.
- **Versatile:** The same framework works for object detection, 3d bounding box estimation, and multi-person pose estimation with minor modification.
- **Fast:** The whole process in a single network feedforward. No NMS post processing is needed. Our DLA-34 model runs at *52* FPS with *37.4* COCO AP.
- **Strong**: Our best single model achieves *45.1*AP on COCO test-dev.
- **Easy to use:** We provide user friendly testing API and webcam demos.
## Main results
### Object Detection on COCO validation
| Backbone | AP / FPS | Flip AP / FPS| Multi-scale AP / FPS |
|--------------|-----------|--------------|-----------------------|
|Hourglass-104 | 40.3 / 14 | 42.2 / 7.8 | 45.1 / 1.4 |
|DLA-34 | 37.4 / 52 | 39.2 / 28 | 41.7 / 4 |
|ResNet-101 | 34.6 / 45 | 36.2 / 25 | 39.3 / 4 |
|ResNet-18 | 28.1 / 142| 30.0 / 71 | 33.2 / 12 |
### Keypoint detection on COCO validation
| Backbone | AP | FPS |
|--------------|-----------|--------------|
|Hourglass-104 | 64.0 | 6.6 |
|DLA-34 | 58.9 | 23 |
### 3D bounding box detection on KITTI validation
|Backbone|FPS|AP-E|AP-M|AP-H|AOS-E|AOS-M|AOS-H|BEV-E|BEV-M|BEV-H|
|--------|---|----|----|----|-----|-----|-----|-----|-----|-----|
|DLA-34 |32 |96.9|87.8|79.2|93.9 |84.3 |75.7 |34.0 |30.5 |26.8 |
All models and details are available in our [Model zoo](readme/MODEL_ZOO.md).
## Installation
Please refer to [INSTALL.md](readme/INSTALL.md) for installation instructions.
## Use CenterNet
We support demo for image/ image folder, video, and webcam.
First, download the models (By default, [ctdet_coco_dla_2x](https://drive.google.com/file/d/18Q3fzzAsha_3Qid6mn4jcIFPeOGUaj1d) for detection and
[multi_pose_dla_3x](https://drive.google.com/file/d/1mC2PAQT_RuHi_9ZMZgkt4rg7BSY2_Lkd) for human pose estimation)
from the [Model zoo](readme/MODEL_ZOO.md) and put them in `CenterNet_ROOT/models/`.
For object detection on images/ video, run:
~~~
python demo.py ctdet --demo /path/to/image/or/folder/or/video --load_model ../models/ctdet_coco_dla_2x.pth
~~~
We provide example images in `CenterNet_ROOT/images/` (from [Detectron](https://github.com/facebookresearch/Detectron/tree/master/demo)). If set up correctly, the output should look like
<p align="center"> <img src='readme/det1.png' align="center" height="230px"> <img src='readme/det2.png' align="center" height="230px"> </p>
For webcam demo, run
~~~
python demo.py ctdet --demo webcam --load_model ../models/ctdet_coco_dla_2x.pth
~~~
Similarly, for human pose estimation, run:
~~~
python demo.py multi_pose --demo /path/to/image/or/folder/or/video/or/webcam --load_model ../models/multi_pose_dla_3x.pth
~~~
The result for the example images should look like:
<p align="center"> <img src='readme/pose1.png' align="center" height="200px"> <img src='readme/pose2.png' align="center" height="200px"> <img src='readme/pose3.png' align="center" height="200px"> </p>
You can add `--debug 2` to visualize the heatmap outputs.
You can add `--flip_test` for flip test.
To use this CenterNet in your own project, you can
~~~
import sys
CENTERNET_PATH = /path/to/CenterNet/src/lib/
sys.path.insert(0, CENTERNET_PATH)
from detectors.detector_factory import detector_factory
from opts import opts
MODEL_PATH = /path/to/model
TASK = 'ctdet' # or 'multi_pose' for human pose estimation
opt = opts().init('{} --load_model {}'.format(TASK, MODEL_PATH).split(' '))
detector = detector_factory[opt.task](opt)
img = image/or/path/to/your/image/
ret = detector.run(img)['results']
~~~
`ret` will be a python dict: `{category_id : [[x1, y1, x2, y2, score], ...], }`
## Benchmark Evaluation and Training
After [installation](readme/INSTALL.md), follow the instructions in [DATA.md](readme/DATA.md) to setup the datasets. Then check [GETTING_STARTED.md](readme/GETTING_STARTED.md) to reproduce the results in the paper.
We provide scripts for all the experiments in the [experiments](experiments) folder.
## Develop
If you are interested in training CenterNet in a new dataset, use CenterNet in a new task, or use a new network architecture for CenterNet, please refer to [DEVELOP.md](readme/DEVELOP.md). Also feel free to send us emails for discussions or suggestions.
## Third-party resources
- CenterNet + embedding learning based tracking: [FairMOT](https://github.com/ifzhang/FairMOT) from [Yifu Zhang](https://github.com/ifzhang).
- Detectron2 based implementation: [CenterNet-better](https://github.com/FateScript/CenterNet-better) from [Feng Wang](https://github.com/FateScript).
- Keras Implementation: [keras-centernet](https://github.com/see--/keras-centernet) from [see--](https://github.com/see--) and [keras-CenterNet](https://github.com/xuannianz/keras-CenterNet) from [xuannianz](https://github.com/xuannianz).
- MXnet implementation: [mxnet-centernet](https://github.com/Guanghan/mxnet-centernet) from [Guanghan Ning](https://github.com/Guanghan).
- Stronger human open estimation models: [centerpose](https://github.com/tensorboy/centerpose) from [tensorboy](https://github.com/tensorboy).
- TensorRT extension with ONNX models: [TensorRT-CenterNet](https://github.com/CaoWGG/TensorRT-CenterNet) from [Wengang Cao](https://github.com/CaoWGG).
- CenterNet + DeepSORT tracking implementation: [centerNet-deep-sort](https://github.com/kimyoon-young/centerNet-deep-sort) from [kimyoon-young](https://github.com/kimyoon-young).
- Blogs on training CenterNet on custom datasets (in Chinese): [ships](https://blog.csdn.net/weixin_42634342/article/details/97756458) from [Rhett Chen](https://blog.csdn.net/weixin_42634342) and [faces](https://blog.csdn.net/weixin_41765699/article/details/100118353) from [linbior](https://me.csdn.net/weixin_41765699).
## License
CenterNet itself is released under the MIT License (refer to the L
没有合适的资源?快使用搜索试试~ 我知道了~
CenterNet-master.zip
共170个文件
py:80个
sh:22个
h:9个
需积分: 5 0 下载量 103 浏览量
2023-10-20
23:53:53
上传
评论
收藏 6.27MB ZIP 举报
温馨提示
CenterNet-master.zip
资源推荐
资源详情
资源评论
收起资源包目录
CenterNet-master.zip (170个子文件)
cpu_nms.c 285KB
dcn_v2_cuda_double.c 19KB
dcn_v2_cuda.c 17KB
dcn_v2.c 1KB
dcn_v2_double.c 1KB
gpu_nms.cpp 263KB
evaluate_object_3d_offline.cpp 33KB
evaluate_object_3d.cpp 32KB
dcn_v2_im2col_cuda_double.cu 20KB
dcn_v2_im2col_cuda.cu 19KB
dcn_v2_psroi_pooling_cuda_double.cu 15KB
dcn_v2_psroi_pooling_cuda.cu 14KB
nms_kernel.cu 5KB
evaluate_object_3d_offline 329KB
.gitignore 1KB
.gitignore 160B
.gitignore 32B
.gitignore 20B
.gitignore 14B
.gitignore 14B
.gitignore 14B
.gitignore 0B
dcn_v2_im2col_cuda_double.h 5KB
dcn_v2_im2col_cuda.h 5KB
dcn_v2_cuda_double.h 3KB
dcn_v2_cuda.h 3KB
dcn_v2_psroi_pooling_cuda_double.h 3KB
dcn_v2_psroi_pooling_cuda.h 3KB
dcn_v2.h 1KB
dcn_v2_double.h 1KB
mail.h 811B
gpu_nms.hpp 146B
33823288584_1d21cf0a26_k.jpg 244KB
34501842524_3c858b3080_k.jpg 229KB
18124840932_e42b3e377c_k.jpg 159KB
17790319373_bd19b24cfc_k.jpg 146KB
19064748793_bb942deea1_k.jpg 137KB
16004479832_a748d55f21_k.jpg 133KB
33887522274_eebd074106_k.jpg 119KB
24274813513_0cfd2ce6d0_k.jpg 111KB
LICENSE 1KB
LICENSE 1KB
LICENSE 1KB
Makefile 94B
Makefile 56B
README.md 9KB
MODEL_ZOO.md 8KB
GETTING_STARTED.md 4KB
DATA.md 4KB
INSTALL.md 2KB
README.md 2KB
DEVELOP.md 1KB
README.md 798B
NOTICE 8KB
NOTICE 917B
fig2.png 2.27MB
pose3.png 998KB
det2.png 799KB
pose2.png 237KB
pose1.png 226KB
det1.png 156KB
dlav0.py 22KB
decode.py 21KB
debugger.py 21KB
opts.py 18KB
pose_dla_dcn.py 17KB
pascal_voc.py 11KB
config.py 11KB
calc_coco_overlap.py 11KB
msra_resnet.py 10KB
resnet_dcn.py 10KB
large_hourglass.py 10KB
imdb.py 9KB
test.py 9KB
multi_pose.py 8KB
losses.py 8KB
image.py 8KB
multi_pose.py 7KB
ddd.py 7KB
voc_eval.py 7KB
ddd.py 7KB
dcn_v2.py 7KB
dcn_v2_func.py 6KB
test.py 6KB
convert_kitti_to_coco.py 6KB
ctdet.py 6KB
exdet.py 6KB
ctdet.py 5KB
setup.py 5KB
coco.py 5KB
data_parallel.py 5KB
exdet.py 5KB
base_detector.py 5KB
ddd_utils.py 4KB
coco_hp.py 4KB
test.py 4KB
visualization.py 4KB
ddd.py 4KB
post_process.py 4KB
multi_pose.py 4KB
共 170 条
- 1
- 2
资源评论
Tosonw
- 粉丝: 88
- 资源: 95
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 本科毕业设计基于C# wpf人脸识别的考勤系统的设计与实现源码.zip
- 基于Ruoyi+uniapp实现学生考勤系统 学生考勤源码+项目说明.zip
- feae6bc968ca68a099455d8b8a8dea35
- 基于Pytorch训练CIRAR10上分类算法.zip
- Pytorch-pytorch深度学习教程之Tensorboard.zip
- 基于C++和Python开发yolov8-face作为人脸检测器dlib作为人脸识别器的人脸考勤系统源码+项目说明.zip
- Pytorch-pytorch深度学习教程之变分自动编码器.zip
- Pytorch-pytorch深度学习教程之神经风格迁移.zip
- Pytorch-pytorch深度学习教程之深度残差网络.zip
- Pytorch-pytorch深度学习教程之循环神经网络.zip
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功