<div align="center">
<img width="100%" src="https://github.com/open-mmlab/mmpose/assets/13503330/5b637d76-41dd-4376-9a7f-854cd120799d"/>
</div>
# RTMPose: Real-Time Multi-Person Pose Estimation toolkit based on MMPose
> [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://arxiv.org/abs/2303.07399)
<div align="center">
English | [ç®ä½ä¸æ](README_CN.md)
</div>
______________________________________________________________________
## Abstract
Recent studies on 2D pose estimation have achieved excellent performance on public benchmarks, yet its application in the industrial community still suffers from heavy model parameters and high latency.
In order to bridge this gap, we empirically study five aspects that affect the performance of multi-person pose estimation algorithms: paradigm, backbone network, localization algorithm, training strategy, and deployment inference, and present a high-performance real-time multi-person pose estimation framework, **RTMPose**, based on MMPose.
Our RTMPose-m achieves **75.8% AP** on COCO with **90+ FPS** on an Intel i7-11700 CPU and **430+ FPS** on an NVIDIA GTX 1660 Ti GPU.
To further evaluate RTMPose's capability in critical real-time applications, we also report the performance after deploying on the mobile device. Our RTMPose-s achieves **72.2% AP** on COCO with **70+ FPS** on a Snapdragon 865 chip, outperforming existing open-source libraries.
With the help of MMDeploy, our project supports various platforms like CPU, GPU, NVIDIA Jetson, and mobile devices and multiple inference backends such as ONNXRuntime, TensorRT, ncnn, etc.
______________________________________________________________________
## ð Table of Contents
- [𥳠ð What's New](#--whats-new-)
- [ð Introduction](#-introduction-)
- [ð Community](#-community-)
- [â¡ Pipeline Performance](#-pipeline-performance-)
- [ð Model Zoo](#-model-zoo-)
- [ð Visualization](#-visualization-)
- [ð Get Started](#-get-started-)
- [ð¨âð« How to Train](#-how-to-train-)
- [ðï¸ How to Deploy](#ï¸-how-to-deploy-)
- [ð Common Usage](#ï¸-common-usage-)
- [ð Inference Speed Test](#-inference-speed-test-)
- [ð Model Test](#-model-test-)
- [ð Citation](#-citation-)
## 𥳠ð What's New [ð](#-table-of-contents)
- Jun. 2023:
- Release 26-keypoint Body models trained on combined datasets.
- May. 2023:
- Add [code examples](./examples/) of RTMPose.
- Release Hand, Face, Body models trained on combined datasets.
- Mar. 2023: RTMPose is released. RTMPose-m runs at 430+ FPS and achieves 75.8 mAP on COCO val set.
## ð Introduction [ð](#-table-of-contents)
<div align=center>
<img src="https://user-images.githubusercontent.com/13503330/221138554-110240d8-e887-4b9a-90b1-2fbdc982e9de.gif" width=400 height=300/><img src="https://user-images.githubusercontent.com/13503330/221125176-85015a13-9648-4f0d-a17c-1cbb469efacf.gif" width=250 height=300/><img src="https://user-images.githubusercontent.com/13503330/221125310-7eeb2212-907e-427f-97af-af799d70a4c5.gif" width=250 height=300/>
</div>
<div align=center>
<img src="https://github.com/open-mmlab/mmpose/assets/13503330/38aa345e-4ceb-4e73-bc37-5e082735e336" width=450 height=300/><img src="https://user-images.githubusercontent.com/13503330/221125888-15c20faf-0ad5-4afb-828b-a71ccb064582.gif" width=450 height=300/>
</div>
<div align=center>
<img src="https://github.com/open-mmlab/mmpose/assets/13503330/2ecbf9f4-6963-4a14-9801-da10c0a65dac" width=300 height=350/><img src="https://user-images.githubusercontent.com/13503330/221138017-10431ab4-e515-4c32-8fa7-8748e2d17a58.gif" width=600 height=350/>
</div>
### ⨠Major Features
- ð **High efficiency and high accuracy**
| Model | AP(COCO) | CPU-FPS | GPU-FPS |
| :---: | :------: | :-----: | :-----: |
| t | 68.5 | 300+ | 940+ |
| s | 72.2 | 200+ | 710+ |
| m | 75.8 | 90+ | 430+ |
| l | 76.5 | 50+ | 280+ |
- ð ï¸ **Easy to deploy**
- Step-by-step deployment tutorials.
- Support various backends including
- ONNX
- TensorRT
- ncnn
- OpenVINO
- etc.
- Support various platforms including
- Linux
- Windows
- NVIDIA Jetson
- ARM
- etc.
- ðï¸ **Design for practical applications**
- Pipeline inference API and SDK for
- Python
- C++
- C#
- JAVA
- etc.
## ð Community [ð](#-table-of-contents)
RTMPose is a long-term project dedicated to the training, optimization and deployment of high-performance real-time pose estimation algorithms in practical scenarios, so we are looking forward to the power from the community. Welcome to share the training configurations and tricks based on RTMPose in different business applications to help more community users!
⨠⨠â¨
- **If you are a new user of RTMPose, we eagerly hope you can fill out this [Google Questionnaire](https://docs.google.com/forms/d/e/1FAIpQLSfzwWr3eNlDzhU98qzk2Eph44Zio6hi5r0iSwfO9wSARkHdWg/viewform?usp=sf_link)/[Chinese version](https://uua478.fanqier.cn/f/xxmynrki), it's very important for our work!**
⨠⨠â¨
Feel free to join our community group for more help:
- WeChat Group:
<div align=left>
<img src="https://user-images.githubusercontent.com/13503330/222647056-875bed70-85ec-455c-9016-c024772915c4.jpg" width=200 />
</div>
- Discord Group:
- ð https://discord.gg/raweFPmdzG ð
## â¡ Pipeline Performance [ð](#-table-of-contents)
**Notes**
- Pipeline latency is tested under skip-frame settings, the detection interval is 5 frames by defaults.
- Flip test is NOT used.
- Env Setup:
- torch >= 1.7.1
- onnxruntime 1.12.1
- TensorRT 8.4.3.1
- ncnn 20221128
- cuDNN 8.3.2
- CUDA 11.3
| Detection Config | Pose Config | Input Size<sup><br>(Det/Pose) | Model AP<sup><br>(COCO) | Pipeline AP<sup><br>(COCO) | Params (M)<sup><br>(Det/Pose) | Flops (G)<sup><br>(Det/Pose) | ORT-Latency(ms)<sup><br>(i7-11700) | TRT-FP16-Latency(ms)<sup><br>(GTX 1660Ti) | Download |
| :------------------------------------------------------------------ | :---------------------------------------------------------------------------- | :---------------------------: | :---------------------: | :------------------------: | :---------------------------: | :--------------------------: | :--------------------------------: | :---------------------------------------: | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| [RTMDet-nano](./rtmdet/person/rtmdet_nano_320-8xb32_coco-person.py) | [RTMPose-t](./rtmpose/body_2d_keypoint/rtmpose-t_8xb256-420e_coco-256x192.py) | 320x320<br>256x192 | 40.3<br>67.1 | 64.4 | 0.99<br/>3.34 | 0.31<br/>0.36 | 12.403 | 2.467 | [det](https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_nano_8xb32-100e_coco-obj365-person-05d8511e.pth)<br/>[pose](https://download.openmmlab.com/mmpose/v1/projects/rtmposev1/rtmpose-tiny_simcc-aic-coco_pt-aic-coco_420e-256x192-cfc8f33d_20230126.pth) |
| [RTMDet-nano](./rtmdet/person/rtmdet_nano_320-8xb32_coco-person.py) | [RTMPose-s](./rtmpose/body_2d_keypoint/rtmpose-s_8xb256-420e_coco-256x1
ControlNet迎来重大更新！控手不再是难题！国产AI之光 DWPose
需积分: 0 8 浏览量
2023-10-04
09:33:53
上传
评论
收藏 101.75MB ZIP 举报
ControlNet迎来重大更新！控手不再是难题！国产AI之光 DWPose （2002个子文件） 
cocoeval.cpp 20KB ROIAlignRotated_cpu.cpp 16KB rtmpose_onnxruntime.cpp 8KB rtmpose.cpp 6KB rtmdet_onnxruntime.cpp 5KB rtmdet.cpp 5KB utils.cpp 5KB vision.cpp 3KB nms_rotated_cpu.cpp 2KB main.cpp 2KB ms_deform_attn_cpu.cpp 1KB main.cpp 1KB box_iou_rotated_cpu.cpp 1KB inference.cpp 1KB rtmpose_tracker_onnxruntime.cpp 1016B vision.cpp 942B readthedocs.css 183B readthedocs.css 183B .DS_Store 6KB .DS_Store 6KB box_iou_rotated_utils.h 11KB deform_conv.h 8KB characterset_convert.h 4KB cocoeval.h 3KB ROIAlignRotated.h 3KB rtmpose_utils.h 2KB ms_deform_attn.h 2KB ms_deform_attn_cuda.h 1KB ms_deform_attn_cpu.h 1KB nms_rotated.h 1KB box_iou_rotated.h 988B utils.h 971B rtmpose.h 890B rtmdet.h 799B rtmpose_tracker_onnxruntime.h 771B rtmpose_onnxruntime.h 749B rtmdet_onnxruntime.h 569B inference.h 273B calibration_160906_band2.json 246KB calibration_160906_band1.json 246KB test_coco_wholebody.json 174KB test_keypoint_partition_metric.json 171KB test_halpe.json 119KB test_ap10k.json 115KB test_posetrack18_human_detections.json 70KB test_coco.json 60KB test_posetrack18_val.json 55KB test_interhand2.6m_joint_3d.json 35KB 012834_mpii_test.json 33KB deepfasion2.json 32KB test_wflw.json 32KB test_coco_det_AP_H_56.json 30KB test_freihand.json 25KB test_ochuman.json 23KB test_rhd.json 23KB test_humanart.json 18KB 003418_mpii_test.json 18KB test_mpii_trb.json 17KB test_panoptic.json 16KB test_animalkingdom.json 13KB test_aic.json 13KB test_interhand2.6m_data.json 12KB test_onehand10k.json 11KB test_300w.json 11KB test_locust.json 11KB test_macaque.json 10KB test_mhp.json 10KB test_fly.json 9KB h36m_coco.json 9KB 009473_mpii_test.json 8KB test_mpii.json 8KB test_crowdpose.json 8KB test_jhmdb_sub1.json 7KB test_horse10.json 7KB test_cofw.json 7KB test_animalpose.json 6KB open_mmlab.json 5KB test_lapa.json 5KB test_aflw.json 5KB test_atrw.json 5KB test_interhand2.6m_camera.json 4KB test_zebra.json 4KB mmcls.json 4KB test_humanart_det_AP_H_56.json 3KB calibration_shelf.json 3KB test_fld.json 3KB body3DScene_00000139.json 2KB body3DScene_00000140.json 2KB calibration_campus.json 2KB body3DScene_00000168.json 2KB body3DScene_00000169.json 2KB test_crowdpose_det_AP_40.json 2KB deprecated.json 217B changelog.md 75KB changelog.md 75KB README.md 72KB README_CN.md 71KB README.md 31KB README_CN.md 28KB guide_to_framework.md 27KB共 2002 条
- 1
- 2
- 3
- 4
- 5
- 6
- 21
资源评论
