MMPose 是一款基于 PyTorch 的姿态分析的开源工具箱，是 OpenMMLab 项目的成员之一

<div align="center"> <img width="100%" src="https://user-images.githubusercontent.com/15977946/225229448-36ff568d-a723-4248-bb19-2df4044ff8e8.png"/> </div> # RTMPose: Real-Time Multi-Person Pose Estimation toolkit based on MMPose > [RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose](https://arxiv.org/abs/2303.07399) <div align="center"> [](https://paperswithcode.com/sota/2d-human-pose-estimation-on-coco-wholebody-1?p=rtmpose-real-time-multi-person-pose) </div> <div align="center"> English | [简体中文](README_CN.md) </div> ______________________________________________________________________ ## Abstract Recent studies on 2D pose estimation have achieved excellent performance on public benchmarks, yet its application in the industrial community still suffers from heavy model parameters and high latency. In order to bridge this gap, we empirically study five aspects that affect the performance of multi-person pose estimation algorithms: paradigm, backbone network, localization algorithm, training strategy, and deployment inference, and present a high-performance real-time multi-person pose estimation framework, **RTMPose**, based on MMPose. Our RTMPose-m achieves **75.8% AP** on COCO with **90+ FPS** on an Intel i7-11700 CPU and **430+ FPS** on an NVIDIA GTX 1660 Ti GPU, and RTMPose-l achieves **67.0% AP** on COCO-WholeBody with **130+ FPS**. To further evaluate RTMPose's capability in critical real-time applications, we also report the performance after deploying on the mobile device. Our RTMPose-s achieves **72.2% AP** on COCO with **70+ FPS** on a Snapdragon 865 chip, outperforming existing open-source libraries. With the help of MMDeploy, our project supports various platforms like CPU, GPU, NVIDIA Jetson, and mobile devices and multiple inference backends such as ONNXRuntime, TensorRT, ncnn, etc.  ______________________________________________________________________ ## 📄 Table of Contents - [🥳 🚀 What's New](#--whats-new-) - [📖 Introduction](#-introduction-) - [🙌 Community](#-community-) - [⚡ Pipeline Performance](#-pipeline-performance-) - [📊 Model Zoo](#-model-zoo-) - [👀 Visualization](#-visualization-) - [😎 Get Started](#-get-started-) - [👨‍🏫 How to Train](#-how-to-train-) - [🏗️ How to Deploy](#️-how-to-deploy-) - [📚 Common Usage](#️-common-usage-) - [🚀 Inference Speed Test](#-inference-speed-test-) - [📊 Model Test](#-model-test-) - [📜 Citation](#-citation-) ## 🥳 🚀 What's New [🔝](#-table-of-contents) - Mar. 2023: RTMPose is released. RTMPose-m runs at 430+ FPS and achieves 75.8 mAP on COCO val set. ## 📖 Introduction [🔝](#-table-of-contents) <div align=center> <img src="https://user-images.githubusercontent.com/13503330/221138554-110240d8-e887-4b9a-90b1-2fbdc982e9de.gif" width=400 height=300/><img src="https://user-images.githubusercontent.com/13503330/221125176-85015a13-9648-4f0d-a17c-1cbb469efacf.gif" width=250 height=300/><img src="https://user-images.githubusercontent.com/13503330/221125310-7eeb2212-907e-427f-97af-af799d70a4c5.gif" width=250 height=300/> </div> <div align=center> <img src="https://user-images.githubusercontent.com/13503330/221125768-8e0d6754-e66d-4941-ac53-ded8db9b60f9.gif" width=450 height=300/><img src="https://user-images.githubusercontent.com/13503330/221125888-15c20faf-0ad5-4afb-828b-a71ccb064582.gif" width=450 height=300/> </div> <div align=center> <img src="https://user-images.githubusercontent.com/13503330/221124560-af84b291-4300-4027-87ae-8c3a201c3f67.gif" width=300 height=350/><img src="https://user-images.githubusercontent.com/13503330/221138017-10431ab4-e515-4c32-8fa7-8748e2d17a58.gif" width=600 height=350/> </div> ### ✨ Major Features - 🚀 **High efficiency and high accuracy** | Model | AP(COCO) | CPU-FPS | GPU-FPS | | :---: | :------: | :-----: | :-----: | | t | 68.5 | 300+ | 940+ | | s | 72.2 | 200+ | 710+ | | m | 75.8 | 90+ | 430+ | | l | 76.5 | 50+ | 280+ | - 🛠️ **Easy to deploy** - Step-by-step deployment tutorials. - Support various backends including - ONNX - TensorRT - ncnn - OpenVINO - etc. - Support various platforms including - Linux - Windows - NVIDIA Jetson - ARM - etc. - 🏗️ **Design for practical applications** - Pipeline inference API and SDK for - Python - C++ - C# - JAVA - etc. ## 🙌 Community [🔝](#-table-of-contents) RTMPose is a long-term project dedicated to the training, optimization and deployment of high-performance real-time pose estimation algorithms in practical scenarios, so we are looking forward to the power from the community. Welcome to share the training configurations and tricks based on RTMPose in different business applications to help more community users! ✨ ✨ ✨ - **If you are a new user of RTMPose, we eagerly hope you can fill out this [Google Questionnaire](https://docs.google.com/forms/d/e/1FAIpQLSfzwWr3eNlDzhU98qzk2Eph44Zio6hi5r0iSwfO9wSARkHdWg/viewform?usp=sf_link)/[Chinese version](https://uua478.fanqier.cn/f/xxmynrki), it's very important for our work!** ✨ ✨ ✨ Feel free to join our community group for more help: - WeChat Group: <div align=left> <img src="https://user-images.githubusercontent.com/13503330/222647056-875bed70-85ec-455c-9016-c024772915c4.jpg" width=200 /> </div> - Discord Group: - 🙌 https://discord.gg/raweFPmdzG 🙌 ## ⚡ Pipeline Performance [🔝](#-table-of-contents) **Notes** - Pipeline latency is tested under skip-frame settings, the detection interval is 5 frames by defaults. - Flip test is NOT used. - Env Setup: - torch >= 1.7.1 - onnxruntime 1.12.1 - TensorRT 8.4.3.1 - ncnn 20221128 - cuDNN 8.3.2 - CUDA 11.3 | Detection Config | Pose Config | Input Size<sup><br>(Det/Pose) | Model AP<sup><br>(COCO) | Pipeline AP<sup><br>(COCO) | Params (M)<sup><br>(Det/Pose) | Flops (G)<sup><br>(Det/Pose) | ORT-Latency(ms)<sup><br>(i7-11700) | TRT-FP16-Latency(ms)<sup><br>(GTX 1660Ti) | Download | | :------------------------------------------------------------------ | :---------------------------------------------------------------------------- | :---------------------------: | :---------------------: | :------------------------: | :---------------------------: | :--------------------------: | :--------------------------------: | :---------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------: | | [RTMDet-nano](./rtmdet/person/rtmdet_nano_320-8xb32_coco-person.py) | [RTMPose-t](./rtmpose/body_2d_keypoint/rtmpose-t_8xb256-420e_coco-256x192.py) | 320x320<br>256x192 | 40.3<br>67.1 | 64.4 | 0.99<br/>3.34 | 0.31<br/>0.36 | 12.403 | 2.467 | [det](https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmdet_nano_8xb32-100e_coco-obj365-person-05d8511e.pth)<br/>[pose](https://download.openmmlab.com/mmpose/v1/projects/rtmpose/rtmpose-tiny_s