目标跟踪-基于目标中心点同时进行目标检测+目标跟踪算法实现-项目源码-优质项目实战.zip

# Tracking Objects as Points Simultaneous object detection and tracking using center points:  ## Abstract Tracking has traditionally been the art of following interest points through space and time. This changed with the rise of powerful deep networks. Nowadays, tracking is dominated by pipelines that perform object detection followed by temporal association, also known as tracking-by-detection. In this paper, we present a simultaneous detection and tracking algorithm that is simpler, faster, and more accurate than the state of the art. Our tracker, CenterTrack, applies a detection model to a pair of images and detections from the prior frame. Given this minimal input, CenterTrack localizes objects and predicts their associations with the previous frame. That's it. CenterTrack is simple, online (no peeking into the future), and real-time. It achieves 67.3% MOTA on the MOT17 challenge at 22 FPS and 89.4% MOTA on the KITTI tracking benchmark at 15 FPS, setting a new state of the art on both datasets. CenterTrack is easily extended to monocular 3D tracking by regressing additional 3D attributes. Using monocular video input, it achieves 28.3% AMOTA@0.2 on the newly released nuScenes 3D tracking benchmark, substantially outperforming the monocular baseline on this benchmark while running at 28 FPS. ## Features at a glance - One-sentence method summary: Our model takes the current frame, the previous frame, and a heatmap rendered from previous tracking results as input, and predicts the current detection heatmap as well as their offsets to centers in the previous frame. - The model can be trained on still **image datasets** if videos are not available. - Easily extends to monocular 3d object tracking, multi-category tracking, and pose tracking. - State-of-the-art performance on MOT17, KITTI, and nuScenes monocular tracking benchmarks. ## Main results ### Pedestrian tracking on MOT17 test set | Detection | MOTA | FPS | |--------------|-----------|--------| |Public | 61.5 | 22 | |Private | 67.8 | 22 | ### 2D vehicle tracking on KITTI test set (with flip test) | MOTA | FPS | |-------------|--------| | 89.44 | 15 | ### 3D tracking on nuScenes test set | AMOTA @ 0.2 | AMOTA | FPS | |---------------|---------|--------| | 27.8 | 4.6 | 28 | Besides benchmark evaluation, we also provide models for 80-category tracking and pose tracking trained on COCO. See the sample visual results below (Video files from [openpose](https://github.com/CMU-Perceptual-Computing-Lab/openpose) and [YOLO](https://pjreddie.com/darknet/yolov2/)). <p align="center"> <img src='readme/coco_det.gif' align="center" height="230px"> </p> <p align="center"> <img src='readme/coco_pose.gif' align="center" height="230px"> </p> All models and details are available in our [Model zoo](readme/MODEL_ZOO.md). ## Installation Please refer to [INSTALL.md](readme/INSTALL.md) for installation instructions. ## Use CenterTrack We support demo for videos, webcam, and image folders. First, download the models (By default, [nuscenes\_3d\_tracking](https://drive.google.com/file/d/1gPQFzqneDtT_PjJRRuyskRsNTRHXovw1) for monocular 3D tracking, [coco_tracking](https://drive.google.com/file/d/11DEfWa0TKYzNqY3CXR51WVvjMb4oRl08) for 80-category detection and [coco_pose_tracking](https://drive.google.com/file/d/1yGFC_Q9wzSHL1d4eZW_44EBB2H42YKYt) for pose tracking) from the [Model zoo](readme/MODEL_ZOO.md) and put them in `CenterNet_ROOT/models/`. We provide a video clip from the [nuScenes dataset](https://www.nuscenes.org/?externalData=all&mapData=all&modalities=Any) in `videos/nuscenes_mini.mp4`. To test monocular 3D tracking on this video, run ~~~ python demo.py tracking,ddd --load_model ../models/nuScenes_3Dtracking.pth --dataset nuscenes --pre_hm --track_thresh 0.1 --demo ../videos/nuscenes_mini.mp4 --test_focal_length 633 ~~~ You will need to specify `test_focal_length` for monocular 3D tracking demo to convert the image coordinate system back to 3D. The value `633` is half of a typical focal length (`~1266`) in nuScenes dataset in input resolution `1600x900`. The mini demo video is in an input resolution of `800x448`, so we need to use a half focal length. You don't need to set the `test_focal_length` when testing on the original nuScenes data. If setup correctly, you will see an output video like: <p align="center"> <img src='readme/nuscenes_3d.gif' align="center" height="230px"> </p> Similarly, for 80-category tracking on images/ video, run: ~~~ python demo.py tracking --load_model ../models/coco_tracking.pth --demo /path/to/image/or/folder/or/video ~~~ If you want to test with person tracking models, you need to add `--num_class 1`: ~~~ python demo.py tracking --load_model ../models/mot17_half.pth --num_class 1 --demo /path/to/image/or/folder/or/video ~~~ For webcam demo, run ~~~ python demo.py tracking --load_model ../models/coco_tracking.pth --demo webcam ~~~ For monocular 3D tracking, run ~~~ python demo.py tracking,ddd --demo webcam --load_model ../models/coco_tracking.pth --demo /path/to/image/or/folder/or/video/or/webcam ~~~ Similarly, for pose tracking, run: ~~~ python demo.py tracking,multi_pose --load_model ../models/coco_pose.pth --demo /path/to/image/or/folder/or/video/or/webcam ~~~ The result for the example images should look like: You can add `--debug 2` to visualize the heatmap and offset predictions. To use this CenterTrack in your own project, you can ~~~ import sys CENTERTRACK_PATH = /path/to/CenterTrack/src/lib/ sys.path.insert(0, CENTERTRACK_PATH) from detector import Detector from opts import opts MODEL_PATH = /path/to/model TASK = 'tracking' # or 'tracking,multi_pose' for pose tracking and 'tracking,ddd' for monocular 3d tracking opt = opts().init('{} --load_model {}'.format(TASK, MODEL_PATH).split(' ')) detector = Detector(opt) images = ['''image read from open cv or from a video'''] for img in images: ret = detector.run(img)['results'] ~~~ Each `ret` will be a list dict: `[{'bbox': [x1, y1, x2, y2], 'tracking_id': id, ...}]` ## Training on custom dataset If you want to train CenterTrack on your own dataset, you can use `--dataset custom` and manually specify the annotation file, image path, input resolutions, and number of categories. You still need to create the annotation files in COCO format (referring to the many `convert_X_to_coco.py` examples in `tools`). For example, you can use the following command to train on our [mot17 experiment](experiments/mot17_half_sc.sh) without using the pre-defined mot dataset file: ~~~ python main.py tracking --exp_id mot17_half_sc --dataset custom --custom_dataset_ann_path ../data/mot17/annotations/train_half.json --custom_dataset_img_path ../data/mot17/train/ --input_h 544 --input_w 960 --num_classes 1 --pre_hm --ltrb_amodal --same_aug --hm_disturb 0.05 --lost_disturb 0.4 --fp_disturb 0.1 --gpus 0,1 ~~~ ## Benchmark Evaluation and Training After [installation](readme/INSTALL.md), follow the instructions in [DATA.md](readme/DATA.md) to setup the datasets. Then check [GETTING_STARTED.md](readme/GETTING_STARTED.md) to reproduce the results in the paper. We provide scripts for all the experiments in the [experiments](experiments) folder.