CVPR 2018第二届AI城市挑战工作坊Track 1和Track 3的获胜方式的正式实施_C语言_代码_下载

# Single-Camera and Inter-Camera Vehicle Tracking and 3D Speed Estimation Based on Fusion of Visual and Semantic Features (Winner of Track 1 and Track 3 at the 2nd AI City Challenge Workshop in CVPR 2018) This repository contains our source code of Track 1 and Track 3 at the [2nd AI City Challenge Workshop](https://www.aicitychallenge.org) in [CVPR 2018](http://cvpr2018.thecvf.com/program/workshops). Our team won in both of the tracks at the challenge. The source code of Track 1 is built in MATLAB and C++, with our trained YOLOv2 model provided. The source code of Track 3 is developed in Python and C++, with our trained YOLOv2 model provided. The code has been tested on Linux and Windows. Dependencies include CUDA, cuDNN and OpenCV. The team members include [Zheng (Thomas) Tang](https://github.com/zhengthomastang), [Gaoang Wang](https://github.com/GaoangW), [Hao (Alex) Xiao](https://github.com/AlexXiao95), and Aotian Zheng. [[Paper](http://openaccess.thecvf.com/content_cvpr_2018_workshops/w3/html/Tang_Single-Camera_and_Inter-Camera_CVPR_2018_paper.html)], [[Slides](https://zhengthomastang.github.io/files/AIC18ICT_slides.pdf)], [[Poster](https://zhengthomastang.github.io/files/AIC18ICT_poster.pdf)], [[The 2nd AI City Challenge @ CVPR 2018](https://www.aicitychallenge.org/2018-ai-city-challenge/)] ## Important Notice The datasets for the 2nd AI City Challenge in CVPR 2018 are no longer available to the public. However, as the 3rd AI City Challenge Workshop was launched at CVPR 2019, they provided a new city-scale dataset for multi-camera vehicle tracking as well as image-based re-identification. They also had a new dataset for traffic anomaly detection. The scale of the datasets and the number of vehicles that are being used for evaluation are both unprecedented. To access the new datasets, please follow the data access instructions at the [AI City Challenge website](https://www.aicitychallenge.org/). You may forward your inquiries to aicitychallenges@gmail.com. ## Introduction ### NVIDIA AI City Challenge Workshop at CVPR 2018 The NVIDIA AI City Challenge Workshop at CVPR 2018 specifically focused on ITS problems such as 1. Estimating traffic flow characteristics, such as speed 2. Leveraging unsupervised approaches to detect anomalies caused by crashes, stalled vehicles, etc. 3. Multi-camera tracking, and object re-identification in urban environments Our team participated in 2 out of 3 tracks: 1. Track 1 (Traffic Flow Analysis) - Participating teams submit results for individual vehicle speed for a test set containing 27 1-minute videos. Performance is evaluated based on ground truth generated by a fleet of control vehicles that were driven during the recording. Evaluation for Challenge Track 1 is based on detection rate of the control vehicles and the root mean square error of the predicted control vehicle speeds. 2. Track 3 (Multi-camera Vehicle Detection and Reidentification) - Participating teams identify all vehicles that are seen passing at least once at all of 4 different locations in a set of 15 videos. Evaluation for Challenge Track 3 is based on detection accuracy and localization sensitivity for a set of ground-truth vehicles that were driven through all camera locations at least once. Detailed information of this challenge can be found [here](https://www.aicitychallenge.org/2018-ai-city-challenge/). Our team achieves rank #1 in both Track 1 and Track 3. The demo video for Track 1 can be viewed [here](https://youtu.be/_i4numqiv7Y). The demo video for Track 3 can be view [here](https://youtu.be/Jlvh_KxHl40). ### Single-camera Tracking (SCT) In SCT, the loss function in our data association algorithm consists of motion, temporal and appearance attributes. Especially, a histogram-based adaptive appearance model is designed to encode long-term appearance change. The change of loss is incorporated with a bottom-up clustering strategy for the association of tracklets. Robust 2D-to-3D projection is achieved with EDA optimization applied to camera calibration for speed estimation. ### Inter-camera Tracking (ICT) The proposed appearance model together with DCNN features, license plates, detected car types and traveling time information are combined for the computation of cost function in ICT. ## Code Structure ### Track 1 Under the [Track1](Track1) folder, there are 6 components: 1. [1_VDO2IMG](Track1/1_VDO2IMG): Converting each video file to a folder of frame images 2. [2_CAM_CAL](Track1/2_CAM_CAL): Semi-automatic camera calibration based on minimization of reprojection error by EDA optimization **With the access to GPS coordinates (using Google Maps or other tools), we suggest you to use our newly developed PnP-based calibration tool [here](https://github.com/zhengthomastang/Cal_PnP) instead.** 3. [3_YOLO_VEH](Track1/3_YOLO_VEH): Extension of the YOLOv2 object detector with our trained model for vehicle detection/classification provided **We strongly encourage users to use the latest [YOLOv4 object detector](https://github.com/AlexeyAB/darknet) instead.** 4. [4_TC_tracker](Track1/4_TC_tracker): Proposed tracklet-clustering-based tracking method **Note that this SCT method has been upgraded to TrackletNet Tracker (TNT). The corresponding paper on arXiv is [here](https://arxiv.org/abs/1811.07258). The source code (training + testing) is provided [here](https://github.com/GaoangW/TNT).** 5. [5_APP_MDL](Track1/5_APP_MDL) **(optional)**: Extraction of histogram-based adaptive apperance models and their comparison 6. [6_SPD_EST](Track1/6_SPD_EST): Speed estimation based on input of tracking results and camera parameters **Detailed description of each package is given in each subfolder.** ### Track 3 Under the [Track3](Track3) folder, there are 3 components: 1. [1_Multi-Camera Vehicle Tracking and Re-identification](Track3/1_Multi-Camera%20Vehicle%20Tracking%20and%20Re-identification): Multi-camera vehicle tracking based on a fusion of histogram-based adaptive appearance models, DCNN features, detected car types and traveling time information 2. [2_YOLO_LP](Track3/2_YOLO_LP): Detection of license plate from each cropped vehicle image based on YOLOv2 with our trained model provided **We strongly encourage users to use the latest [YOLOv4 object detector](https://github.com/AlexeyAB/darknet) instead.** 3. [3_LP_COMP](Track3/3_LP_COMP): Comparison of license plates under low resolution **Detailed description of each package is given in each subfolder.** The output of [1_Multi-Camera Vehicle Tracking and Re-identification](Track3/1_Multi-Camera%20Vehicle%20Tracking%20and%20Re-identification) is the similarity scores between pairs of vehicles for comparison. It can be converted into a distance score by inverse proportion. The output of [3_LP_COMP](Track3/3_LP_COMP) is the distance score between each two license plates. The final distance score between two vehicles is the multiplication of the above two distance scores. ## References Please consider to cite these papers in your publications if it helps your research: @inproceedings{Tang18AIC, author = {Zheng Tang and Gaoang Wang and Hao Xiao and Aotian Zheng and Jenq-Neng Hwang}, title = {Single-camera and inter-camera vehicle tracking and {3D} speed estimation based on fusion of visual and semantic features}, booktitle = {Proc. CVPR Workshops}, pages = {108--115}, year = {2018} } @misc{Tang17AIC, author = {Zheng Tang and Gaoang Wang and Tao Liu and Young-Gun Lee and Adwin Jahn and Xu Liu and Xiaodong He and Jenq-Neng Hwang}, title = {Multiple-kernel based vehicle tracking using {3D} deformable model and camera self-calibration}, howpublished = {arXiv:1708.06831}, year = {2017} } ## Disclaimer For any question you can contact [Zheng (Thomas) Tang](https://github.com/zhengthomastang).