yolact_person.rar

# **Y**ou **O**nly **L**ook **A**t **C**oefficien**T**s ``` ██╗ ██╗ ██████╗ ██╗ █████╗ ██████╗████████╗ ╚██╗ ██╔╝██╔═══██╗██║ ██╔══██╗██╔════╝╚══██╔══╝ ╚████╔╝ ██║ ██║██║ ███████║██║ ██║ ╚██╔╝ ██║ ██║██║ ██╔══██║██║ ██║ ██║ ╚██████╔╝███████╗██║ ██║╚██████╗ ██║ ╚═╝ ╚═════╝ ╚══════╝╚═╝ ╚═╝ ╚═════╝ ╚═╝ ``` A simple, fully convolutional model for real-time instance segmentation. This is the code for [our paper](https://arxiv.org/abs/1904.02689). #### ICCV update (v1.1) released! Check out the ICCV trailer here: [](https://www.youtube.com/watch?v=0pMfmo8qfpQ) Read [the changelog](CHANGELOG.md) for details on, well, what changed. Oh, and the paper got updated too with pascal results and an appendix with box mAP. Some examples from our base model (33.5 fps on a Titan Xp and 29.8 mAP on COCO's `test-dev`):    # Installation - Set up a Python3 environment. - Install [Pytorch](http://pytorch.org/) 1.0.1 (or higher) and TorchVision. - Install some other packages: ```Shell # Cython needs to be installed before pycocotools pip install cython pip install opencv-python pillow pycocotools matplotlib ``` - Clone this repository and enter it: ```Shell git clone https://github.com/dbolya/yolact.git cd yolact ``` - If you'd like to train YOLACT, download the COCO dataset and the 2014/2017 annotations. Note that this script will take a while and dump 21gb of files into `./data/coco`. ```Shell sh data/scripts/COCO.sh ``` - If you'd like to evaluate YOLACT on `test-dev`, download `test-dev` with this script. ```Shell sh data/scripts/COCO_test.sh ``` # Evaluation As of April 5th, 2019 here are our latest models along with their FPS on a Titan Xp and mAP on `test-dev`: | Image Size | Backbone | FPS | mAP | Weights | | |:----------:|:-------------:|:----:|:----:|----------------------------------------------------------------------------------------------------------------------|--------| | 550 | Resnet50-FPN | 42.5 | 28.2 | [yolact_resnet50_54_800000.pth](https://drive.google.com/file/d/1yp7ZbbDwvMiFJEq4ptVKTYTI2VeRDXl0/view?usp=sharing) | [Mirror](https://ucdavis365-my.sharepoint.com/:u:/g/personal/yongjaelee_ucdavis_edu/EUVpxoSXaqNIlssoLKOEoCcB1m0RpzGq_Khp5n1VX3zcUw) | | 550 | Darknet53-FPN | 40.0 | 28.7 | [yolact_darknet53_54_800000.pth](https://drive.google.com/file/d/1dukLrTzZQEuhzitGkHaGjphlmRJOjVnP/view?usp=sharing) | [Mirror](https://ucdavis365-my.sharepoint.com/:u:/g/personal/yongjaelee_ucdavis_edu/ERrao26c8llJn25dIyZPhwMBxUp2GdZTKIMUQA3t0djHLw) | 550 | Resnet101-FPN | 33.0 | 29.8 | [yolact_base_54_800000.pth](https://drive.google.com/file/d/1UYy3dMapbH1BnmtZU4WH1zbYgOzzHHf_/view?usp=sharing) | [Mirror](https://ucdavis365-my.sharepoint.com/:u:/g/personal/yongjaelee_ucdavis_edu/EYRWxBEoKU9DiblrWx2M89MBGFkVVB_drlRd_v5sdT3Hgg) | 700 | Resnet101-FPN | 23.6 | 31.2 | [yolact_im700_54_800000.pth](https://drive.google.com/file/d/1lE4Lz5p25teiXV-6HdTiOJSnS7u7GBzg/view?usp=sharing) | [Mirror](https://ucdavis365-my.sharepoint.com/:u:/g/personal/yongjaelee_ucdavis_edu/Eagg5RSc5hFEhp7sPtvLNyoBjhlf2feog7t8OQzHKKphjw) To evalute the model, put the corresponding weights file in the `./weights` directory and run one of the following commands. ## Quantitative Results on COCO ```Shell # Quantitatively evaluate a trained model on the entire validation set. Make sure you have COCO downloaded as above. # This should get 29.92 validation mask mAP last time I checked. python eval.py --trained_model=weights/yolact_base_54_800000.pth # Output a COCOEval json to submit to the website or to use the run_coco_eval.py script. # This command will create './results/bbox_detections.json' and './results/mask_detections.json' for detection and instance segmentation respectively. python eval.py --trained_model=weights/yolact_base_54_800000.pth --output_coco_json # You can run COCOEval on the files created in the previous command. The performance should match my implementation in eval.py. python run_coco_eval.py # To output a coco json file for test-dev, make sure you have test-dev downloaded from above and go python eval.py --trained_model=weights/yolact_base_54_800000.pth --output_coco_json --dataset=coco2017_testdev_dataset ``` ## Qualitative Results on COCO ```Shell # Display qualitative results on COCO. From here on I'll use a confidence threshold of 0.15. python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --display ``` ## Benchmarking on COCO ```Shell # Run just the raw model on the first 1k images of the validation set python eval.py --trained_model=weights/yolact_base_54_800000.pth --benchmark --max_images=1000 ``` ## Images ```Shell # Display qualitative results on the specified image. python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --image=my_image.png # Process an image and save it to another file. python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --image=input_image.png:output_image.png # Process a whole folder of images. python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --images=path/to/input/folder:path/to/output/folder ``` ## Video ```Shell # Display a video in real-time. "--video_multiframe" will process that many frames at once for improved performance. # If you want, use "--display_fps" to draw the FPS directly on the frame. python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --video_multiframe=4 --video=my_video.mp4 # Display a webcam feed in real-time. If you have multiple webcams pass the index of the webcam you want instead of 0. python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --video_multiframe=4 --video=0 # Process a video and save it to another file. This uses the same pipeline as the ones above now, so it's fast! python eval.py --trained_model=weights/yolact_base_54_800000.pth --score_threshold=0.15 --top_k=15 --video_multiframe=4 --video=input_video.mp4:output_video.mp4 ``` As you can tell, `eval.py` can do a ton of stuff. Run the `--help` command to see everything it can do. ```Shell python eval.py --help ``` # Training By default, we train on COCO. Make sure to download the entire dataset using the commands above. - To train, grab an imagenet-pretrained model and put it in `./weights`. - For Resnet101, download `resnet101_reducedfc.pth` from [here](https://drive.google.com/file/d/1tvqFPd4bJtakOlmn-uIA492g2qurRChj/view?usp=sharing). - For Resnet50, download `resnet50-19c8e357.pth` from [here](https://drive.google.com/file/d/1Jy3yCdbatgXa5YYIdTCRrSV0S9V5g1rn/view?usp=sharing). - For Darknet53, download `darknet53.pth` from [here](https://drive.google.com/file/d/17Y431j4sagFpSReuPNoFcj9h7azDTZFf/view?usp=sharing). - Run one of the training commands below. - Note that you can press ctrl+c while training and it will save an `*_interrupt.pth` file at the current iteration. - All weights are saved in the `./weights` directory by default with the file name `<conf