第三届计图Jittor人工智能挑战赛-大规模无监督语义分割.zip

# Jittor Competition: Large Scale Unsupervised Semantic Segmentation (PASS + SAM) ## Introduction  Baseline:PASS, containing four steps. 1) A randomly initialized model is trained with self-supervision of pretext tasks。 2) A pixel-attention-based clustering scheme to obtain pseudo categories and assign generated categories to each image pixel. 3) Fine-tune the pre-trained model with the generated pseudo labels to improve the segmentation quality. 4) During inference, the LUSS model assigns generated labels to each pixel of images, same to the supervised model. This repo provides four solutions to improve mIoU score based on PASS model and Segment Anything Model. Please refer to [USAGE](./USAGE.md) for details. ## Visualization <img src="vis/visualization.png"> Results in th first line comes from PASS baseline. After improvement, object localization is more accurate and multiple objects can be detected. ## Description **Adapter** <div align="center"> <img src="vis/adapter.png" height=250> </div> **Saliency map** <div align="center"> <img src="vis/saliency.png" height=160> </div> **Progressive upsampling** <div align="center"> <img src="vis/progressive_upsampling.png" height=180> </div> **Semantic voting** <div align="center"><img src="vis/sam1.png" height=200><img src="vis/sam2.png" height=200> </div> **Performance** <img src="vis/performance.png"> ## Install Project implemented on 4 2080ti gpus. Training time in total is around 2 days. #### Environment - ubuntu 20.04 LTS - python >= 3.7 - jittor >= 1.6.1 ## Train ``` bash scipts/modified_luss50_pass_jt.sh ``` ## Inference ``` python test.py ``` ## Citation ``` @article{gao2022luss, title={Large-scale Unsupervised Semantic Segmentation}, author={Gao, Shanghua and Li, Zhong-Yu and Yang, Ming-Hsuan and Cheng, Ming-Ming and Han, Junwei and Torr, Philip}, journal=TPAMI, year={2022} } ``` ## Acknowledgement Code is based on [PASS](https://github.com/LUSSeg/PASS), refering to [segment-anything](https://github.com/facebookresearch/segment-anything) and [dino](https://github.com/facebookresearch/dino).