通过自我监督促进小样本视觉学习.zip

# *Boosting Few-Shot Visual Learning with Self-Supervision* The current project page provides [pytorch](http://pytorch.org/) code that implements the following ICCV 2019 paper: **Title:** "Boosting Few-Shot Visual Learning with Self-Supervision" **Authors:** Spyros Gidaris, Andrei Bursuc, Nikos Komodakis, Patrick Pérez, and Matthieu Cord **Abstract:** Few-shot learning and self-supervised learning address different facets of the same problem: how to train a model with little or no labeled data. Few-shot learning aims for optimization methods and models that can learn efficiently to recognize patterns in the low data regime. Self-supervised learning focuses instead on unlabeled data and looks into it for the supervisory signal to feed high capacity deep neural networks. In this work we exploit the complementarity of these two domains and propose an approach for improving few-shot learning through self-supervision. We use self-supervision as an auxiliary task in a few-shot learning pipeline, enabling feature extractors to learn richer and more transferable visual representations while still using few annotated samples. Through self-supervision, our approach can be naturally extended towards using diverse unlabeled data from other datasets in the few-shot setting. We report consistent improvements across an array of architectures, datasets and self-supervision techniques. If you find the code useful in your research, please consider citing our paper: ```BibTeX @inproceedings{gidaris2019boosting, title={Boosting Few-Shot Visual Learning with Self-Supervision}, author={Gidaris, Spyros and Bursuc, Andrei and Komodakis, Nikos and P{\'e}rez, Patrick and Cord, Matthieu}, booktitle={Proceedings of the IEEE International Conference on Computer Vision}, year={2019} } ``` ### License This code is released under the MIT License (refer to the LICENSE file for details). ## Preparation ### Pre-requisites * Python 3.7 * Pytorch >= 1.0.0 * CUDA 10.0 or higher ### Installation **(1)** Clone the repo: ```bash $ git clone https://github.com/valeoai/BF3S ``` **(2)** Install this repository and the dependencies using pip: ```bash $ pip install -e ./BF3S ``` With this, you can edit the BF3S code on the fly and import function and classes of BF3S in other projects as well. **(3)** Optional. To uninstall this package, run: ```bash $ pip uninstall BF3S ``` **(4)** Create *dataset* and *experiment* directories: ```bash $ cd BF3S $ mkdir ./datasets $ mkdir ./experiments ``` You can take a look at the [Dockerfile](./Dockerfile) if you are uncertain the about steps to install this project. ### Necessary datasets. **(1) MiniImagenet** To download the MiniImagenet dataset go to this [github page](https://github.com/gidariss/FewShotWithoutForgetting) and follow the instructions there. Then, set in [bf3s/datasets/mini_imagenet_dataset.py](bf3s/datasets/mini_imagenet.py#L16) the path to where the dataset resides in your machine. **(2) tiered-MiniImagenet** To download the tiered-MiniImagenet dataset go to this [github page](https://github.com/renmengye/few-shot-ssl-public) and follow the instructions there. Then, set in [bf3s/datasets/tiered_mini_imagenet_dataset.py](bf3s/datasets/tiered_mini_imagenet.py#L18) the path to where the dataset resides in your machine. **(3) ImageNet-FS** Download the ImageNet dataset and set in [bf3s/datasets/imagenet_dataset.py](bf3s/datasets/imagenet.py#L19) the path to where the dataset resides in your machine. **(4) CIFAR-FS** The dataset will be automatically downloaded when you run the code. Set in [bf3s/datasets/cifar100_fewshot_dataset.py](bf3s/datasets/cifar100_fewshot.py#L16) the path to where the dataset should be downloaded. ### Download pre-trained models (optional). **(1)** Download the models trained on the MiniImageNet dataset. ```bash # Run from the BF3S directory $ mkdir ./experiments/miniImageNet $ cd ./experiments/miniImageNet # WRN-28-10-based Cosine Classifier (CC) with rotation prediction self-supervision model. $ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip $ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip # WRN-28-10-based CC with rotation prediction self-supervision model trained with extra unlabeled images from tiered-MiniImageNet. $ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled.zip $ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled.zip # WRN-28-10-based CC with location prediction self-supervision model. $ wget https://github.com/valeoai/BF3S/releases/download/0.1.0/WRNd28w10CosineClassifierLocSelfsupervision.zip $ unzip WRNd28w10CosineClassifierLocSelfsupervision.zip $ cd ../../ ``` **(2)** Download the model trained on the CIFAR-FS dataset. ```bash # Run from the BF3S directory $ mkdir ./experiments/cifar $ cd ./experiments/cifar # WRN-28-10-based CC with rotation prediction self-supervision model. $ wget https://github.com/valeoai/BF3S/releases/download/0.2.0/WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip $ unzip WRNd28w10CosineClassifierRotAugRotSelfsupervision.zip $ cd ../../ ``` **(3)** Download the model trained on the ImageNet-FS dataset. ```bash # Run from the BF3S directory $ mkdir ./experiments/ImageNet $ cd ./experiments/ImageNet # ResNet10-based CC with rotation prediction self-supervision model. $ wget https://github.com/valeoai/BF3S/releases/download/0.3.0/ResNet10CosineClassifierRotSelfsupervision.zip $ unzip ResNet10CosineClassifierRotSelfsupervision.zip $ cd ../../ ``` ## Train and test Cosine Classifier (CC) based few-shot models [3] with self-supervision. ### MiniImageNet: CC model with rotation prediction self-supervision. To train and test the WRN-28-10 based Cosine Classifier model with rotation prediction self-supervision run (you can skip the training step if you have downloaded the pre-trained model): ```bash # Run from the BF3S directory # Train the model. $ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision # Test the model on the 1-shot setting. $ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=1 --num_episodes=2000 # Expected 5-way classification accuracy: 62.81% with confidence interval +/- 0.46% # Test the model on the 5-shot setting. $ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision --num_novel=5 --num_train=5 --num_episodes=2000 # Expected 5-way classification accuracy: 80.00% with confidence interval +/- 0.34% ``` Note that the configuration file (of the above experiment) specified by the `config` variable is located here: `./config/miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision.py`. All the experiment configuration files are placed in the `./config/` directory. ### MiniImageNet: CC model with rotation prediction self-supervision and exploiting unlabeled images from tiered-MiniImageNet. To train and test the WRN-28-10 based CC few-shot model with rotation prediction self-supervision which also exploits (with the *semi-supervised learning* setting) unlabeled images from tiered-MiniImageNet, run (you can skip the training step if you have downloaded the pre-trained model): ```bash # Run from the BF3S directory # Train the model. $ python scripts/train_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled # Test the model on the 1-shot setting. $ python scripts/test_fewshot.py --config=miniImageNet/WRNd28w10CosineClassifierRotAugRotSelfsupervision_SemisupervisedTieredUnlabeled --num_novel=5 --num_train=1 --num_episodes=2000 # Expected 5-way classification accuracy: 64.03% with confidence interval +/- 0.46% # Test the model on the 5-shot