深度学习预训练库，集成各种经典backbone，基于OpenMMLab-MMPretrain库！！！

<div align="center"> <img src="../resources/mmpt-logo.png" width="600"/> <div>&nbsp;</div> <div align="center"> <b><font size="5">OpenMMLab website</font></b> ^{<a href="https://openmmlab.com"> <i><font size="4">HOT</font></i> </a>} &nbsp;&nbsp;&nbsp;&nbsp; <b><font size="5">OpenMMLab platform</font></b> ^{<a href="https://platform.openmmlab.com"> <i><font size="4">TRY IT OUT</font></i> </a>} </div> <div>&nbsp;</div> [](https://pypi.org/project/mmpretrain) [](https://mmpretrain.readthedocs.io/en/latest/) [](https://github.com/open-mmlab/mmpretrain/actions) [](https://codecov.io/gh/open-mmlab/mmpretrain) [](https://github.com/open-mmlab/mmpretrain/blob/main/LICENSE) [](https://github.com/open-mmlab/mmpretrain/issues) [](https://github.com/open-mmlab/mmpretrain/issues) [📘 Documentation](https://mmpretrain.readthedocs.io/en/latest/) | [🛠️ Installation](https://mmpretrain.readthedocs.io/en/latest/get_started.html#installation) | [👀 Model Zoo](https://mmpretrain.readthedocs.io/en/latest/modelzoo_statistics.html) | [🆕 Update News](https://mmpretrain.readthedocs.io/en/latest/notes/changelog.html) | [🤔 Reporting Issues](https://github.com/open-mmlab/mmpretrain/issues/new/choose) <img src="https://user-images.githubusercontent.com/36138628/230307505-4727ad0a-7d71-4069-939d-b499c7e272b7.png" width="400"/> English | [简体中文](/docs/README_zh-CN.mdCN.md) </div> </div> <div align="center"> <a href="https://openmmlab.medium.com/" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/219255827-67c1a27f-f8c5-46a9-811d-5e57448c61d1.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://discord.gg/raweFPmdzG" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218347213-c080267f-cbb6-443e-8532-8e1ed9a58ea9.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://twitter.com/OpenMMLab" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218346637-d30c8a0f-3eba-4699-8131-512fb06d46db.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://www.youtube.com/openmmlab" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/218346691-ceb2116a-465a-40af-8424-9f30d2348ca9.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://space.bilibili.com/1293512903" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/219026751-d7d14cce-a7c9-4e82-9942-8375fca65b99.png" width="3%" alt="" /></a> <img src="https://user-images.githubusercontent.com/25839884/218346358-56cc8e2f-a2b8-487f-9088-32480cceabcf.png" width="3%" alt="" /> <a href="https://www.zhihu.com/people/openmmlab" style="text-decoration:none;"> <img src="https://user-images.githubusercontent.com/25839884/219026120-ba71e48b-6e94-4bd4-b4e9-b7d175b5e362.png" width="3%" alt="" /></a> </div> ## Introduction MMPreTrain is an open source pre-training toolbox based on PyTorch. It is a part of the [OpenMMLab](https://openmmlab.com/) project. The `main` branch works with **PyTorch 1.8+**. ### Major features - Various backbones and pretrained models - Rich training strategies (supervised learning, self-supervised learning, multi-modality learning etc.) - Bag of training tricks - Large-scale training configs - High efficiency and extensibility - Powerful toolkits for model analysis and experiments - Various out-of-box inference tasks. - Image Classification - Image Caption - Visual Question Answering - Visual Grounding - Retrieval (Image-To-Image, Text-To-Image, Image-To-Text) https://github.com/open-mmlab/mmpretrain/assets/26739999/e4dcd3a2-f895-4d1b-a351-fbc74a04e904 ## What's new 🌟 v1.0.0rc8 was released in 22/05/2023 - Support multiple **multi-modal** algorithms and inferencers. You can explore these features by the [gradio demo](https://github.com/open-mmlab/mmpretrain/tree/main/projects/gradio_demo)! - Add EVA-02, Dino-V2, ViT-SAM and GLIP backbones. - Register torchvision transforms into MMPretrain, you can now easily integrate torchvision's data augmentations in MMPretrain. See [the doc](https://mmpretrain.readthedocs.io/en/latest/api/data_process.html#torchvision-transforms) 🌟 v1.0.0rc7 was released in 07/04/2023 - Integrated Self-supervised learning algorithms from **MMSelfSup**, such as **MAE**, **BEiT**, etc. - Support **RIFormer**, a simple but effective vision backbone by removing token mixer. - Add t-SNE visualization. - Refactor dataset pipeline visualization. Update of previous versions - Support **LeViT**, **XCiT**, **ViG**, **ConvNeXt-V2**, **EVA**, **RevViT**, **EfficientnetV2**, **CLIP**, **TinyViT** and **MixMIM** backbones. - Reproduce the training accuracy of **ConvNeXt** and **RepVGG**. - Support confusion matrix calculation and plot. - Support **multi-task** training and testing. - Support Test-time Augmentation. - Upgrade API to get pre-defined models of MMPreTrain. - Refactor BEiT backbone and support v1/v2 inference. This release introduced a brand new and flexible training & test engine, but it's still in progress. Welcome to try according to [the documentation](https://mmpretrain.readthedocs.io/en/latest/). And there are some BC-breaking changes. Please check [the migration tutorial](https://mmpretrain.readthedocs.io/en/latest/migration.html). Please refer to [changelog](https://mmpretrain.readthedocs.io/en/latest/notes/changelog.html) for more details and other release history. ## Installation Below are quick steps for installation: ```shell conda create -n open-mmlab python=3.8 pytorch==1.10.1 torchvision==0.11.2 cudatoolkit=11.3 -c pytorch -y conda activate open-mmlab pip install openmim git clone https://github.com/open-mmlab/mmpretrain.git cd mmpretrain mim install -e . ``` Please refer to [installation documentation](https://mmpretrain.readthedocs.io/en/latest/get_started.html) for more detailed installation and dataset preparation. For multi-modality models support, please install the extra dependencies by: ```shell mim install -e ".[multimodal]" ``` ## User Guides We provided a series of tutorials about the basic usage of MMPreTrain for new users: - [Learn about Configs](https://mmpretrain.readthedocs.io/en/latest/user_guides/config.html) - [Prepare Dataset](https://mmpretrain.readthedocs.io/en/latest/user_guides/dataset_prepare.html) - [Inference with existing models](https://mmpretrain.readthedocs.io/en/latest/user_guides/inference.html) - [Train](https://mmpretrain.readthedocs.io/en/latest/user_guides/train.html) - [Test](https://mmpretrain.readthedocs.io/en/latest/user_guides/test.html) - [Downstream tasks](https://mmpretrain.readthedocs.io/en/latest/user_guides/downstream.html) For more information, please refer to [our documentation](https://mmpretrain.readthedocs.io/en/latest/). ## Model zoo Resul