低代码，在线表单，工作流程，智能报表，大屏设计，应用设计，自动化运维，数据治理，商业智能，任务编排。.zip

# 🦙 LaMa: Resolution-robust Large Mask Inpainting with Fourier Convolutions Official implementation by Samsung Research by Roman Suvorov, Elizaveta Logacheva, Anton Mashikhin, Anastasia Remizova, Arsenii Ashukha, Aleksei Silvestrov, Naejin Kong, Harshith Goka, Kiwoong Park, Victor Lempitsky. <p align="center" "font-size:30px;"> 🔥🔥🔥 <br> <b> LaMa generalizes surprisingly well to much higher resolutions (~2k❗️) than it saw during training (256x256), and achieves the excellent performance even in challenging scenarios, e.g. completion of periodic structures.</b> </p> [[Project page](https://saic-mdal.github.io/lama-project/)] [[arXiv](https://arxiv.org/abs/2109.07161)] [[Supplementary](https://ashukha.com/projects/lama_21/lama_supmat_2021.pdf)] [[BibTeX](https://senya-ashukha.github.io/projects/lama_21/paper.txt)] [[Casual GAN Papers Summary](https://www.casualganpapers.com/large-masks-fourier-convolutions-inpainting/LaMa-explained.html)] <p align="center"> <a href="https://colab.research.google.com/github/saic-mdal/lama/blob/master//colab/LaMa_inpainting.ipynb"> <img src="https://colab.research.google.com/assets/colab-badge.svg"/> </a> <br> Try out in Google Colab </p> <p align="center"> <img src="https://raw.githubusercontent.com/senya-ashukha/senya-ashukha.github.io/master/projects/lama_21/ezgif-4-0db51df695a8.gif" /> </p> <p align="center"> <img src="https://raw.githubusercontent.com/senya-ashukha/senya-ashukha.github.io/master/projects/lama_21/gif_for_lightning_v1_white.gif" /> </p> # Non-official 3rd party apps: (Feel free to share your app/implementation/demo by creating an issue) - [https://cleanup.pictures](https://cleanup.pictures/) - a simple interactive object removal tool by [@cyrildiagne](https://twitter.com/cyrildiagne) - [lama-cleaner](https://github.com/Sanster/lama-cleaner) by [@Sanster](https://github.com/Sanster/lama-cleaner) is a self-host version of [https://cleanup.pictures](https://cleanup.pictures/) - Integrated to [Huggingface Spaces](https://huggingface.co/spaces) with [Gradio](https://github.com/gradio-app/gradio). See demo: [](https://huggingface.co/spaces/akhaliq/lama) by [@AK391](https://github.com/AK391) - Telegram bot [@MagicEraserBot](https://t.me/MagicEraserBot) by [@Moldoteck](https://github.com/Moldoteck), [code](https://github.com/Moldoteck/MagicEraser) # Environment setup Clone the repo: `git clone https://github.com/saic-mdal/lama.git` There are three options of an environment: 1. Python virtualenv: ``` virtualenv inpenv --python=/usr/bin/python3 source inpenv/bin/activate pip install torch==1.8.0 torchvision==0.9.0 cd lama pip install -r requirements.txt ``` 2. Conda ``` % Install conda for Linux, for other OS download miniconda at https://docs.conda.io/en/latest/miniconda.html wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh -b -p $HOME/miniconda $HOME/miniconda/bin/conda init bash cd lama conda env create -f conda_env.yml conda activate lama conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch -y pip install pytorch-lightning==1.2.9 ``` 3. Docker: No actions are needed 🎉. # Inference <a name="prediction"></a> Run ``` cd lama export TORCH_HOME=$(pwd) && export PYTHONPATH=. ``` **1. Download pre-trained models** Install tool for yandex disk link extraction: ``` pip3 install wldhx.yadisk-direct ``` The best model (Places2, Places Challenge): ``` curl -L $(yadisk-direct https://disk.yandex.ru/d/ouP6l8VJ0HpMZg) -o big-lama.zip unzip big-lama.zip ``` All models (Places & CelebA-HQ): ``` curl -L $(yadisk-direct https://disk.yandex.ru/d/EgqaSnLohjuzAg) -o lama-models.zip unzip lama-models.zip ``` **2. Prepare images and masks** Download test images: ``` curl -L $(yadisk-direct https://disk.yandex.ru/d/xKQJZeVRk5vLlQ) -o LaMa_test_images.zip unzip LaMa_test_images.zip ``` <details> <summary>OR prepare your data:</summary> 1) Create masks named as `[images_name]_maskXXX[image_suffix]`, put images and masks in the same folder. - You can use the [script](#test_datasets) for random masks generation. - Check the format of the files: ``` image1_mask001.png image1.png image2_mask001.png image2.png ``` 2) Specify `image_suffix`, e.g. `.png` or `.jpg` or `_input.jpg` in `configs/prediction/default.yaml`. </details> **3. Predict** On the host machine: python3 bin/predict.py model.path=$(pwd)/big-lama indir=$(pwd)/LaMa_test_images outdir=$(pwd)/output python bin/predict.py model.path=$(pwd)/LaMa_models/lama-places/lama-fourier indir=$(pwd)/LaMa_test_images/test outdir=$(pwd)/LaMa_test_images/output **OR** in the docker The following command will pull the docker image from Docker Hub and execute the prediction script ``` bash docker/2_predict.sh $(pwd)/big-lama $(pwd)/LaMa_test_images $(pwd)/output device=cpu ``` Docker cuda: TODO # Train and Eval ⚠️ Warning: The training is not fully tested yet, e.g., did not re-training after refactoring ⚠️ Make sure you run: ``` cd lama export TORCH_HOME=$(pwd) && export PYTHONPATH=. ``` Then download models for _perceptual loss_: mkdir -p ade20k/ade20k-resnet50dilated-ppm_deepsup/ wget -P ade20k/ade20k-resnet50dilated-ppm_deepsup/ http://sceneparsing.csail.mit.edu/model/pytorch/ade20k-resnet50dilated-ppm_deepsup/encoder_epoch_20.pth ## Places ⚠️ NB: FID/SSIM/LPIPS metric values for Places that we see in LaMa paper are computed on 30000 images that we produce in evaluation section below. For more details on evaluation data check [[Section 3. Dataset splits in Supplementary](https://ashukha.com/projects/lama_21/lama_supmat_2021.pdf#subsection.3.1)] ⚠️ On the host machine: # Download data from http://places2.csail.mit.edu/download.html # Places365-Standard: Train(105GB)/Test(19GB)/Val(2.1GB) from High-resolution images section wget http://data.csail.mit.edu/places/places365/train_large_places365standard.tar wget http://data.csail.mit.edu/places/places365/val_large.tar wget http://data.csail.mit.edu/places/places365/test_large.tar # Unpack train/test/val data and create .yaml config for it bash fetch_data/places_standard_train_prepare.sh bash fetch_data/places_standard_test_val_prepare.sh # Sample images for test and viz at the end of epoch bash fetch_data/places_standard_test_val_sample.sh bash fetch_data/places_standard_test_val_gen_masks.sh # Run training python3 bin/train.py -cn lama-fourier location=places_standard # To evaluate trained model and report metrics as in our paper # we need to sample previously unseen 30k images and generate masks for them bash fetch_data/places_standard_evaluation_prepare_data.sh # Infer model on thick/thin/medium masks in 256 and 512 and run evaluation # like this: python3 bin/predict.py \ model.path=$(pwd)/experiments/<user>_<date:time>_lama-fourier_/ \ indir=$(pwd)/places_standard_dataset/evaluation/random_thick_512/ \ outdir=$(pwd)/inference/random_thick_512 model.checkpoint=last.ckpt python3 bin/evaluate_predicts.py \ $(pwd)/configs/eval2_gpu.yaml \ $(pwd)/places_standard_dataset/evaluation/random_thick_512/ \ $(pwd)/inference/random_thick_512 \ $(pwd)/inference/random_thick_512_metrics.csv Docker: TODO ## CelebA On the host machine: # Make shure you are in lama folder cd lama export TORCH_HOME=$(pwd) && export PYTHONPATH=. # Download CelebA-HQ dataset # Download data256x256.zip from https://drive.google.com/drive/folders/11Vz0fqHS2rXDb5pprgTjpD7S2BAJhi1P # unzip & split into train/test/visualization & create config for it bash fetch_data/celebahq_dataset_pr