AnimeGANv2.zip

## PyTorch Implementation of [AnimeGANv2](https://github.com/TachibanaYoshino/AnimeGANv2) **Updates** * `2021-10-17` Add weights for [FacePortraitV2](#additional-model-weights) * `2021-11-07` Thanks to [ak92501](https://twitter.com/ak92501), a web demo is integrated to [Huggingface Spaces](https://huggingface.co/spaces) with [Gradio](https://github.com/gradio-app/gradio). See demo: [](https://huggingface.co/spaces/akhaliq/AnimeGANv2) * `2021-11-07` Thanks to [xhlulu](https://github.com/xhlulu), the `torch.hub` model is now available. See [Torch Hub Usage](#torch-hub-usage). * `2021-11-07` Add FacePortraitV2 style demo to a telegram bot. See [@face2stickerbot](https://t.me/face2stickerbot) by [sxela](https://github.com/sxela) ## Basic Usage **Weight Conversion from the Original Repo (Requires TensorFlow 1.x)** ``` git clone https://github.com/TachibanaYoshino/AnimeGANv2 python convert_weights.py ``` **Inference** ``` python test.py --input_dir [image_folder_path] --device [cpu/cuda] ``` **Results from converted [[Paprika]](https://drive.google.com/file/d/1K_xN32uoQKI8XmNYNLTX5gDn1UnQVe5I/view?usp=sharing) style model** (input image, original tensorflow result, pytorch result from left to right) <img src="./samples/compare/1.jpg" width="960"> &nbsp; <img src="./samples/compare/2.jpg" width="960"> &nbsp; <img src="./samples/compare/3.jpg" width="960"> &nbsp; **Note:** Training code not included / Results from converted weights slightly different due to the [bilinear upsample issue](https://github.com/pytorch/pytorch/issues/10604) ## Additional Model Weights **Webtoon Face** [[ckpt]](https://drive.google.com/file/d/10T6F3-_RFOCJn6lMb-6mRmcISuYWJXGc) <details> <summary>samples</summary> Trained on <b>256x256</b> face images. Distilled from [webtoon face model](https://github.com/bryandlee/naver-webtoon-faces/blob/master/README.md#face2webtoon) with L2 + VGG + GAN Loss and CelebA-HQ images. See `test_faces.ipynb` for details. <img src="./samples/face_results.jpg" width="512"> &nbsp; </details> **Face Portrait v1** [[ckpt]](https://drive.google.com/file/d/1WK5Mdt6mwlcsqCZMHkCUSDJxN1UyFi0-) <details> <summary>samples</summary> Trained on <b>512x512</b> face images. [](https://colab.research.google.com/drive/1jCqcKekdtKzW7cxiw_bjbbfLsPh-dEds?usp=sharing)  [📺](https://youtu.be/CbMfI-HNCzw?t=317)  </details> **Face Portrait v2** [[ckpt]](https://drive.google.com/uc?id=18H3iK09_d54qEDoWIc82SyWB2xun4gjU) <details> <summary>samples</summary> Trained on <b>512x512</b> face images. Compared to v1, `🔻beautify` `🔺robustness` [](https://colab.research.google.com/drive/1jCqcKekdtKzW7cxiw_bjbbfLsPh-dEds?usp=sharing)   🦑 🎮 🔥  </details> ## Torch Hub Usage You can load Animegan v2 via `torch.hub`: ```python import torch model = torch.hub.load('bryandlee/animegan2-pytorch', 'generator').eval() # convert your image into tensor here out = model(img_tensor) ``` You can load with various configs (more details in [the torch docs](https://pytorch.org/docs/stable/hub.html)): ```python model = torch.hub.load( "bryandlee/animegan2-pytorch:main", "generator", pretrained=True, # or give URL to a pretrained model device="cuda", # or "cpu" if you don't have a GPU progress=True, # show progress ) ``` Currently, the following `pretrained` shorthands are available: ```python model = torch.hub.load("bryandlee/animegan2-pytorch:main", "generator", pretrained="celeba_distill") model = torch.hub.load("bryandlee/animegan2-pytorch:main", "generator", pretrained="face_paint_512_v1") model = torch.hub.load("bryandlee/animegan2-pytorch:main", "generator", pretrained="face_paint_512_v2") model = torch.hub.load("bryandlee/animegan2-pytorch:main", "generator", pretrained="paprika") ``` You can also load the `face2paint` util function. First, install dependencies: ``` pip install torchvision Pillow numpy ``` Then, import the function using `torch.hub`: ```python face2paint = torch.hub.load( 'bryandlee/animegan2-pytorch:main', 'face2paint', size=512, device="cpu" ) img = Image.open(...).convert("RGB") out = face2paint(model, img) ```