Python-StarGAN面向跨域图像变换的统一对抗生成网络可实现面部表情性别迁移

<p align="center"><img width="40%" src="jpg/logo.jpg" /></p> -------------------------------------------------------------------------------- This repository provides a PyTorch implementation of [StarGAN](https://arxiv.org/abs/1711.09020). StarGAN can flexibly translate an input image to any desired target domain using only a single generator and a discriminator. The demo video for StarGAN can be found [here](https://www.youtube.com/watch?v=EYjdLppmERE). <p align="center"><img width="100%" src="jpg/main.jpg" /></p> <br/> ## Paper [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://arxiv.org/abs/1711.09020) <br/> [Yunjey Choi](https://github.com/yunjey)^1,2, [Minje Choi](https://github.com/mjc92)^1,2, [Munyoung Kim](https://www.facebook.com/munyoung.kim.1291)^2,3, [Jung-Woo Ha](https://www.facebook.com/jungwoo.ha.921)², [Sung Kim](https://www.cse.ust.hk/~hunkim/)^2,4, and [Jaegul Choo](https://sites.google.com/site/jaegulchoo/)^1,2    <br/> ¹Korea University, ²Clova AI Research (NAVER Corp.), ³The College of New Jersey, ⁴HKUST <br/> IEEE Conference on Computer Vision and Pattern Recognition ([CVPR](http://cvpr2018.thecvf.com/)), 2018 (<b>Oral</b>) <br/> ## Dependencies * [Python 3.5+](https://www.continuum.io/downloads) * [PyTorch 0.4.0+](http://pytorch.org/) * [TensorFlow 1.3+](https://www.tensorflow.org/) (optional for tensorboard) <br/> ## Usage ### 1. Cloning the repository ```bash $ git clone https://github.com/yunjey/StarGAN.git $ cd StarGAN/ ``` ### 2. Downloading the dataset To download the CelebA dataset: ```bash $ bash download.sh celeba ``` To download the RaFD dataset, you must request access to the dataset from [the Radboud Faces Database website](http://www.socsci.ru.nl:8180/RaFD2/RaFD?p=main). Then, you need to create a folder structure as described [here](https://github.com/yunjey/StarGAN/blob/master/jpg/RaFD.md). ### 3. Training To train StarGAN on CelebA, run the training script below. See [here](https://github.com/yunjey/StarGAN/blob/master/jpg/CelebA.md) for a list of selectable attributes in the CelebA dataset. If you change the `selected_attrs` argument, you should also change the `c_dim` argument accordingly. ```bash $ python main.py --mode train --dataset CelebA --image_size 128 --c_dim 5 \ --sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \ --model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \ --selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young ``` To train StarGAN on RaFD: ```bash $ python main.py --mode train --dataset RaFD --image_size 128 --c_dim 8 \ --sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs \ --model_save_dir stargan_rafd/models --result_dir stargan_rafd/results ``` To train StarGAN on both CelebA and RafD: ```bash $ python main.py --mode=train --dataset Both --image_size 256 --c_dim 5 --c2_dim 8 \ --sample_dir stargan_both/samples --log_dir stargan_both/logs \ --model_save_dir stargan_both/models --result_dir stargan_both/results ``` To train StarGAN on your own dataset, create a folder structure in the same format as [RaFD](https://github.com/yunjey/StarGAN/blob/master/jpg/RaFD.md) and run the command: ```bash $ python main.py --mode train --dataset RaFD --rafd_crop_size CROP_SIZE --image_size IMG_SIZE \ --c_dim LABEL_DIM --rafd_image_dir TRAIN_IMG_DIR \ --sample_dir stargan_custom/samples --log_dir stargan_custom/logs \ --model_save_dir stargan_custom/models --result_dir stargan_custom/results ``` ### 4. Testing To test StarGAN on CelebA: ```bash $ python main.py --mode test --dataset CelebA --image_size 128 --c_dim 5 \                 --sample_dir stargan_celeba/samples --log_dir stargan_celeba/logs \ --model_save_dir stargan_celeba/models --result_dir stargan_celeba/results \ --selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young ``` To test StarGAN on RaFD: ```bash $ python main.py --mode test --dataset RaFD --image_size 128 \ --c_dim 8 --rafd_image_dir data/RaFD/test \                 --sample_dir stargan_rafd/samples --log_dir stargan_rafd/logs \ --model_save_dir stargan_rafd/models --result_dir stargan_rafd/results ``` To test StarGAN on both CelebA and RaFD: ```bash $ python main.py --mode test --dataset Both --image_size 256 --c_dim 5 --c2_dim 8 \                 --sample_dir stargan_both/samples --log_dir stargan_both/logs \ --model_save_dir stargan_both/models --result_dir stargan_both/results ``` To test StarGAN on your own dataset: ```bash $ python main.py --mode test --dataset RaFD --rafd_crop_size CROP_SIZE --image_size IMG_SIZE \ --c_dim LABEL_DIM --rafd_image_dir TEST_IMG_DIR \ --sample_dir stargan_custom/samples --log_dir stargan_custom/logs \ --model_save_dir stargan_custom/models --result_dir stargan_custom/results ``` ### 5. Pretrained model To download a pretrained model checkpoint, run the script below. The pretrained model checkpoint will be downloaded and saved into `./stargan_celeba_256/models` directory. ```bash $ bash download.sh pretrained-celeba-256x256 ``` To translate images using the pretrained model, run the evaluation script below. The translated images will be saved into `./stargan_celeba_256/results` directory. ```bash $ python main.py --mode test --dataset CelebA --image_size 256 --c_dim 5 \ --selected_attrs Black_Hair Blond_Hair Brown_Hair Male Young \ --model_save_dir='stargan_celeba_256/models' \ --result_dir='stargan_celeba_256/results' ``` <br/> ## Results ### 1. Facial Attribute Transfer on CelebA <p align="center"><img width="100%" src="jpg/result_celeba1.jpg" /></p> ### 2. Facial Expression Synthesis on RaFD <p align="center"><img width="100%" src="jpg/result_rafd.jpg" /></p> ### 3. Facial Expression Synthesis on CelebA <p align="center"><img width="100%" src="jpg/result_celeba2.jpg" /></p> <br/> ## Citation If this work is useful for your research, please cite our [paper](https://arxiv.org/abs/1711.09020): ``` @InProceedings{StarGAN2018, author = {Choi, Yunjey and Choi, Minje and Kim, Munyoung and Ha, Jung-Woo and Kim, Sunghun and Choo, Jaegul}, title = {StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation}, booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2018} } ``` <br/> ## Acknowledgement This work was mainly done while the first author did a research internship at [Clova AI Research, NAVER](https://clova.ai/en/research/research-area-detail.html?id=0). We thank all the researchers at NAVER, especially Donghyun Kwak, for insightful discussions.