[![License CC BY-NC-SA 4.0](https://img.shields.io/badge/license-CC4.0-blue.svg)](https://raw.githubusercontent.com/nvlabs/SPADE/master/LICENSE.md)
![Python 3.6](https://img.shields.io/badge/python-3.6-green.svg)
# Semantic Image Synthesis with SPADE
![GauGAN demo](https://nvlabs.github.io/SPADE//images/ocean.gif)
### [Project page](https://nvlabs.github.io/SPADE/) | [Paper](https://arxiv.org/abs/1903.07291) | [Online Interactive Demo of GauGAN](https://www.nvidia.com/en-us/research/ai-playground/) | [GTC 2019 demo](https://youtu.be/p5U4NgVGAwg) | [Youtube Demo of GauGAN](https://youtu.be/MXWm6w4E5q0)
Semantic Image Synthesis with Spatially-Adaptive Normalization.<br>
[Taesung Park](http://taesung.me/), [Ming-Yu Liu](http://mingyuliu.net/), [Ting-Chun Wang](https://tcwang0509.github.io/), and [Jun-Yan Zhu](http://people.csail.mit.edu/junyanz/).<br>
In CVPR 2019 (Oral).
### [License](https://raw.githubusercontent.com/nvlabs/SPADE/master/LICENSE.md)
Copyright (C) 2019 NVIDIA Corporation.
All rights reserved.
Licensed under the [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode) (**Attribution-NonCommercial-ShareAlike 4.0 International**)
The code is released for academic research use only. For commercial use, please contact [researchinquiries@nvidia.com](researchinquiries@nvidia.com).
## Installation
Clone this repo.
```bash
git clone https://github.com/NVlabs/SPADE.git
cd SPADE/
```
This code requires PyTorch 1.0 and python 3+. Please install dependencies by
```bash
pip install -r requirements.txt
```
This code also requires the Synchronized-BatchNorm-PyTorch rep.
```
cd models/networks/
git clone https://github.com/vacancy/Synchronized-BatchNorm-PyTorch
cp -rf Synchronized-BatchNorm-PyTorch/sync_batchnorm .
cd ../../
```
To reproduce the results reported in the paper, you would need an NVIDIA DGX1 machine with 8 V100 GPUs.
## Dataset Preparation
For COCO-Stuff, Cityscapes or ADE20K, the datasets must be downloaded beforehand. Please download them on the respective webpages. In the case of COCO-stuff, we put a few sample images in this code repo.
**Preparing COCO-Stuff Dataset**. The dataset can be downloaded [here](https://github.com/nightrome/cocostuff). In particular, you will need to download train2017.zip, val2017.zip, stuffthingmaps_trainval2017.zip, and annotations_trainval2017.zip. The images, labels, and instance maps should be arranged in the same directory structure as in `datasets/coco_stuff/`. In particular, we used an instance map that combines both the boundaries of "things instance map" and "stuff label map". To do this, we used a simple script `datasets/coco_generate_instance_map.py`. Please install `pycocotools` using `pip install pycocotools` and refer to the script to generate instance maps.
**Preparing ADE20K Dataset**. The dataset can be downloaded [here](http://data.csail.mit.edu/places/ADEchallenge/ADEChallengeData2016.zip), which is from [MIT Scene Parsing BenchMark](http://sceneparsing.csail.mit.edu/). After unzipping the datgaset, put the jpg image files `ADEChallengeData2016/images/` and png label files `ADEChallengeData2016/annotatoins/` in the same directory.
There are different modes to load images by specifying `--preprocess_mode` along with `--load_size`. `--crop_size`. There are options such as `resize_and_crop`, which resizes the images into square images of side length `load_size` and randomly crops to `crop_size`. `scale_shortside_and_crop` scales the image to have a short side of length `load_size` and crops to `crop_size` x `crop_size` square. To see all modes, please use `python train.py --help` and take a look at `data/base_dataset.py`. By default at the training phase, the images are randomly flipped horizontally. To prevent this use `--no_flip`.
## Generating Images Using Pretrained Model
Once the dataset is ready, the result images can be generated using pretrained models.
1. Download the tar of the pretrained models from the [Google Drive Folder](https://drive.google.com/file/d/12gvlTbMvUcJewQlSEaZdeb2CdOB-b8kQ/view?usp=sharing), save it in 'checkpoints/', and run
```
cd checkpoints
tar xvf checkpoints.tar.gz
cd ../
```
2. Generate images using the pretrained model.
```bash
python test.py --name [type]_pretrained --dataset_mode [dataset] --dataroot [path_to_dataset]
```
`[type]_pretrained` is the directory name of the checkpoint file downloaded in Step 1, which should be one of `coco_pretrained`, `ade20k_pretrained`, and `cityscapes_pretrained`. `[dataset]` can be one of `coco`, `ade20k`, and `cityscapes`, and `[path_to_dataset]`, is the path to the dataset. If you are running on CPU mode, append `--gpu_ids -1`.
3. The outputs images are stored at `./results/[type]_pretrained/` by default. You can view them using the autogenerated HTML file in the directory.
## Generating Landscape Image using GauGAN
In the paper and the demo video, we showed GauGAN, our interactive app that generates realistic landscape images from the layout users draw. The model was trained on landscape images scraped from Flickr.com. We released an online demo that has the same features. Please visit [https://www.nvidia.com/en-us/research/ai-playground/](https://www.nvidia.com/en-us/research/ai-playground/). The model weights are not released.
## Training New Models
New models can be trained with the following commands.
1. Prepare dataset. To train on the datasets shown in the paper, you can download the datasets and use `--dataset_mode` option, which will choose which subclass of `BaseDataset` is loaded. For custom datasets, the easiest way is to use `./data/custom_dataset.py` by specifying the option `--dataset_mode custom`, along with `--label_dir [path_to_labels] --image_dir [path_to_images]`. You also need to specify options such as `--label_nc` for the number of label classes in the dataset, `--contain_dontcare_label` to specify whether it has an unknown label, or `--no_instance` to denote the dataset doesn't have instance maps.
2. Train.
```bash
# To train on the Facades or COCO dataset, for example.
python train.py --name [experiment_name] --dataset_mode facades --dataroot [path_to_facades_dataset]
python train.py --name [experiment_name] --dataset_mode coco --dataroot [path_to_coco_dataset]
# To train on your own custom dataset
python train.py --name [experiment_name] --dataset_mode custom --label_dir [path_to_labels] -- image_dir [path_to_images] --label_nc [num_labels]
```
There are many options you can specify. Please use `python train.py --help`. The specified options are printed to the console. To specify the number of GPUs to utilize, use `--gpu_ids`. If you want to use the second and third GPUs for example, use `--gpu_ids 1,2`.
To log training, use `--tf_log` for Tensorboard. The logs are stored at `[checkpoints_dir]/[name]/logs`.
## Testing
Testing is similar to testing pretrained models.
```bash
python test.py --name [name_of_experiment] --dataset_mode [dataset_mode] --dataroot [path_to_dataset]
```
Use `--results_dir` to specify the output directory. `--how_many` will specify the maximum number of images to generate. By default, it loads the latest checkpoint. It can be changed using `--which_epoch`.
## Code Structure
- `train.py`, `test.py`: the entry point for training and testing.
- `trainers/pix2pix_trainer.py`: harnesses and reports the progress of training.
- `models/pix2pix_model.py`: creates the networks, and compute the losses
- `models/networks/`: defines the architecture of all models
- `options/`: creates option lists using `argparse` package. More individuals are dynamically added in other files as well. Please see the section below.
- `data/`: defines the class for loading images and label maps.
## Options
This code repo contains many options. Some options belong to only one specific model, and some options have different default values depending on other options. To address this, the `BaseOptio
没有合适的资源?快使用搜索试试~ 我知道了~
Python-SemanticImageSynthesiswithSPADE空间自适应归一化的语义图像合成
共121个文件
png:45个
py:34个
jpg:27个
需积分: 10 5 下载量 53 浏览量
2019-08-11
06:52:08
上传
评论
收藏 21.9MB ZIP 举报
温馨提示
Semantic Image Synthesis with SPADE:空间自适应归一化的语义图像合成
资源推荐
资源详情
资源评论
收起资源包目录
Python-SemanticImageSynthesiswithSPADE空间自适应归一化的语义图像合成 (121个子文件)
glab.css 3KB
ocean.gif 1012KB
treepond.gif 812KB
.gitignore 56B
index.html 10KB
demo.html 91B
teaser_high_res.jpg 957KB
flickr.jpg 609KB
teaser.jpg 554KB
coco_comparison.jpg 527KB
000000138805.jpg 213KB
000000001818.jpg 198KB
000000197384.jpg 194KB
000000029286.jpg 182KB
000000350505.jpg 179KB
000000001268.jpg 177KB
000000284465.jpg 166KB
000000000139.jpg 158KB
000000184101.jpg 149KB
000000426773.jpg 144KB
000000001584.jpg 135KB
000000000785.jpg 131KB
000000475177.jpg 111KB
000000001993.jpg 108KB
000000017914.jpg 100KB
000000001490.jpg 99KB
000000371376.jpg 97KB
000000580986.jpg 93KB
000000500044.jpg 92KB
paper_thumbnail.jpg 63KB
000000203744.jpg 51KB
teaser_fb.jpg 49KB
000000001503.jpg 15KB
b5m.js 14KB
lib.js 6KB
popup.js 1KB
LICENSE.md 19KB
README.md 1B
README.md 9KB
paper_spade_arxiv.pdf 5.57MB
teaser_high_res_uncompressed.png 7.81MB
teaser_high_res_uncompressed_small.png 1.27MB
method.png 175KB
000000284465.png 11KB
000000284465.png 9KB
000000350505.png 9KB
000000000139.png 8KB
000000350505.png 8KB
000000475177.png 8KB
000000001268.png 8KB
000000000139.png 8KB
000000475177.png 8KB
000000001268.png 7KB
000000029286.png 7KB
000000029286.png 7KB
000000426773.png 7KB
000000426773.png 6KB
000000001584.png 5KB
000000001993.png 5KB
000000500044.png 5KB
000000001584.png 5KB
000000500044.png 5KB
000000001993.png 5KB
000000371376.png 5KB
000000371376.png 5KB
000000197384.png 4KB
000000001818.png 4KB
000000197384.png 4KB
000000017914.png 4KB
000000017914.png 4KB
000000580986.png 4KB
000000580986.png 4KB
000000000785.png 3KB
000000000785.png 3KB
000000138805.png 3KB
000000001818.png 3KB
000000138805.png 3KB
000000203744.png 3KB
000000001503.png 2KB
000000203744.png 2KB
000000001503.png 2KB
000000001490.png 2KB
000000001490.png 2KB
000000184101.png 1KB
000000184101.png 1KB
pix2pix_model.py 10KB
util.py 9KB
base_options.py 9KB
generator.py 7KB
visualizer.py 7KB
coco.py 5KB
loss.py 5KB
architecture.py 4KB
normalization.py 4KB
discriminator.py 4KB
base_dataset.py 4KB
pix2pix_dataset.py 4KB
train_options.py 3KB
image_folder.py 3KB
iter_counter.py 3KB
共 121 条
- 1
- 2
资源评论
weixin_39840924
- 粉丝: 492
- 资源: 1万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功