# TecoGAN
This repository contains source code and materials for the TecoGAN project, i.e. code for a TEmporally COherent GAN for video super-resolution.
_Authors: Mengyu Chu, You Xie, Laura Leal-Taixe, Nils Thuerey. Technical University of Munich._
This repository so far contains the code for the TecoGAN _inference_
and _training_. Data generation, i.e., download, will follow soon.
Pre-trained models are also available below, you can find links for downloading and instructions below.
The video and pre-print of our paper can be found here:
Video: <https://www.youtube.com/watch?v=pZXFXtfd-Ak>
Preprint: <https://arxiv.org/pdf/1811.09393.pdf>
![TecoGAN teaser image](resources/teaser.jpg)
### Additional Generated Outputs
Our method generates fine details that
persist over the course of long generated video sequences. E.g., the mesh structures of the armor,
the scale patterns of the lizard, and the dots on the back of the spider highlight the capabilities of our method.
Our spatio-temporal discriminator plays a key role to guide the generator network towards producing coherent detail.
<img src="resources/tecoGAN-lizard.gif" alt="Lizard" width="900"/><br>
<img src="resources/tecoGAN-armour.gif" alt="Armor" width="900"/><br>
<img src="resources/tecoGAN-spider.gif" alt="Spider" width="600" hspace="150"/><br>
### Running the TecoGAN Model
Below you can find a quick start guide for running a trained TecoGAN model.
For further explanations of the parameters take a look at the runGan.py file.
Note: evaluation (test case 2) currently requires an Nvidia GPU with `CUDA`.
`tkinter` is also required and may be installed via the `python3-tk` package.
```bash
# Install tensorflow1.8+,
pip3 install --ignore-installed --upgrade tensorflow-gpu # or tensorflow
# Install PyTorch (only necessary for the metric evaluations) and other things...
pip3 install -r requirements.txt
# Download our TecoGAN model, the _Vid4_ and _TOS_ scenes shown in our paper and video.
python3 runGan.py 0
# Run the inference mode on the calendar scene.
# You can take a look of the parameter explanations in the runGan.py, feel free to try other scenes!
python3 runGan.py 1
# Evaluate the results with 4 metrics, PSNR, LPIPS[1], and our temporal metrics tOF and tLP with pytorch.
# Take a look at the paper for more details!
python3 runGan.py 2
```
### Train the TecoGAN Model
#### 1. Prepare the Training Data
The training and validation dataset can be downloaded with the following commands into a chosen directory `TrainingDataPath`. Note: online video downloading requires youtube-dl.
```bash
# Install youtube-dl for online video downloading
pip install --user --upgrade youtube-dl
# take a look of the parameters first:
python3 dataPrepare.py --help
# To be on the safe side, if you just want to see what will happen, the following line won't download anything,
# and will only save information into log file.
# TrainingDataPath is still important, it the directory where logs are saved: TrainingDataPath/log/logfile_mmddHHMM.txt
python3 dataPrepare.py --start_id 2000 --duration 120 --disk_path TrainingDataPath --TEST
# This will create 308 subfolders under TrainingDataPath, each with 120 frames, from 28 online videos.
# It takes a long time.
python3 dataPrepare.py --start_id 2000 --duration 120 --REMOVE --disk_path TrainingDataPath
```
Once ready, please update the parameter TrainingDataPath in runGAN.py (for case 3 and case 4), and then you can start training with the downloaded data!
Note: most of the data (272 out of 308 sequences) are the same as the ones we used for the published models, but some (36 out of 308) are not online anymore. Hence the script downloads suitable replacements.
#### 2. Train the Model
This section gives command to train a new TecoGAN model. Detail and additional parameters can be found in the runGan.py file. Note: the tensorboard gif summary requires ffmpeg.
```bash
# Install ffmpeg for the gif summary
sudo apt-get install ffmpeg # or conda install ffmpeg
# Train the TecoGAN model, based on our FRVSR model
# Please check and update the following parameters:
# - VGGPath, it uses ./model/ by default. The VGG model is ca. 500MB
# - TrainingDataPath (see above)
# - in main.py you can also adjust the output directory of the testWhileTrain() function if you like (it will write into a train/ sub directory by default)
python3 runGan.py 3
# Train without Dst, (i.e. a FRVSR model)
python3 runGan.py 4
# View log via tensorboard
tensorboard --logdir='ex_TecoGANmm-dd-hh/log' --port=8008
```
### Tensorboard GIF Summary Example
<img src="resources/gif_summary_example.gif" alt="gif_summary_example" width="600" hspace="150"/><br>
### Acknowledgements
This work was funded by the ERC Starting Grant realFlow (ERC StG-2015-637014).
Part of the code is based on LPIPS[1], Photo-Realistic SISR[2] and gif_summary[3].
### Reference
[1] [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric (LPIPS)](https://github.com/richzhang/PerceptualSimilarity)
[2] [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://github.com/brade31919/SRGAN-tensorflow.git)
[3] [gif_summary](https://colab.research.google.com/drive/1vgD2HML7Cea_z5c3kPBcsHUIxaEVDiIc)
TUM I15 <https://ge.in.tum.de/> , TUM <https://www.tum.de/>
没有合适的资源?快使用搜索试试~ 我知道了~
Python-TecoGAN时序一致GAN视频超分辨率实现
共68个文件
png:41个
py:16个
gif:4个
5星 · 超过95%的资源 需积分: 44 68 下载量 83 浏览量
2019-08-11
03:50:34
上传
评论 27
收藏 17.96MB ZIP 举报
温馨提示
This repo will contain source code and materials for the TecoGAN project, i.e. code for a TEmporally COherent GAN
资源推荐
资源详情
资源评论
收起资源包目录
Python-TecoGAN时序一致GAN视频超分辨率实现.zip (68个子文件)
TecoGAN-master
LPIPSmodels
v0.1
alex.pth 6KB
util.py 14KB
pretrained_networks.py 6KB
LPIPSsource.txt 328B
networks_basic.py 10KB
base_model.py 2KB
dist_model.py 13KB
dataPrepare.py 7KB
lib
dataloader.py 19KB
ops.py 22KB
frvsr.py 4KB
data
tst_scene_render.py 4KB
video.py 7KB
common.py 7KB
Teco.py 30KB
main.py 21KB
requirements.txt 173B
LICENSE 11KB
metrics.py 10KB
LR
calendar
0039.png 58KB
0029.png 58KB
0002.png 59KB
0007.png 59KB
0011.png 59KB
0016.png 59KB
0026.png 59KB
0010.png 59KB
0013.png 59KB
0038.png 58KB
0008.png 59KB
0005.png 59KB
0004.png 59KB
0012.png 59KB
0009.png 59KB
0036.png 58KB
0022.png 59KB
0017.png 59KB
0040.png 58KB
0019.png 59KB
0006.png 59KB
0001.png 59KB
0032.png 58KB
0015.png 59KB
0025.png 59KB
0031.png 58KB
0037.png 58KB
0023.png 59KB
0018.png 59KB
0024.png 59KB
0030.png 58KB
0041.png 58KB
0034.png 58KB
0035.png 58KB
0021.png 59KB
0027.png 58KB
0020.png 59KB
0014.png 59KB
0028.png 58KB
0033.png 58KB
0003.png 59KB
README.md 5KB
resources
teaser.jpg 117KB
tecoGAN-armour.gif 4.58MB
tecoGAN-spider.gif 1.69MB
tecoGAN-lizard.gif 5.75MB
gif_summary_example.gif 3.6MB
.gitignore 1KB
runGan.py 13KB
共 68 条
- 1
资源评论
- aiwozhonghuajin2019-11-29感谢分享,有一定帮助。
weixin_39840650
- 粉丝: 413
- 资源: 1万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功