# [GACELA - Generative adversarial context encoder for audio inpainting](https://arxiv.org/abs/2005.05032)
We introduce GACELA, a generative adversarial network (GAN) designed to restore missing musical audio data with a duration ranging between hundreds of milliseconds to a few seconds, i.e., to perform long-gap audio inpainting. While previous work either addressed shorter gaps or relied on exemplars by copying available information from other signal parts, GACELA addresses the inpainting of long gaps in two aspects. First, it considers various time scales of audio information by relying on five parallel discriminators with increasing resolution of receptive fields. Second, it is conditioned not only on the available information surrounding the gap, i.e., the context, but also on the latent variable of the conditional GAN. This addresses the inherent multi-modality of audio inpainting at such long gaps and provides the option of user-defined inpainting. GACELA was tested in listening tests on music signals of varying complexity and gap durations ranging from 375ms to 1500ms. While our subjects were often able to detect the inpaintings, the severity of the artifacts decreased from unacceptable to mildly disturbing. GACELA represents a framework capable to integrate future improvements such as processing of more auditory-related features or more explicit musical features.
# Installation
Install the requirements with `pip install -r requirements.txt`. Since ltfatpy is not available on windows, the code can only be used with Linux or Mac.
The datasets used for the experiments are available:
| Dataset | Type | Details |
| ------------- |:-------------| -----|
| [Lakh](https://colinraffel.com/projects/lmd/) | Midi | Used LMD-matched |
| [Maestro](https://magenta.tensorflow.org/datasets/maestro) | Midi & piano | Use full dataset |
| [Free Music Archive](https://github.com/mdeff/fma)| General music | Used only rock song fom fma-small |
# Instructions
On the folder 'train paper networks' you can find the python scripts we used to train the networks in the paper. To retrain those, just change the dataset folder to where your data is stored.
To train new networks, just run `python train.py --experiment_name awesome_name --data_folder /path/to/your/mp3/or/wav/data/`. We trained GACELA for 7 days on a NVIDIA Titan pascal X, and the trained models occupy 250Mb. At inference time, GACELA's generator produces a batch of 64 gaps in 14ms.
To test GACELA, either use the `Test GACELA` notebook in the main folder, or use one of the examples provided in the `notebooks` folder.
## Resources
- The pre-print can be read on [arxiv](https://arxiv.org/abs/2005.05032).
- The paper was published by [IEEE Journal of Selected Topics in Signal Processing](https://ieeexplore.ieee.org/document/9257074).
- To hear examples please go to the [accompanying website](https://andimarafioti.github.io/GACELA/).
- The checkpoints used for the evaluation of the [paper](https://arxiv.org/abs/2005.05032) can be downloaded [here](https://zenodo.org/record/3897144). Please extract the archives in the folder `saved_results`. Alternatively, you can simply run the script `download_checkpoints.py`:
```
python download_checkpoints.py
```
### Acknowledgments
This project accompanies the research work on audio inpainting of large gaps done at the Acoustics Research Institute in Vienna collaborating with the Swiss Data Science Center. The paper was published by [IEEE Journal of Selected Topics in Signal Processing](https://ieeexplore.ieee.org/document/9257074).
We specially thank Michael Mihocic for running the experiments at the Acoustics Research Institute's laboratory during the coronavirus pandemic as well as the subjects for their participation.
没有合适的资源?快使用搜索试试~ 我知道了~
用于音频修复的生成对抗上下文编码器.zip
共226个文件
flac:93个
py:34个
scss:28个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 168 浏览量
2023-03-22
21:48:38
上传
评论
收藏 21.86MB ZIP 举报
温馨提示
用于音频修复的生成对抗上下文编码器
资源推荐
资源详情
资源评论
收起资源包目录
用于音频修复的生成对抗上下文编码器.zip (226个子文件)
main.css 58KB
fontawesome-all.min.css 55KB
detailsPage.css 1KB
tecnologias.css 735B
noscript.css 292B
idiomas.css 262B
home.css 212B
fa-solid-900.eot 188KB
fa-brands-400.eot 126KB
fa-regular-400.eot 34KB
fma-rock_real_18.flac 325KB
fma-rock_gap_18.flac 320KB
fma-rock_GAN_18.flac 314KB
gap-inContext-1.flac 297KB
gap-inContext-0.flac 296KB
4.flac 294KB
7.flac 279KB
fma-rock_gap_15.flac 275KB
fma-rock_real_15.flac 275KB
fma-rock_GAN_15.flac 265KB
8.flac 255KB
maestro-midi_gap_17.flac 252KB
3.flac 242KB
maestro-midi_real_17.flac 240KB
maestro-recordings_GAN_18.flac 240KB
maestro-recordings_GAN_18.flac 240KB
gap-inContext-62.flac 239KB
gap-inContext-61.flac 239KB
maestro-midi_gap_16.flac 238KB
6.flac 238KB
fma-rock_real_16.flac 236KB
fma-rock_gap_14.flac 235KB
fma-rock_real_14.flac 235KB
fma-rock_gap_16.flac 235KB
maestro-midi_real_16.flac 234KB
5.flac 234KB
gap-inContext-16.flac 233KB
gap-inContext-17.flac 232KB
maestro-recordings_GAN-long_18.flac 232KB
maestro-midi_GAN_17.flac 231KB
9.flac 229KB
1.flac 226KB
maestro-midi_GAN_16.flac 226KB
fma-rock_GAN_16.flac 226KB
fma-rock_GAN_14.flac 225KB
Lakh_gap_2.flac 225KB
gap-inContext-10.flac 220KB
gap-inContext-9.flac 219KB
maestro-recordings_gap_18.flac 219KB
maestro-midi_gap_22.flac 218KB
Lakh_real_2.flac 217KB
maestro-recordings_gap_21.flac 215KB
maestro-recordings_GAN_22.flac 214KB
maestro-recordings_GAN-long_22.flac 213KB
maestro-midi_real_22.flac 211KB
maestro-recordings_real_18.flac 210KB
Lakh_GAN_2.flac 209KB
Lakh_gap_4.flac 204KB
Lakh_gap_5.flac 204KB
maestro-recordings_gap_19.flac 204KB
maestro-midi_GAN_22.flac 203KB
maestro-recordings_real_22.flac 203KB
Lakh_gap_3.flac 202KB
maestro-recordings_real_18.flac 198KB
Lakh_real_3.flac 196KB
2.flac 195KB
Lakh_real_4.flac 194KB
maestro-recordings_GAN-long_21.flac 192KB
maestro-recordings_GAN_21.flac 190KB
maestro-recordings_GAN_21.flac 190KB
Lakh_real_5.flac 190KB
Lakh_GAN_4.flac 189KB
Lakh_GAN_3.flac 188KB
maestro-midi_gap_20.flac 188KB
maestro-recordings_gap_20.flac 187KB
maestro-midi_real_20.flac 185KB
Lakh_GAN_5.flac 185KB
maestro-recordings_real_21.flac 181KB
maestro-recordings_GAN-short_18.flac 180KB
maestro-recordings_real_21.flac 180KB
maestro-recordings_GAN-long_19.flac 179KB
maestro-recordings_GAN_19.flac 178KB
maestro-recordings_GAN_19.flac 178KB
maestro-midi_GAN_20.flac 176KB
maestro-recordings_GAN-short_22.flac 174KB
maestro-recordings_GAN-long_20.flac 172KB
maestro-recordings_GAN_20.flac 171KB
maestro-recordings_GAN_20.flac 171KB
maestro-recordings_real_19.flac 169KB
maestro-recordings_real_19.flac 169KB
maestro-recordings_real_20.flac 158KB
maestro-recordings_GAN-short_21.flac 157KB
maestro-recordings_GAN-short_19.flac 157KB
maestro-recordings_real_20.flac 156KB
maestro-recordings_GAN-short_20.flac 154KB
gap-1.flac 19KB
gap-0.flac 19KB
gap-62.flac 15KB
gap-16.flac 14KB
gap-61.flac 14KB
共 226 条
- 1
- 2
- 3
资源评论
快撑死的鱼
- 粉丝: 1w+
- 资源: 9154
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 基于Python的新能源承载力计算及界面设计源码 - HAINING-DG
- 基于Java的本科探索学习项目设计源码 - 本科探索
- 基于Javascript和Python的微商城项目设计源码 - MicroMall
- 基于Java的网上订餐系统设计源码 - online ordering system
- 基于Javascript的超级美眉网络资源管理应用模块设计源码
- 基于Typescript和PHP的编程知识储备库设计源码 - study-php
- Screenshot_2024-05-28-11-40-58-177_com.tencent.mm.jpg
- 基于Dart的Flutter小提琴调音器APP设计源码 - violinhelper
- 基于JavaScript和CSS的随寻订购网页设计源码 - web-order
- 基于MATLAB的声纹识别系统设计源码 - VoiceprintRecognition
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功