![mockingbird](https://user-images.githubusercontent.com/12797292/131216767-6eb251d6-14fc-4951-8324-2722f0cd4c63.jpg)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)](http://choosealicense.com/licenses/mit/)
> English | [涓枃](README-CN.md)
## Features
馃實 **Chinese** supported mandarin and tested with multiple datasets: aidatatang_200zh, magicdata, aishell3, data_aishell, and etc.
馃ぉ **PyTorch** worked for pytorch, tested in version of 1.9.0(latest in August 2021), with GPU Tesla T4 and GTX 2060
馃實 **Windows + Linux** run in both Windows OS and linux OS (even in M1 MACOS)
馃ぉ **Easy & Awesome** effect with only newly-trained synthesizer, by reusing the pretrained encoder/vocoder
馃實 **Webserver Ready** to serve your result with remote calling
### [DEMO VIDEO](https://www.bilibili.com/video/BV17Q4y1B7mY/)
## Quick Start
### 1. Install Requirements
> Follow the original repo to test if you got all environment ready.
**Python 3.7 or higher ** is needed to run the toolbox.
* Install [PyTorch](https://pytorch.org/get-started/locally/).
> If you get an `ERROR: Could not find a version that satisfies the requirement torch==1.9.0+cu102 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2 )` This error is probably due to a low version of python, try using 3.9 and it will install successfully
* Install [ffmpeg](https://ffmpeg.org/download.html#get-packages).
* Run `pip install -r requirements.txt` to install the remaining necessary packages.
* Install webrtcvad `pip install webrtcvad-wheels`(If you need)
> Note that we are using the pretrained encoder/vocoder but synthesizer, since the original model is incompatible with the Chinese sympols. It means the demo_cli is not working at this moment.
### 2. Prepare your models
You can either train your models or use existing ones:
#### 2.1. Train synthesizer with your dataset
* Download dataset and unzip: make sure you can access all .wav in folder
* Preprocess with the audios and the mel spectrograms:
`python pre.py <datasets_root>`
Allowing parameter `--dataset {dataset}` to support aidatatang_200zh, magicdata, aishell3, data_aishell, etc.If this parameter is not passed, the default dataset will be aidatatang_200zh.
* Train the synthesizer:
`python synthesizer_train.py mandarin <datasets_root>/SV2TTS/synthesizer`
* Go to next step when you see attention line show and loss meet your need in training folder *synthesizer/saved_models/*.
#### 2.2 Use pretrained model of synthesizer
> Thanks to the community, some models will be shared:
| author | Download link | Preview Video | Info |
| --- | ----------- | ----- |----- |
| @myself | https://pan.baidu.com/s/1fMh9IlgKJlL2PIiRTYDUvw [Baidu](https://pan.baidu.com/s/1fMh9IlgKJlL2PIiRTYDUvw) code锛歰m7f | | 25k steps trained by multiple datasets
|@FawenYo | https://drive.google.com/file/d/1H-YGOUHpmqKxJ9FRc6vAjPuqQki24UbC/view?usp=sharing https://u.teknik.io/AYxWf.pt | [input](https://github.com/babysor/MockingBird/wiki/audio/self_test.mp3) [output](https://github.com/babysor/MockingBird/wiki/audio/export.wav) | 200k steps with local accent of Taiwan, plz use w/ code before 10-22-2021
|@miven| https://pan.baidu.com/s/1PI-hM3sn5wbeChRryX-RCQ code锛�2021 | https://www.bilibili.com/video/BV1uh411B7AD/ | plz use w/ code before 10-22-2021
#### 2.3 Train vocoder (Optional)
> note: vocoder has little difference in effect, so you may not need to train a new one.
* Preprocess the data:
`python vocoder_preprocess.py <datasets_root> -m <synthesizer_model_path>`
> `<datasets_root>` replace with your dataset root锛宍<synthesizer_model_path>`replace with directory of your best trained models of sythensizer, e.g. *sythensizer\saved_mode\xxx*
* Train the wavernn vocoder:
`python vocoder_train.py mandarin <datasets_root>`
* Train the hifigan vocoder
`python vocoder_train.py mandarin <datasets_root> hifigan`
### 3. Launch
#### 3.1 Using the web server
You can then try to run:`python web.py` and open it in browser, default as `http://localhost:8080`
#### 3.2 Using the Toolbox
You can then try the toolbox:
`python demo_toolbox.py -d <datasets_root>`
## Reference
> This repository is forked from [Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning) which only support English.
| URL | Designation | Title | Implementation source |
| --- | ----------- | ----- | --------------------- |
| [1803.09017](https://arxiv.org/abs/1803.09017) | GlobalStyleToken (synthesizer)| Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis | This repo |
| [2010.05646](https://arxiv.org/abs/2010.05646) | HiFi-GAN (vocoder)| Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis | This repo |
|[**1806.04558**](https://arxiv.org/pdf/1806.04558.pdf) | **SV2TTS** | **Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis** | This repo |
|[1802.08435](https://arxiv.org/pdf/1802.08435.pdf) | WaveRNN (vocoder) | Efficient Neural Audio Synthesis | [fatchord/WaveRNN](https://github.com/fatchord/WaveRNN) |
|[1703.10135](https://arxiv.org/pdf/1703.10135.pdf) | Tacotron (synthesizer) | Tacotron: Towards End-to-End Speech Synthesis | [fatchord/WaveRNN](https://github.com/fatchord/WaveRNN)
|[1710.10467](https://arxiv.org/pdf/1710.10467.pdf) | GE2E (encoder)| Generalized End-To-End Loss for Speaker Verification | This repo |
## F Q&A
#### 1.Where can I download the dataset?
| Dataset | Original Source | Alternative Sources |
| --- | ----------- | ---------------|
| aidatatang_200zh | [OpenSLR](http://www.openslr.org/62/) | [Google Drive](https://drive.google.com/file/d/110A11KZoVe7vy6kXlLb6zVPLb_J91I_t/view?usp=sharing) |
| magicdata | [OpenSLR](http://www.openslr.org/68/) | [Google Drive (Dev set)](https://drive.google.com/file/d/1g5bWRUSNH68ycC6eNvtwh07nX3QhOOlo/view?usp=sharing) |
| aishell3 | [OpenSLR](https://www.openslr.org/93/) | [Google Drive](https://drive.google.com/file/d/1shYp_o4Z0X0cZSKQDtFirct2luFUwKzZ/view?usp=sharing) |
| data_aishell | [OpenSLR](https://www.openslr.org/33/) | |
> After unzip aidatatang_200zh, you need to unzip all the files under `aidatatang_200zh\corpus\train`
#### 2.What is`<datasets_root>`?
If the dataset path is `D:\data\aidatatang_200zh`,then `<datasets_root>` is`D:\data`
#### 3.Not enough VRAM
Train the synthesizer锛歛djust the batch_size in `synthesizer/hparams.py`
```
//Before
tts_schedule = [(2, 1e-3, 20_000, 12), # Progressive training schedule
(2, 5e-4, 40_000, 12), # (r, lr, step, batch_size)
(2, 2e-4, 80_000, 12), #
(2, 1e-4, 160_000, 12), # r = reduction factor (# of mel frames
(2, 3e-5, 320_000, 12), # synthesized for each decoder iteration)
(2, 1e-5, 640_000, 12)], # lr = learning rate
//After
tts_schedule = [(2, 1e-3, 20_000, 8), # Progressive training schedule
(2, 5e-4, 40_000, 8), # (r, lr, step, batch_size)
(2, 2e-4, 80_000, 8), #
(2, 1e-4, 160_000, 8), # r = reduction factor (# of mel frames
(2, 3e-5, 320_000, 8), # synthesized for each decoder iteration)
(2, 1e-5, 640_000, 8)], # lr = learning rate
```
Train Vocoder-Preprocess the data锛歛djust the batch_size in `synthesizer/hparams.py`
```
//Before
### Data Preprocessing
max_mel_frames = 900,
rescale = True,
rescaling_max = 0.9,
synthesis_batch_size = 16, # For vocoder preprocessing and inference.
//After
### Data Preprocessing
max_mel_frames = 900,
rescale = True,
rescaling_max = 0.9,
synthesis_batch_size = 8, # For vocoder preprocessing and inference.
```
Train Vocoder-Train the vocoder锛歛djust the batch_size in `vocoder/wavernn/hparams.py`
```
//Before
# Training
voc_batch_size = 100
voc_lr = 1e-4
voc_gen_at_checkpoint = 5
v
没有合适的资源?快使用搜索试试~ 我知道了~
MockingBird是一个可以克隆您的声音并生成任意语音内容的AI拟声系统 .rar
共108个文件
py:77个
js:8个
txt:4个
需积分: 5 1 下载量 65 浏览量
2023-07-05
15:43:51
上传
评论 2
收藏 109.33MB RAR 举报
温馨提示
使用python 创建高级键盘记录程序 -Crash Course 2020 使用python创建高级键盘记录程序,该功能可以发送电子邮件,获取屏幕截图,录制麦克风,获取计算机信息和剪贴板。所有功能共同创建键盘记录器和类似间谍软件的有效负载。 加快Python的异国期权定价 在金融领域,有时可以将计算效率直接转换为交易利润。Quants面临着权衡研究效率和计算效率的挑战。使用Python可以产生简洁的研究代码,从而提高研究效率。但是,众所周知,普通的Python代码很慢并且不适合生产。在本文中,我将探讨如何使用Python GPU库在异国期权定价领域实现最先进的性能。 开始用Python的API收集数据 对Python的API进行了友好(且并非完全令人恐惧)的介绍。 使用Keras和TensorFlow进行基于内容的图像检索的自动编码器 在本教程中,您将学习如何使用卷积自动编码器通过Keras和TensorFlow创建基于内容的图像检索系统(即图像搜索引擎)。 从头开始使用Python构建神经网络(第1部分) 编写每一行代码,并理解其工作原理。
资源推荐
资源详情
资源评论
收起资源包目录
MockingBird是一个可以克隆您的声音并生成任意语音内容的AI拟声系统 .rar (108个子文件)
DOCKERFILE 162B
.gitattributes 26B
.gitignore 289B
说明.htm 4KB
index.html 16KB
mp3-engine.js 472KB
eruda.min.js 398KB
jquery.js 282KB
recorder-core.js 32KB
mp3.js 12KB
frequency.histogram.view.js 12KB
lib.fft.js 3KB
wav.js 2KB
launch.json 1KB
config_16k_.json 631B
settings.json 45B
README-CN.md 12KB
README.md 9KB
CODE_OF_CONDUCT.md 5KB
java历史进程.pdf 214KB
mockingbird.png 89KB
bird-sm.png 40KB
bird.png 39KB
mb.png 6KB
pretrained.pt 51.35MB
g_hifigan.pt 49.58MB
pretrained.pt 16.3MB
ui.py 25KB
tacotron.py 21KB
fatchord_version.py 15KB
__init__.py 14KB
train.py 13KB
train.py 11KB
demo_cli.py 11KB
models.py 10KB
logmmse.py 9KB
preprocess.py 8KB
inference.py 8KB
audio.py 8KB
inference.py 7KB
deepmind_version.py 7KB
meldataset.py 7KB
visualizations.py 7KB
model.py 6KB
preprocess.py 5KB
train.py 5KB
hparams.py 5KB
__init__.py 5KB
distribution.py 5KB
audio.py 5KB
train.py 5KB
global_style_token.py 5KB
synthesize.py 4KB
preprocess_speaker.py 4KB
pre.py 4KB
vocoder_train.py 3KB
plot.py 3KB
synthesizer_dataset.py 3KB
display.py 3KB
vocoder_dataset.py 3KB
synthesizer_preprocess_audio.py 3KB
encoder_preprocess.py 3KB
vocoder_preprocess.py 3KB
cleaners.py 2KB
encoder_train.py 2KB
audio.py 2KB
text.py 2KB
speaker_verification_dataset.py 2KB
numbers.py 2KB
demo_toolbox.py 2KB
_cmudict.py 2KB
synthesizer_train.py 2KB
speaker.py 2KB
hparams.py 2KB
inference.py 2KB
inference.py 2KB
profiler.py 1KB
audio.py 1KB
random_cycler.py 1KB
utils.py 1KB
synthesizer_preprocess_embeds.py 1KB
argutils.py 1KB
__init__.py 1KB
config.py 1KB
gen_wavernn.py 1001B
params_data.py 901B
utterance.py 855B
synthesizer.py 805B
symbols.py 779B
modelutils.py 767B
speaker_batch.py 616B
preprocess_transcript.py 596B
env.py 394B
__init__.py 358B
web.py 295B
gst_hyperparameters.py 261B
default.py 248B
utterance.py 222B
params_model.py 196B
__init__.py 181B
共 108 条
- 1
- 2
资源评论
野生的狒狒
- 粉丝: 1916
- 资源: 1884
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功