![mockingbird](https://user-images.githubusercontent.com/12797292/131216767-6eb251d6-14fc-4951-8324-2722f0cd4c63.jpg)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)](http://choosealicense.com/licenses/mit/)
> English | [涓枃](README-CN.md)
## Features
馃實 **Chinese** supported mandarin and tested with multiple datasets: aidatatang_200zh, magicdata, aishell3, data_aishell, and etc.
馃ぉ **PyTorch** worked for pytorch, tested in version of 1.9.0(latest in August 2021), with GPU Tesla T4 and GTX 2060
馃實 **Windows + Linux** run in both Windows OS and linux OS (even in M1 MACOS)
馃ぉ **Easy & Awesome** effect with only newly-trained synthesizer, by reusing the pretrained encoder/vocoder
馃實 **Webserver Ready** to serve your result with remote calling
### [DEMO VIDEO](https://www.bilibili.com/video/BV17Q4y1B7mY/)
## Quick Start
### 1. Install Requirements
> Follow the original repo to test if you got all environment ready.
**Python 3.7 or higher ** is needed to run the toolbox.
* Install [PyTorch](https://pytorch.org/get-started/locally/).
> If you get an `ERROR: Could not find a version that satisfies the requirement torch==1.9.0+cu102 (from versions: 0.1.2, 0.1.2.post1, 0.1.2.post2 )` This error is probably due to a low version of python, try using 3.9 and it will install successfully
* Install [ffmpeg](https://ffmpeg.org/download.html#get-packages).
* Run `pip install -r requirements.txt` to install the remaining necessary packages.
* Install webrtcvad `pip install webrtcvad-wheels`(If you need)
> Note that we are using the pretrained encoder/vocoder but synthesizer, since the original model is incompatible with the Chinese sympols. It means the demo_cli is not working at this moment.
### 2. Prepare your models
You can either train your models or use existing ones:
#### 2.1 Train encoder with your dataset (Optional)
* Preprocess with the audios and the mel spectrograms:
`python encoder_preprocess.py <datasets_root>` Allowing parameter `--dataset {dataset}` to support the datasets you want to preprocess. Only the train set of these datasets will be used. Possible names: librispeech_other, voxceleb1, voxceleb2. Use comma to sperate multiple datasets.
* Train the encoder: `python encoder_train.py my_run <datasets_root>/SV2TTS/encoder`
> For training, the encoder uses visdom. You can disable it with `--no_visdom`, but it's nice to have. Run "visdom" in a separate CLI/process to start your visdom server.
#### 2.2 Train synthesizer with your dataset
* Download dataset and unzip: make sure you can access all .wav in folder
* Preprocess with the audios and the mel spectrograms:
`python pre.py <datasets_root>`
Allowing parameter `--dataset {dataset}` to support aidatatang_200zh, magicdata, aishell3, data_aishell, etc.If this parameter is not passed, the default dataset will be aidatatang_200zh.
* Train the synthesizer:
`python synthesizer_train.py mandarin <datasets_root>/SV2TTS/synthesizer`
* Go to next step when you see attention line show and loss meet your need in training folder *synthesizer/saved_models/*.
#### 2.3 Use pretrained model of synthesizer
> Thanks to the community, some models will be shared:
| author | Download link | Preview Video | Info |
| --- | ----------- | ----- |----- |
| @author | https://pan.baidu.com/s/1iONvRxmkI-t1nHqxKytY3g [Baidu](https://pan.baidu.com/s/1iONvRxmkI-t1nHqxKytY3g) 4j5d | | 75k steps trained by multiple datasets
| @author | https://pan.baidu.com/s/1fMh9IlgKJlL2PIiRTYDUvw [Baidu](https://pan.baidu.com/s/1fMh9IlgKJlL2PIiRTYDUvw) code锛歰m7f | | 25k steps trained by multiple datasets, only works under version 0.0.1
|@FawenYo | https://drive.google.com/file/d/1H-YGOUHpmqKxJ9FRc6vAjPuqQki24UbC/view?usp=sharing https://u.teknik.io/AYxWf.pt | [input](https://github.com/babysor/MockingBird/wiki/audio/self_test.mp3) [output](https://github.com/babysor/MockingBird/wiki/audio/export.wav) | 200k steps with local accent of Taiwan, only works under version 0.0.1
|@miven| https://pan.baidu.com/s/1PI-hM3sn5wbeChRryX-RCQ code锛�2021 | https://www.bilibili.com/video/BV1uh411B7AD/ | only works under version 0.0.1
#### 2.4 Train vocoder (Optional)
> note: vocoder has little difference in effect, so you may not need to train a new one.
* Preprocess the data:
`python vocoder_preprocess.py <datasets_root> -m <synthesizer_model_path>`
> `<datasets_root>` replace with your dataset root锛宍<synthesizer_model_path>`replace with directory of your best trained models of sythensizer, e.g. *sythensizer\saved_mode\xxx*
* Train the wavernn vocoder:
`python vocoder_train.py mandarin <datasets_root>`
* Train the hifigan vocoder
`python vocoder_train.py mandarin <datasets_root> hifigan`
### 3. Launch
#### 3.1 Using the web server
You can then try to run:`python web.py` and open it in browser, default as `http://localhost:8080`
#### 3.2 Using the Toolbox
You can then try the toolbox:
`python demo_toolbox.py -d <datasets_root>`
## Reference
> This repository is forked from [Real-Time-Voice-Cloning](https://github.com/CorentinJ/Real-Time-Voice-Cloning) which only support English.
| URL | Designation | Title | Implementation source |
| --- | ----------- | ----- | --------------------- |
| [1803.09017](https://arxiv.org/abs/1803.09017) | GlobalStyleToken (synthesizer)| Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis | This repo |
| [2010.05646](https://arxiv.org/abs/2010.05646) | HiFi-GAN (vocoder)| Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis | This repo |
|[**1806.04558**](https://arxiv.org/pdf/1806.04558.pdf) | **SV2TTS** | **Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis** | This repo |
|[1802.08435](https://arxiv.org/pdf/1802.08435.pdf) | WaveRNN (vocoder) | Efficient Neural Audio Synthesis | [fatchord/WaveRNN](https://github.com/fatchord/WaveRNN) |
|[1703.10135](https://arxiv.org/pdf/1703.10135.pdf) | Tacotron (synthesizer) | Tacotron: Towards End-to-End Speech Synthesis | [fatchord/WaveRNN](https://github.com/fatchord/WaveRNN)
|[1710.10467](https://arxiv.org/pdf/1710.10467.pdf) | GE2E (encoder)| Generalized End-To-End Loss for Speaker Verification | This repo |
## F Q&A
#### 1.Where can I download the dataset?
| Dataset | Original Source | Alternative Sources |
| --- | ----------- | ---------------|
| aidatatang_200zh | [OpenSLR](http://www.openslr.org/62/) | [Google Drive](https://drive.google.com/file/d/110A11KZoVe7vy6kXlLb6zVPLb_J91I_t/view?usp=sharing) |
| magicdata | [OpenSLR](http://www.openslr.org/68/) | [Google Drive (Dev set)](https://drive.google.com/file/d/1g5bWRUSNH68ycC6eNvtwh07nX3QhOOlo/view?usp=sharing) |
| aishell3 | [OpenSLR](https://www.openslr.org/93/) | [Google Drive](https://drive.google.com/file/d/1shYp_o4Z0X0cZSKQDtFirct2luFUwKzZ/view?usp=sharing) |
| data_aishell | [OpenSLR](https://www.openslr.org/33/) | |
> After unzip aidatatang_200zh, you need to unzip all the files under `aidatatang_200zh\corpus\train`
#### 2.What is`<datasets_root>`?
If the dataset path is `D:\data\aidatatang_200zh`,then `<datasets_root>` is`D:\data`
#### 3.Not enough VRAM
Train the synthesizer锛歛djust the batch_size in `synthesizer/hparams.py`
```
//Before
tts_schedule = [(2, 1e-3, 20_000, 12), # Progressive training schedule
(2, 5e-4, 40_000, 12), # (r, lr, step, batch_size)
(2, 2e-4, 80_000, 12), #
(2, 1e-4, 160_000, 12), # r = reduction factor (# of mel frames
(2, 3e-5, 320_000, 12), # synthesized for each decoder iteration)
(2, 1e-5, 640_000, 12)], # lr = learning rate
//After
tts_schedule = [(2, 1e-3, 20_000, 8), # Progressive training schedule
(2, 5e-4, 40_000, 8), # (r, lr, step, batch_size)
(2, 2e-4, 80_000, 8), #
(2, 1e-4, 160_000, 8), # r = reduction factor (# of mel frames
没有合适的资源?快使用搜索试试~ 我知道了~
声音克隆代码,让你在5秒钟内克隆声音
共156个文件
py:77个
pyc:42个
js:8个
需积分: 0 22 下载量 88 浏览量
2023-08-21
20:37:41
上传
评论 1
收藏 439.91MB RAR 举报
温馨提示
这是一个声音克隆代码,它可以在短短的5秒钟内帮助您实现声音的克隆。无论是语音、音乐还是环境音效,只需将源声音输入代码并设置相关参数,即可快速生成克隆音频。这个代码采用高级算法和深度学习技术,能够准确捕捉原声音的特征和音质,并以极快的速度生成克隆音频。克隆声音的质量和真实度都非常高,几乎无法区分与原声音的差异。无论是用于音乐制作、电影后期处理还是语音合成等领域,这个声音克隆代码都将是一个强大的工具。它不仅能提高工作效率,还能为用户带来更多创造性的可能性。
资源推荐
资源详情
资源评论
收起资源包目录
声音克隆代码,让你在5秒钟内克隆声音 (156个子文件)
DOCKERFILE 162B
.gitattributes 26B
.gitignore 289B
.gitignore 184B
index.html 17KB
MockingBird-main.iml 707B
mp3-engine.js 472KB
eruda.min.js 398KB
jquery.js 282KB
recorder-core.js 32KB
mp3.js 12KB
frequency.histogram.view.js 12KB
lib.fft.js 3KB
wav.js 2KB
launch.json 1KB
config_16k_.json 631B
settings.json 45B
README-CN.md 12KB
README.md 10KB
CODE_OF_CONDUCT.md 5KB
mockingbird.png 89KB
bird-sm.png 40KB
bird.png 39KB
mb.png 6KB
rty4_56k.pt 374.85MB
pretrained.pt 51.35MB
g_hifigan.pt 49.58MB
pretrained.pt 16.3MB
ui.py 26KB
tacotron.py 22KB
fatchord_version.py 15KB
__init__.py 15KB
train.py 13KB
train.py 12KB
demo_cli.py 11KB
models.py 10KB
logmmse.py 9KB
preprocess.py 8KB
inference.py 8KB
audio.py 8KB
inference.py 8KB
deepmind_version.py 7KB
meldataset.py 7KB
visualizations.py 7KB
model.py 6KB
preprocess.py 5KB
global_style_token.py 5KB
hparams.py 5KB
train.py 5KB
__init__.py 5KB
distribution.py 5KB
audio.py 5KB
train.py 5KB
synthesize.py 4KB
preprocess_speaker.py 4KB
pre.py 4KB
vocoder_train.py 3KB
plot.py 3KB
synthesizer_dataset.py 3KB
display.py 3KB
vocoder_dataset.py 3KB
synthesizer_preprocess_audio.py 3KB
encoder_preprocess.py 3KB
vocoder_preprocess.py 3KB
cleaners.py 2KB
encoder_train.py 2KB
audio.py 2KB
text.py 2KB
speaker_verification_dataset.py 2KB
numbers.py 2KB
demo_toolbox.py 2KB
_cmudict.py 2KB
synthesizer_train.py 2KB
speaker.py 2KB
hparams.py 2KB
inference.py 2KB
inference.py 2KB
profiler.py 1KB
audio.py 1KB
random_cycler.py 1KB
utils.py 1KB
synthesizer_preprocess_embeds.py 1KB
argutils.py 1KB
__init__.py 1KB
config.py 1KB
gen_wavernn.py 1001B
params_data.py 901B
utterance.py 855B
synthesizer.py 805B
symbols.py 779B
modelutils.py 767B
speaker_batch.py 616B
preprocess_transcript.py 596B
env.py 394B
__init__.py 358B
web.py 295B
gst_hyperparameters.py 261B
default.py 248B
utterance.py 222B
params_model.py 196B
共 156 条
- 1
- 2
资源评论
LewGarben
- 粉丝: 1w+
- 资源: 2
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- 基于QT实现的简单的停车场管理系统详细文档+全部资料+高分项目.zip
- 基于QT实现的银行管理系统详细文档+全部资料+高分项目.zip
- 基于QT实现的一个简单的个人网盘系统,分为好友操作和文件操作两部分。详细文档+全部资料+高分项目.zip
- 基于Qt实现的组态软件运行时系统原型详细文档+全部资料+高分项目.zip
- 基于Qt与MySQL的管理系统详细文档+全部资料+高分项目.zip
- 基于QT与C++的地铁自动售票系统详细文档+全部资料+高分项目.zip
- 基于Qt与C++开发的车载音乐播放系统详细文档+全部资料+高分项目.zip
- thinkphp6内核学生成绩管理系统源码 内附安装说明 站长亲测
- 基于粒子群的PMU优化配置 软件:MATLAB 介绍:电力系统PMU优化配置,为了使电力系统达到完全可观,以PMU配置数量最少为目标函数,运用粒子群算法进行优化处理,在IEEE30 39 57 118
- record_20241224_09_16_49.mp3
- Python实例-Python分块拆分txt文件中的数据
- Python实例-Python汇总各单位Excel档领料记录并加总每日领用次数
- Python实例-Python制作图形用户界面(GUI)让操作可视化
- mmexport1729869897900.jpg
- IMG_20241222_075106.jpg
- ThinkPHP5 MVC框架图书管理系统源码
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功