<div align="center">
<img src='https://user-images.githubusercontent.com/4397546/229094115-862c747e-7397-4b54-ba4a-bd368bfe2e0f.png' width='500px'/>
<!--<h2> ð SadTalkerï¼ <span style="font-size:12px">Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation </span> </h2> -->
<a href='https://arxiv.org/abs/2211.12194'><img src='https://img.shields.io/badge/ArXiv-PDF-red'></a> <a href='https://sadtalker.github.io'><img src='https://img.shields.io/badge/Project-Page-Green'></a> [](https://colab.research.google.com/github/Winfredy/SadTalker/blob/main/quick_demo.ipynb) [](https://huggingface.co/spaces/vinthony/SadTalker) [](https://colab.research.google.com/github/camenduru/stable-diffusion-webui-colab/blob/main/video/stable/stable_diffusion_1_5_video_webui_colab.ipynb) [](https://replicate.com/cjwbw/sadtalker)
<div>
<a target='_blank'>Wenxuan Zhang <sup>*,1,2</sup> </a> 
<a href='https://vinthony.github.io/' target='_blank'>Xiaodong Cun <sup>*,2</a> 
<a href='https://xuanwangvc.github.io/' target='_blank'>Xuan Wang <sup>3</sup></a> 
<a href='https://yzhang2016.github.io/' target='_blank'>Yong Zhang <sup>2</sup></a> 
<a href='https://xishen0220.github.io/' target='_blank'>Xi Shen <sup>2</sup></a>  </br>
<a href='https://yuguo-xjtu.github.io/' target='_blank'>Yu Guo<sup>1</sup> </a> 
<a href='https://scholar.google.com/citations?hl=zh-CN&user=4oXBp9UAAAAJ' target='_blank'>Ying Shan <sup>2</sup> </a> 
<a target='_blank'>Fei Wang <sup>1</sup> </a> 
</div>
<br>
<div>
<sup>1</sup> Xi'an Jiaotong University   <sup>2</sup> Tencent AI Lab   <sup>3</sup> Ant Group  
</div>
<br>
<i><strong><a href='https://arxiv.org/abs/2211.12194' target='_blank'>CVPR 2023</a></strong></i>
<br>
<br>
<b>TL;DR: single portrait image ðââï¸ + audio ð¤ = talking head video ð.</b>
<br>
</div>
## ð¥ Highlight
- ð¥ The extension of the [stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui) is online. Checkout more details [here](docs/webui_extension.md).
https://user-images.githubusercontent.com/4397546/231495639-5d4bb925-ea64-4a36-a519-6389917dac29.mp4
- ð¥ `full image mode` is online! checkout [here](https://github.com/Winfredy/SadTalker#full-bodyimage-generation) for more details.
| still+enhancer in v0.0.1 | still + enhancer in v0.0.2 | [input image @bagbag1815](https://twitter.com/bagbag1815/status/1642754319094108161) |
|:--------------------: |:--------------------: | :----: |
| <video src="https://user-images.githubusercontent.com/48216707/229484996-5d7be64f-2553-4c9e-a452-c5cf0b8ebafe.mp4" type="video/mp4"> </video> | <video src="https://user-images.githubusercontent.com/4397546/230717873-355b7bf3-d3de-49f9-a439-9220e623fce7.mp4" type="video/mp4"> </video> | <img src='./examples/source_image/full_body_2.png' width='380'>
- ð¥ Several new mode, eg, `still mode`, `reference mode`, `resize mode` are online for better and custom applications.
- ð¥ Happy to see more community demos at [bilibili](https://search.bilibili.com/all?keyword=sadtalker&from_source=webtop_search&spm_id_from=333.1007&search_source=3
), [Youtube](https://www.youtube.com/results?search_query=sadtalker&sp=CAM%253D) and [twitter #sadtalker](https://twitter.com/search?q=%23sadtalker&src=typed_query).
## ð Changelog (Previous changelog can be founded [here](docs/changlelog.md))
- __[2023.06.12]__: add more new features in WEBUI extension, see the discussion [here](https://github.com/OpenTalker/SadTalker/discussions/386).
- __[2023.06.05]__: release a new 512 beta face model. Fixed some bugs and improve the performance.
- __[2023.04.15]__: Adding automatic1111 colab by @camenduru, thanks for this awesome colab: [](https://colab.research.google.com/github/camenduru/stable-diffusion-webui-colab/blob/main/video/stable/stable_diffusion_1_5_video_webui_colab.ipynb).
- __[2023.04.12]__: adding a more detailed sd-webui installation document, fixed reinstallation problem.
- __[2023.04.12]__: Fixed the sd-webui safe issues becasue of the 3rd packages, optimize the output path in `sd-webui-extension`.
- __[2023.04.08]__: âï¸âï¸âï¸ In v0.0.2, we add a logo watermark to the generated video to prevent abusing since it is very realistic.
- __[2023.04.08]__: v0.0.2, full image animation, adding baidu driver for download checkpoints. Optimizing the logic about enhancer.
## ð§ TODO: See the Discussion https://github.com/OpenTalker/SadTalker/issues/280
## If you have any problem, please view our [FAQ](docs/FAQ.md) before opening an issue.
## âï¸ 1. Installation.
Tutorials from communities: [ä¸æwindowsæç¨](https://www.bilibili.com/video/BV1Dc411W7V6/) | [æ¥æ¬èªã³ã¼ã¹](https://br-d.fanbox.cc/posts/5685086?utm_campaign=manage_post_page&utm_medium=share&utm_source=twitter)
### Linux:
1. Installing [anaconda](https://www.anaconda.com/), python and git.
2. Creating the env and install the requirements.
```bash
git clone https://github.com/Winfredy/SadTalker.git
cd SadTalker
conda create -n sadtalker python=3.8
conda activate sadtalker
pip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu113
conda install ffmpeg
pip install -r requirements.txt
### tts is optional for gradio demo.
### pip install TTS
```
### Windows ([ä¸æwindowsæç¨](https://www.bilibili.com/video/BV1Dc411W7V6/)):
1. Install [Python 3.10.6](https://www.python.org/downloads/windows/), checking "Add Python to PATH".
2. Install [git](https://git-scm.com/download/win) manually (OR `scoop install git` via [scoop](https://scoop.sh/)).
3. Install `ffmpeg`, following [this instruction](https://www.wikihow.com/Install-FFmpeg-on-Windows) (OR using `scoop install ffmpeg` via [scoop](https://scoop.sh/)).
4. Download our SadTalker repository, for example by running `git clone https://github.com/Winfredy/SadTalker.git`.
5. Download the `checkpoint` and `gfpgan` [belowâ](https://github.com/Winfredy/SadTalker#-2-download-trained-models).
5. Run `start.bat` from Windows Explorer as normal, non-administrator, user, a gradio WebUI demo will be started.
### Macbook:
More tips about installnation on Macbook and the Docker file can be founded [here](docs/install.md)
## ð¥ 2. Download Trained Models.
You can run the following script to put all the models in the right place.
```bash
bash scripts/download_models.sh
```
Other alternatives:
> we also provide an offline patch (`gfpgan/`), thus, no model will be downloaded when generating.
**Google Driver**: download our pre-trained model from [ this link (main checkpoints)](https://drive.google.com/file/d/1gwWh45pF7aelNP_P78uDJL8Sycep-K7j/view?usp=sharing) and [ gfpgan (offline patch)](https://drive.google.com/file/d/19AIBsmfcHW6BRJmeqSFlG5fL445Xmsyi?usp=sharing)
**Github Release Page**: download all the files from the [lastest github release page](https://github.com/Winfredy/SadTalker/releases), and then, put it in ./checkpoints.
**ç¾åº¦äºç**: we provided the downloaded model in [checkpoints, æåç : sadt.](https://pan.baidu.com/s/1P4fRgk9gaSutZnn8YW034Q?pwd=sadt
SadTalker-main.zip
179 浏览量
2023-07-21
06:17:08
上传
评论 1
收藏 67.35MB ZIP 举报
SadTalker-main.zip （197个子文件） 
webui.bat 275B
using_ref_video.gif 7.74MB example_full_enhanced.gif 5.51MB free_view_result.gif 5.35MB resize_no.gif 2.04MB resize_good.gif 1.65MB example_crop.gif 1.48MB example_full.gif 1.39MB example_crop_still.gif 1.19MB example_full_crop.gif 817KB .gitignore 3KB quick_demo.ipynb 7KB full4.jpeg 26KB LICENSE 1KB BBRegressorParam_r.mat 22KB similarity_Lm3D_all.mat 994B README.md 14KB README.md 8KB speed_benchmark.md 6KB best_practice.md 5KB changlelog.md 2KB webui_extension.md 2KB FAQ.md 2KB install.md 2KB face3d.md 1KB install.md 1KB eval.md 655B modelzoo.md 0B
WDA_KatieHill_000.mp4 3.38MB WDA_AlexandriaOcasioCortez_000.mp4 2.15MB
art_4.png 3.46MB art_8.png 2.97MB art_17.png 2MB art_16.png 1.41MB art_3.png 1.29MB art_9.png 1.2MB art_5.png 1.17MB art_2.png 812KB art_0.png 733KB art_12.png 704KB art_20.png 694KB art_15.png 657KB art_14.png 635KB full3.png 617KB art_13.png 617KB art_10.png 556KB art_7.png 509KB art_1.png 478KB art_11.png 477KB art_19.png 462KB people_0.png 238KB full_body_2.png 134KB full_body_1.png 122KB art_18.png 115KB happy.png 108KB sad.png 108KB art_6.png 99KB sad1.png 44KB happy1.png 43KB sadtalker_logo.png 34KB networks.py 20KB util.py 20KB eval_ijbc.py 17KB verification.py 16KB base_model.py 13KB batchnorm.py 13KB my_awing_arch.py 12KB bfm.py 12KB animate.py 11KB generator.py 11KB facerecon_model.py 11KB visualizer.py 10KB onnx_helper.py 10KB onnx_ijbc.py 10KB partial_fc.py 9KB inference.py 7KB base_options.py 7KB iresnet.py 7KB launcher.py 7KB preprocess.py 7KB gradio_demo.py 7KB make_animation.py 7KB extension.py 7KB iresnet2060.py 7KB keypoint_detector.py 7KB util.py 6KB predict.py 6KB croper.py 6KB cvae.py 6KB hparams.py 6KB model2safetensor.py 6KB train.py 6KB template_model.py 6KB dense_motion.py 6KB app_sadtalker.py 6KB generate_facerender_batch.py 6KB extract_kp_videos_safe.py 6KB test_audio2coeff.py 5KB skin_mask.py 5KB utils_callbacks.py 5KB共 197 条
- 1
- 2
资源评论
