深度学习工具链:facechain

<p align="center"> <br> <img src="https://modelscope.oss-cn-beijing.aliyuncs.com/modelscope.gif" width="400"/> <br> <h1>FaceChain</h1> <p> # Introduction 如果您熟悉中文，可以阅读[中文版本的README](./README_ZH.md)。 FaceChain is a deep-learning toolchain for generating your Digital-Twin. With a minimum of 1 portrait-photo, you can create a Digital-Twin of your own and start generating personal portraits in different settings (multiple styles now supported!). You may train your Digital-Twin model and generate photos via FaceChain's Python scripts, or via the familiar Gradio interface. FaceChain is powered by [ModelScope](https://github.com/modelscope/modelscope). <p align="center"> ModelScope Studio <a href="https://modelscope.cn/studios/CVstudio/cv_human_portrait/summary">🤖<a></a>&nbsp ｜ HuggingFace Space <a href="https://huggingface.co/spaces/modelscope/FaceChain">🤗</a>&nbsp </p> <br>  # News - High performance inpainting for single & double person, Simplify User Interface. (September 09th, 2023 UTC) - More Technology Details can be seen in [Paper](https://arxiv.org/abs/2308.14256). (August 30th, 2023 UTC) - Add validate & ensemble for Lora training, and InpaintTab(hide in gradio for now). (August 28th, 2023 UTC) - Add pose control module. (August 27th, 2023 UTC) - Add robust face lora training module, enhance the performance of one pic training & style-lora blending. (August 27th, 2023 UTC) - HuggingFace Space is available now! You can experience FaceChain directly with <a href="https://huggingface.co/spaces/modelscope/FaceChain">🤗</a> (August 25th, 2023 UTC) - Add awesome prompts! Refer to: [awesome-prompts-facechain](resources/awesome-prompts-facechain.txt) (August 18th, 2023 UTC) - Support a series of new style models in a plug-and-play fashion. (August 16th, 2023 UTC) - Support customizable prompts. (August 16th, 2023 UTC) - Colab notebook is available now! You can experience FaceChain directly with [](https://colab.research.google.com/github/modelscope/facechain/blob/main/facechain_demo.ipynb). (August 15th, 2023 UTC) # To-Do List - Support more style models (such as those on Civitai). --on-going, hot - Support more beauty-retouch effects - Support latest foundation models such as SDXL - Support high resolution - Support group photo scenario, e.g, multi-person - Provide more funny apps # Citation Please cite FaceChain in your publications if it helps your research ``` @article{liu2023facechain, title={FaceChain: A Playground for Identity-Preserving Portrait Generation}, author={Liu, Yang and Yu, Cheng and Shang, Lei and Wu, Ziheng and Wang, Xingjun and Zhao, Yuze and Zhu, Lin and Cheng, Chen and Chen, Weitao and Xu, Chao and Xie, Haoyu and Yao, Yuan and Zhou, Wenmeng and Chen Yingda and Xie, Xuansong and Sun, Baigui}, journal={arXiv preprint arXiv:2308.14256}, year={2023} } ``` # Installation ## Compatibility Verification We have verified e2e execution on the following environment: - python: py3.8, py3.10 - pytorch: torch2.0.0, torch2.0.1 - tensorflow: 2.8.0, tensorflow-cpu - CUDA: 11.7 - CUDNN: 8+ - OS: Ubuntu 20.04, CentOS 7.9 - GPU: Nvidia-A10 24G ## Resource Requirement - GPU: About 19G - Disk: About 50GB ## Installation Guide The following installation methods are supported: ### 1. ModelScope notebook【recommended】 The ModelScope Notebook offers a free-tier that allows ModelScope user to run the FaceChain application with minimum setup, refer to [ModelScope Notebook](https://modelscope.cn/my/mynotebook/preset) ```shell # Step1: 我的notebook -> PAI-DSW -> GPU环境 # Step2: Entry the Notebook cell，clone FaceChain from github: !GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/modelscope/facechain.git --depth 1 # Step3: Change the working directory to facechain: import os os.chdir('/mnt/workspace/facechain') # You may change to your own path print(os.getcwd()) !pip3 install gradio !pip3 install controlnet_aux==0.0.6 !pip3 install python-slugify !python3 app.py # Step4: click "public URL" or "local URL", upload your images to # train your own model and then generate your digital twin. ``` Alternatively, you may also purchase a [PAI-DSW](https://www.aliyun.com/activity/bigdata/pai/dsw) instance (using A10 resource), with the option of ModelScope image to run FaceChain following similar steps. ### 2. Docker If you are familiar with using docker, we recommend to use this way: ```shell # Step1: Prepare the environment with GPU on local or cloud, we recommend to use Alibaba Cloud ECS, refer to: https://www.aliyun.com/product/ecs # Step2: Download the docker image (for installing docker engine, refer to https://docs.docker.com/engine/install/） docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.0 # Step3: run the docker container docker run -it --name facechain -p 7860:7860 --gpus all registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.7.1-py38-torch2.0.1-tf1.15.5-1.8.0 /bin/bash (Note: you may need to install the nvidia-container-runtime, refer to https://github.com/NVIDIA/nvidia-container-runtime) # Step4: Install the gradio in the docker container: pip3 install gradio pip3 install controlnet_aux==0.0.6 pip3 install python-slugify # Step5 clone facechain from github GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/modelscope/facechain.git --depth 1 cd facechain python3 app.py # Note: FaceChain currently assume single-GPU, if your environment has multiple GPU, please use the following instead: # CUDA_VISIBLE_DEVICES=0 python3 app.py # Step6 Run the app server: click "public URL" --> in the form of: https://xxx.gradio.live ``` ### 3. Conda Virtual Environment Use the conda virtual environment, and refer to [Anaconda](https://docs.anaconda.com/anaconda/install/) to manage your dependencies. After installation, execute the following commands: (Note: mmcv has strict environment requirements and might not be compatible in some cases. It's recommended to use Docker.) ```shell conda create -n facechain python=3.8 # Verified environments: 3.8 and 3.10 conda activate facechain GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/modelscope/facechain.git --depth 1 cd facechain pip3 install -r requirements.txt pip3 install -U openmim mim install mmcv-full==1.7.0 # Navigate to the facechain directory and run: python3 app.py # Note: FaceChain currently assume single-GPU, if your environment has multiple GPU, please use the following instead: # CUDA_VISIBLE_DEVICES=0 python3 app.py # Finally, click on the URL generated in the log to access the web page. ``` **Note**: After the app service is successfully launched, go to the URL in the log, enter the "Image Customization" tab, click "Select Image to Upload", and choose at least one image with a face. Then, click "Start Training" to begin model training. After the training is completed, there will be corresponding displays in the log. Afterwards, switch to the "Image Experience" tab and click "Start Inference" to generate your own digital image. *Note* For windows user, you should pay attention to following steps: ```shell 1. reinstall package pytorch and numpy compatible with tensorflow 2. install mmcv-full by pip: pip3 install mmcv-full ``` **If you want to use the `Audio Driven Talking Head` tab, please refer to the installation guide in [installation_for_talkinghead](doc/installation_for_talkinghead.md).** ### 4. Colab notebook | Colab | Info | --- | --- | [](https://colab.research.google.com/github/modelscope/facechain/blob/main/facechain_demo.ipynb) | FaceChain Installation on Colab # Script Execution FaceChain supports direct training and inf