FaceChain: A Playground for Human-centric Artificial Intelligence Generated Content
Yang Liu,Cheng Yu,Lei Shang,Yongyi He,Ziheng Wu,Xingjun Wang,Chao Xu,Haoyu Xie,Weida Wang,Yuze Zhao,Lin Zhu,Chen Cheng,Weitao Chen,Yuan Yao,Wenmeng Zhou,Jiaqi Xu,Qiang Wang,Yingda Chen,Xuansong Xie,Baigui Sun
DOI: https://doi.org/10.48550/arXiv.2308.14256
2023-12-14
Abstract:Recent advancement in personalized image generation have unveiled the intriguing capability of pre-trained text-to-image models on learning identity information from a collection of portrait images. However, existing solutions are vulnerable in producing truthful details, and usually suffer from several defects such as (i) The generated face exhibit its own unique characteristics, \ie facial shape and facial feature positioning may not resemble key characteristics of the input, and (ii) The synthesized face may contain warped, blurred or corrupted regions. In this paper, we present FaceChain, a personalized portrait generation framework that combines a series of customized image-generation model and a rich set of face-related perceptual understanding models (\eg, face detection, deep face embedding extraction, and facial attribute recognition), to tackle aforementioned challenges and to generate truthful personalized portraits, with only a handful of portrait images as input. Concretely, we inject several SOTA face models into the generation procedure, achieving a more efficient label-tagging, data-processing, and model post-processing compared to previous solutions, such as DreamBooth ~\cite{ruiz2023dreambooth} , InstantBooth ~\cite{shi2023instantbooth} , or other LoRA-only approaches ~\cite{hu2021lora} . Besides, based on FaceChain, we further develop several applications to build a broader playground for better showing its value, including virtual try on and 2D talking head. We hope it can grow to serve the burgeoning needs from the communities. Note that this is an ongoing work that will be consistently refined and improved upon. FaceChain is open-sourced under Apache-2.0 license at \url{<a class="link-external link-https" href="https://github.com/modelscope/facechain" rel="external noopener nofollow">this https URL</a>}.
Computer Vision and Pattern Recognition,Artificial Intelligence