FaceChain: A Playground for Identity-Preserving Portrait Generation
Yang Liu,Cheng Yu,Lei Shang,Ziheng Wu,Xingjun Wang,Yuze Zhao,Lin Zhu,Chen Cheng,Weitao Chen,Chao Xu,Haoyu Xie,Yuan Yao,Wenmeng Zhou,Yingda Chen,Xuansong Xie,Baigui Sun
DOI: https://doi.org/10.48550/arxiv.2308.14256
2023-01-01
Abstract:: Recent advancement in personalized image generation have unveiled the intriguing capability of pre-trained text-to-image models on learning identity information from a collection of portrait images. However, existing solutions can be vulnerable in producing truthful details, and usually suffer from several defects such as (i) The generated face exhibit its own unique characteristics, i.e . facial shape and facial feature positioning may not re-semble key characteristics of the input, and (ii) The synthesized face may contain warped, blurred or corrupted regions. In this paper, we present FaceChain, a personalized portrait generation framework that combines a series of customized image-generation model and a rich set of face-related perceptual understanding models ( e.g ., face detection, deep face embedding extraction, and facial attribute recognition), to tackle aforementioned challenges and to generate truthful personalized portraits, with only a handful of portrait images as input. Concretely, we inject several SOTA face models into the generation procedure, achieving a more efficient label-tagging, data-processing, and model post-processing compared to previous solutions, such as DreamBooth [9] , InstantBooth [11] , or other LoRA-only approaches [3] . Through the development of FaceChain, we have identified several potential directions to accelerate development of Face/Human-Centric AIGC research and application. We have designed FaceChain as a framework comprised of pluggable components that can be easily adjusted to accommodate different styles and personalized needs. We hope it can grow to serve the burgeoning needs from the communities. Note that this is an ongoing work that will be consistently refined and improved upon. FaceChain is open-sourced under Apache-2.0 license at https://github.com/modelscope/facechain .