An Efficient Attribute-Preserving Framework for Face Swapping
Tianyi Wang,Zian Li,Ruixia Liu,Yinglong Wang,Liqiang Nie
DOI: https://doi.org/10.1109/tmm.2024.3354573
IF: 7.3
2024-01-01
IEEE Transactions on Multimedia
Abstract:By leveraging deep neural networks, recent face swapping techniques have performed admirably in generating faces that maintain consistent identities. Nevertheless, while these methods accurately transfer source identities, they often struggle to preserve important attributes (such as head poses, expressions, and gaze directions) in the target faces. As a consequence, the current research in this domain has not resulted in satisfactory performance. In this article, we propose an efficient attribute-preserving framework, called AP-Swap, for short, for face swapping. Our approach incorporates two innovative modules designed specifically to preserve critical facial attributes. First, we propose a global residual attribute-preserving encoder (GRAPE), which adaptively extracts globally complete attribute features from target faces. Second, in addition to the regular network streams for the source and target facial images, we introduce a network stream that takes into account the facial landmarks of the target faces. This additional stream enables our landmark-guided feature entanglement module (LFEM), which efficiently preserves fine-grained facial attributes by conducting a landmark-based attribute-preserving (LBAP) operation. Through extensive quantitative and qualitative experiments, we demonstrate the superiority of AP-Swap over other state-of-the-art methods in terms of facial attribute preservation and model efficiency, along with satisfactory identity consistency performance.
computer science, information systems,telecommunications, software engineering