SimSwap++: Towards Faster and High-Quality Identity Swapping
Xuanhong Chen,Bingbing Ni,Yutian Liu,Naiyuan Liu,Zhilin Zeng,Hang Wang
DOI: https://doi.org/10.1109/tpami.2023.3307156
IF: 23.6
2023-01-01
IEEE Transactions on Pattern Analysis and Machine Intelligence
Abstract:Face identity editing (FIE) shows great value in AI content creation. Low-resolution FIE approaches have achieved tremendous progress, but high-quality FIE struggles. Two major challenges hinder higher-resolution and higher-performance development of FIE: lack of high-resolution dataset and unacceptable complexity forbidding for mobile platforms. To address both issues, we establish a novel large-scale, high-quality dataset tailored for FIE. Based on our SimSwap [1], we propose an upgraded version named SimSwap++ with significantly boosted model efficiency. SimSwap++ features two major innovations for high-performance model compression. Firstly, a novel computational primitive named Conditional Dynamic Convolution (CD-Conv) is proposed to address the inefficiency of conditional schemes (e.g., AdaIN) in tiny models. CD-Conv achieves anisotropic processing and injection with significantly lower complexity compared to standard conditional operators, e.g., modulated convolution. Secondly, a Morphable Knowledge Distillation (MKD) is presented to further trim the overall model. Unlike conventional homogeneous teacher-student structures, MKD is designed to be heterogeneous and mutually compensable, endowing the student with the multi-path morphable property; thus, our student maximally inherits the teacher' knowledge after distillation while further reducing its complexity through structure re-parameterization. Extensive experiments demonstrate that our SimSwap++ achieves state-of-the-art performance ( 97.55% ID accuracy on FaceForensics++) with extremely low complexity (2.5 GFLOPs).
computer science, artificial intelligence,engineering, electrical & electronic