CLN: Complementary Learning Network for 3D Face Reconstruction and Alignment

Kangbo Wu,Xitie Zhang,Xing Zheng,Suping Wu,Yongrong Cao,Zhiyuan Zhou,Kehua Ma

DOI: https://doi.org/10.1007/978-3-031-44210-0_13

2023-01-01

Abstract:In the complex and changeable unconstrained state, the 3D Morphable Model (3DMM) parameters regression methods lack the ability to express local details and they are difficult to enrich geometric details. In this paper, we propose a complementary learning network (CLN), which aims to improve the ability to extract global discriminative features and capture local details through complementary learning between global and local information. Specifically, we first elaborately design a cross-domain self-shuffling data augmentation method to simulate the case of the inconspicuous or obscured face information under unconstrained conditions, thus increasing the sensitivity of the network to local information. Different from other methods, our augmented data contains global data that can guide the network in reasoning about the inconspicuous areas of the face in special lighting and partially obscured areas of the face in large poses. To better achieve the complementary between global information and local details, we design a complementary learning network in which one stream extracts global discriminative features from the original input image. Meanwhile, another stream extracts local detailed features from the shuffled image. Then, we adopt the coordinate attention transformer module to enforce our network’s ability to capture the correlation between local information. Finally, we linearly fuse the global information that constrains the geometric shape of the face and the local information of the rich geometric details to improve the regression accuracy of the network. Extensive experiments on AFLW, AFLW2000-3D, and other datasets demonstrate that our method is superior to previous methods.