Facial Attention based Convolutional Neural Network for 2D+3D Facial Expression Recognition

Yang Jiao,Yi Niu,Yuting Zhang,Fu Li,Chunbo Zou,Guangming Shi

DOI: https://doi.org/10.1109/VCIP47243.2019.8965843

2019-01-01

Abstract:Discriminative facial parts are essential for facial expression recognition (FER) tasks because of small inter-class differences and large intra-class variations in expression images. Existing methods localize discriminative regions with the aid of extra facial landmarks, such as action units (AU). However, it consumes a lot of manpower in manually labeling. To address this problem, in this paper, we propose an advanced facial attention based convolutional neural network (FA-CNN) for 2D+3D FER. The main contribution of FA-CNN is the facial attention mechanism, which enables the network to localize the discriminative regions automatically from multi-modality expression images without dense landmark annotations. Experimental results conducted on BU-3DFE demonstrate that FA-CNN achieves state-of-the-art performance comparing with the existing 2D+3D FER techniques, and the discriminative facial parts estimated by the facial attention mechanism are highly interpretable and consistent with human perception.