Abstract:Facial expression recognition (FER) is still one challenging research due to the small inter-class discrepancy in the facial expression data. In view of the significance of facial crucial regions for FER, many existing researches utilize the prior information from some annotated crucial points to improve the performance of FER. However, it is complicated and time-consuming to manually annotate facial crucial points, especially for vast wild expression images. Based on this, a local non-local joint network is proposed to adaptively light up the facial crucial regions in feature learning of FER in this paper. In the proposed method, two parts are constructed based on facial local and non-local information respectively, where an ensemble of multiple local networks are proposed to extract local features corresponding to multiple facial local regions and a non-local attention network is addressed to explore the significance of each local region. Especially, the attention weights obtained by the non-local network is fed into the local part to achieve the interactive feedback between the facial global and local information. Interestingly, the non-local weights corresponding to local regions are gradually updated and higher weights are given to more crucial regions. Moreover, U-Net is employed to extract the integrated features of deep semantic information and low hierarchical detail information of expression images. Finally, experimental results illustrate that the proposed method achieves more competitive performance compared with several state-of-the art methods on five benchmark datasets. Noticeably, the analyses of the non-local weights corresponding to local regions demonstrate that the proposed method can automatically enhance some crucial regions in the process of feature learning without any facial landmark information.

Faceknow: Facial Expression Recognition by a Global-Local Network with a Sub-Images-Related Contextual Attention Mechanism

A facial expression recognition network based on attention double branch enhanced fusion

Robust facial expression recognition with global-local joint representation learning

Emotion Recognition via Environmental Context and Human Body

Attention mechanism-based CNN for facial expression recognition

Adaptively Enhancing Facial Expression Crucial Regions via Local Non-Local Joint Network

Two-pathway attention network for real-time facial expression recognition

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Diarrhoea and acute respiratory infections among under-five children in slums: Evidence from India

Facial Expression Recognition from a Single Face Image Based on Deep Learning and Broad Learning

A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition

Automatic Facial Expression Recognition Based on a Deep Convolutional-Neural-network Structure

Facial Expression Recognition System Based On Deep Residual Fusion Neural Network

Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

The Facial Expression Recognition Method Based on Image Fusion and CNN

Facial Expression Recognition with an Attention Network Using a Single Depth Image

Facial Expression Recognition Based on Local–global Information Reasoning and Spatial Distribution of Landmark Features

Research on facial expression recognition based on Multimodal data fusion and neural network

Hybrid Attention-Aware Learning Network for Facial Expression Recognition in the Wild