Abstract:This paper proposes a regional adaptive correlation network (RACN) to explore more effective description of structural information of faces and enrich the expression feature representation. The network consists of two branches. The proposed second‐order regional correlation network (SRCN) is composed of autocorrelation matrix calculation modules and network layers, and the obtained regional correlation features are united with the global features extracted from the parallel branch; finally, expression classification is performed after assigning weights to the features of both branches through a channel attention mechanism. To address the problem that the features extracted by CNN‐based facial expression recognition (FER) do not consider structural information, a region adaptive correlation deep network (RACN) is proposed. The network consists of two branches. In one branch, the features obtained by applying CNN to facial sub‐blocks are used as the input of the proposed second‐order region correlation network (SRCN), which obtains structural features by adaptively learning the correlation of facial regions. Furthermore, they are fused with the parallel branch‐extracted global features to obtain a comprehensive high‐semantic feature representation. Finally, weights are assigned to the two features through the channel attention mechanism for more accurate expression classification. Experimental results show that our method can effectively extract expression features in an end‐to‐end manner, improve the accuracy of FER, and achieve competitive performance without relying on any a priori knowledge. And the region‐adaptive correlation feature extraction branch RACN can be applied to other deep learning networks to extract discriminative structural‐adaptive features. To the best of our knowledge, our work is the first to enrich the feature representation for end‐to‐end static FER by adaptively obtaining more discriminative regional adaptive correlation feature vectors via the autocorrelation matrix combined with CNN compared to the existing literature.

Relation and context augmentation network for facial expression recognition

Relation-aware Network for Facial Expression Recognition

Multi-relations Aware Network for in-the-wild Facial Expression Recognition

An Improved SimAM Based CNN for Facial Expression Recognition

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Relation-Aware Facial Expression Recognition

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Multi-level Spatial and Semantic Enhancement Network for Expression Recognition

Facial expression recognition through multi-level features extraction and fusion

Facial Expression Recognition Based on Convolutional Neural Network with Sparse Representation

Learning Associative Representation for Facial Expression Recognition.

Improving Ensemble Learning Performance With Complementary Neural Networks For Facial Expression Recognition

Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion

MCF-Net: Fusion Network of Facial and Scene Features for Expression Recognition in the Wild

Regional Self-Attention Convolutional Neural Network for Facial Expression Recognition

Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields

Facial expression recognition based on regional adaptive correlation

Facial Expression Recognition Model Based on CNN and Data Augmentation Method

Hybrid Attention-Aware Learning Network for Facial Expression Recognition in the Wild