Abstract:This paper proposes a regional adaptive correlation network (RACN) to explore more effective description of structural information of faces and enrich the expression feature representation. The network consists of two branches. The proposed second‐order regional correlation network (SRCN) is composed of autocorrelation matrix calculation modules and network layers, and the obtained regional correlation features are united with the global features extracted from the parallel branch; finally, expression classification is performed after assigning weights to the features of both branches through a channel attention mechanism. To address the problem that the features extracted by CNN‐based facial expression recognition (FER) do not consider structural information, a region adaptive correlation deep network (RACN) is proposed. The network consists of two branches. In one branch, the features obtained by applying CNN to facial sub‐blocks are used as the input of the proposed second‐order region correlation network (SRCN), which obtains structural features by adaptively learning the correlation of facial regions. Furthermore, they are fused with the parallel branch‐extracted global features to obtain a comprehensive high‐semantic feature representation. Finally, weights are assigned to the two features through the channel attention mechanism for more accurate expression classification. Experimental results show that our method can effectively extract expression features in an end‐to‐end manner, improve the accuracy of FER, and achieve competitive performance without relying on any a priori knowledge. And the region‐adaptive correlation feature extraction branch RACN can be applied to other deep learning networks to extract discriminative structural‐adaptive features. To the best of our knowledge, our work is the first to enrich the feature representation for end‐to‐end static FER by adaptively obtaining more discriminative regional adaptive correlation feature vectors via the autocorrelation matrix combined with CNN compared to the existing literature.

R-FENet: A Region-based Facial Expression Recognition Method Inspired by Semantic Information of Action Units

SDNET: Lightweight Facial Expression Recognition For Sample Disequilibrium.

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Region Attention Networks for Pose and Occlusion Robust Facial Expression Recognition

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Facial expression recognition based on regional adaptive correlation

Facial Expression Recognition Based on Zero-Addition Pretext Training and Feature Conjunction-Selection Network in Human–Robot Interaction

An Improved SimAM Based CNN for Facial Expression Recognition

Two-pathway attention network for real-time facial expression recognition

SAANet: Siamese Action-Units Attention Network for Improving Dynamic Facial Expression Recognition

Relation-aware Network for Facial Expression Recognition

Adaptively Enhancing Facial Expression Crucial Regions via Local Non-Local Joint Network

Expression Empowered ResiDen Network for Facial Action Unit Detection

NA-Resnet: neighbor block and optimized attention module for global-local feature extraction in facial expression recognition

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP

FLEPNet: Feature Level Ensemble Parallel Network for Facial Expression Recognition

A facial expression recognition network based on attention double branch enhanced fusion

A Region Group Adaptive Attention Model for Subtle Expression Recognition

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

Facial Expression Recognition Network with Slow Convolution and Zero-parameter Attention Mechanism

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry