Abstract:This paper proposes a regional adaptive correlation network (RACN) to explore more effective description of structural information of faces and enrich the expression feature representation. The network consists of two branches. The proposed second‐order regional correlation network (SRCN) is composed of autocorrelation matrix calculation modules and network layers, and the obtained regional correlation features are united with the global features extracted from the parallel branch; finally, expression classification is performed after assigning weights to the features of both branches through a channel attention mechanism. To address the problem that the features extracted by CNN‐based facial expression recognition (FER) do not consider structural information, a region adaptive correlation deep network (RACN) is proposed. The network consists of two branches. In one branch, the features obtained by applying CNN to facial sub‐blocks are used as the input of the proposed second‐order region correlation network (SRCN), which obtains structural features by adaptively learning the correlation of facial regions. Furthermore, they are fused with the parallel branch‐extracted global features to obtain a comprehensive high‐semantic feature representation. Finally, weights are assigned to the two features through the channel attention mechanism for more accurate expression classification. Experimental results show that our method can effectively extract expression features in an end‐to‐end manner, improve the accuracy of FER, and achieve competitive performance without relying on any a priori knowledge. And the region‐adaptive correlation feature extraction branch RACN can be applied to other deep learning networks to extract discriminative structural‐adaptive features. To the best of our knowledge, our work is the first to enrich the feature representation for end‐to‐end static FER by adaptively obtaining more discriminative regional adaptive correlation feature vectors via the autocorrelation matrix combined with CNN compared to the existing literature.

Facial expression recognition algorithm based on feature fusion adaptive weighted HLAC

Robust Face Recognition by Fusion Local Singular Value Feature and Deformable Model

A new facial feature based on the fusion of texture and shape characteristics

An Adaptive Fuzzy Fusion Framework for Face Recognition under Illumination Variation Based on Local Multiple Patterns.

Weight-Adapted Convolution Neural Network for Facial Expression Recognition

A Fusion Framework For Face Recognition Under Varying Illumination Based On Multi-Scale Analysis

Weighted-fusion feature of MB-LBPUH and HOG for facial expression recognition

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

A hierarchical algorithm with multi-feature fusion for facial expression recognition

Information Fusion in Attention Networks Using Adaptive and Multi-level Factorized Bilinear Pooling for Audio-visual Emotion Recognition

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP

A facial expression recognition network based on attention double branch enhanced fusion

An Enhanced LBP Feature Based on Facial Expression Recognition

Research on facial expression recognition based on Multimodal data fusion and neural network

The Facial Expression Recognition Method Based on Image Fusion and CNN

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

Facial expression recognition through multi-level features extraction and fusion

Facial expression recognition based on regional adaptive correlation

Local and Global Feature Attention Fusion Network for Face Recognition

Feature Level Analysis for 3D Facial Expression Recognition.

Adaptive key-frame selection-based facial expression recognition via multi-cue dynamic features hybrid fusion