Abstract:Recent studies have shown that deep learning has presented great potential in facial expression recognition (FER) tasks and attracted more and more researchers’ attention. Many existing methods have achieved good results on facial expression images in the laboratory environment. However, there are still great challenges for FER in the wild environment where facial expression images are more complex and diverse than those in the laboratory. In this paper, we propose a new method for FER from the perspective of multi-level features extraction and fusion. Different from the existing feature extraction network where only a single convolution kernel scale is present, we propose a feature extraction module with different convolutional kernel scales, which extracts multi-level features as the output of the whole feature extraction network. Further, we do not directly use these multi-level features but propose a feature fusion module with global and local attention to adaptively fuse these different level features in pairs with a top-down way and construct a new facial expression feature. To relieve the overfitting effect caused by data imbalance, we employ label smoothing and L2 regularization strategies to further guide our model forward in a better direction. Through extensive experiments, we demonstrate our method achieves accuracies of 88.08% on RAFDB, 88.11% on FERPlus and 59.38% on AffectNet, respectively, which are very competitive performances. Moreover, our multi-level feature fusion approach enables traditional convolutional backbone networks to improve performance by 0.64–1.13% on FER tasks.

Expression Recognition Based on Fusion Features Extraction and LLE Method

Robust Face Recognition by Fusion Local Singular Value Feature and Deformable Model

An Adaptive Fuzzy Fusion Framework for Face Recognition under Illumination Variation Based on Local Multiple Patterns.

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Combination of Local Multiple Patterns and Exponential Discriminant Analysis for facial recognition

Facial expression recognition through multi-level features extraction and fusion

Diarrhoea and acute respiratory infections among under-five children in slums: Evidence from India

A facial expression recognition network based on attention double branch enhanced fusion

The Facial Expression Recognition Method Based on Image Fusion and CNN

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

A Navel Facial Expression Recognition Method Based On Extreme Learning Machine

An Enhanced LBP Feature Based on Facial Expression Recognition

Enhanced Dual-Level Representations for Facial Expression Recognition

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Facial expression recognition method based on deep convolutional neural network combined with improved LBP features

Facial Expression Recognition Based on Local–global Information Reasoning and Spatial Distribution of Landmark Features

Facial Expression Recognition System Based on Multiple Feature Integration

Weighted-fusion feature of MB-LBPUH and HOG for facial expression recognition

Feature Level Analysis for 3D Facial Expression Recognition.

Fusion of Locally Linear Embedding and Principal Component Analysis for Face Recognition (FLLEPCA)

A Robust Facial Expression Recognition Algorithm Based on Multi-Rate Feature Fusion Scheme