Abstract:Recent studies have shown that deep learning has presented great potential in facial expression recognition (FER) tasks and attracted more and more researchers’ attention. Many existing methods have achieved good results on facial expression images in the laboratory environment. However, there are still great challenges for FER in the wild environment where facial expression images are more complex and diverse than those in the laboratory. In this paper, we propose a new method for FER from the perspective of multi-level features extraction and fusion. Different from the existing feature extraction network where only a single convolution kernel scale is present, we propose a feature extraction module with different convolutional kernel scales, which extracts multi-level features as the output of the whole feature extraction network. Further, we do not directly use these multi-level features but propose a feature fusion module with global and local attention to adaptively fuse these different level features in pairs with a top-down way and construct a new facial expression feature. To relieve the overfitting effect caused by data imbalance, we employ label smoothing and L2 regularization strategies to further guide our model forward in a better direction. Through extensive experiments, we demonstrate our method achieves accuracies of 88.08% on RAFDB, 88.11% on FERPlus and 59.38% on AffectNet, respectively, which are very competitive performances. Moreover, our multi-level feature fusion approach enables traditional convolutional backbone networks to improve performance by 0.64–1.13% on FER tasks.

3D Facial Expression Recognition Using Deep Feature Fusion CNN

Multimodal 2D+3D Facial Expression Recognition with Deep Fusion Convolutional Neural Network

Accurate Facial Parts Localization and Deep Learning for 3D Facial Expression Recognition

Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Network.

Leveraging 3D Blendshape for Facial Expression Recognition Using CNN

The Facial Expression Recognition Method Based on Image Fusion and CNN

Landmarks-assisted Collaborative Deep Framework for Automatic 4D Facial Expression Recognition.

Deep Representation of Facial Geometric and Photometric Attributes for Automatic 3D Facial Expression Recognition

Towards Reading Beyond Faces for Sparsity-aware 3D/4D Affect Recognition

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

Automatic facial expression recognition on a single 3D face by exploring shape deformation.

Discriminative Attention-based Convolutional Neural Network for 3D Facial Expression Recognition.

Facial Expression Recognition Based On Global And Local Feature Fusion With Cnns

3D Facial Expression Recognition Algorithm Using Local Threshold Binary Pattern and Histogram of Oriented Gradient

Facial Expression Recognition in Video Using 3D-CNN Deep Features Discrimination

Facial Attention based Convolutional Neural Network for 2D+3D Facial Expression Recognition

Fully Automatic 3D Facial Expression Recognition Using Local Depth Features

Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks

Facial Expression Recognition Using Hierarchical Features with Deep Comprehensive Multipatches Aggregation Convolutional Neural Networks

Facial expression recognition through multi-level features extraction and fusion

Deep Spatial-Temporal Feature Fusion for Facial Expression Recognition in Static Images