Abstract:Facial expression recognition (FER) is a significant task for the machines to understand the emotional changes in human beings. However, accurate hand-crafted features that are highly related to changes in expression are difficult to extract because of the influences of individual difference and variations in emotional intensity. Therefore, features that can accurately describe the changes in facial expressions are urgently required. Method: A weighted mixture deep neural network (WMDNN) is proposed to automatically extract the features that are effective for FER tasks. Several pre-processing approaches, such as face detection, rotation rectification, and data augmentation, are implemented to restrict the regions for FER. Two channels of facial images, including facial grayscale images and their corresponding local binary pattern (LBP) facial images, are processed by WMDNN. Expression-related features of facial grayscale images are extracted by fine-tuning a partial VGG16 network, the parameters of which are initialized using VGG16 model trained on ImageNet database. Features of LBP facial images are extracted by a shallow convolutional neural network (CNN) built based on DeepID. The outputs of both channels are fused in a weighted manner. The result of final recognition is calculated using softmax classification. Results: Experimental results indicate that the proposed algorithm can recognize six basic facial expressions (happiness, sadness, anger, disgust, fear, and surprise) with high accuracy. The average recognition accuracies for benchmarking data sets “CK+,”“JAFFE,”and “Oulu-CASIA”are 0.970, 0.922, and 0.923, respectively. Conclusions: The proposed FER method outperforms the state-of-the-art FER methods based on the hand-crafted features or deep networks using one channel. Compared with the deep networks that use multiple channels, our proposed network can achieve comparable performance with easier procedures. Fine-tuning is effective to FER tasks with a well pre-trained model if sufficient samples cannot be collected.

ERR-Net: Facial Expression Removal and Recognition Network with Residual Image

Cgan Based Facial Expression Recognition for Human-Robot Interaction

SDNET: Lightweight Facial Expression Recognition For Sample Disequilibrium.

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

A Novel Attention Residual Network Expression Recognition Method

Facial Expression Recognition System Based On Deep Residual Fusion Neural Network

Learning Expression Features via Deep Residual Attention Networks for Facial Expression Recognition From Video Sequences

FaceNet2ExpNet: Regularizing a Deep Face Recognition Net for Expression Recognition

Research on Expression Recognition Algorithm Based on Attention Mechanism

Two-pathway attention network for real-time facial expression recognition

A facial expression recognition network based on attention double branch enhanced fusion

Facial Expression Recognition Using Deep Neural Network

Attention mechanism-based CNN for facial expression recognition

A-MobileNet: An approach of facial expression recognition

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Facial Expression Recognition Via Coarse-Grained and Fine-Grained Feature Representation

Facial Expression Recognition Using Weighted Mixture Deep Neural Network Based on Double-Channel Facial Images

Uncertain and Biased Facial Expression Recognition Based on Depthwise Separable Convolutional Neural Network with Embedded Attention Mechanism

Enriched Long-Term Recurrent Convolutional Network for Facial Micro-Expression Recognition