Abstract:The emotion perception capability could endow machines with deeper feedback evaluation of human–machine interaction (HMI), which makes the experience of HMI more harmonious and natural. Currently, how to obtain the robust surface and semantic facial representation features has not been fully explored, moreover, the parameters and computation of conventional convolutional neural network models, which are adopted in facial expression recognition (FER) are too heavy, it is not conducive to the deployment of the HMI system. Therefore, this article proposes a representation reinforcement network (RRN) and transfer self-training (TST) based efficient FER method. Our designed RRN is mainly composed of two modules, i.e., surface representation reinforcement (SurRR) module and semantic representation reinforcement (SemaRR) module. SurRR module fuses the facial features in different dimensions and highlights key feature messaging nodes in feature maps. SemaRR module establishes the deep semantic representation of global facial regions on spatial and channel dimensions. Therefore, RRN has stronger feature extraction capability when the parameters and computational complexity are significantly reduced, and TST is proposed for contributing to the training effect of RRN at the condition of insufficient training data. We tested our method on the extended Cohn-Kanade dataset (CK+), Radboud faces database (RaFD), FERPLUS, and the real-world affective faces database (RAF-DB) datasets, and the recognition accuracy are 100%, 98.62%, 89.64%, and 88.72%, respectively. Furthermore, the preliminary application experiment, which demonstrates the feasibility of our method in practice during HMI.

Facial Emotion Recognition Based on Sobel-Resnet

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Facial expression recognition using adapted residual based deep neural network

Facial Sentiment Classification Based on Resnet-18 Model

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

Facial Emotion Recognition Based on Auxiliary Classifiers and SENet Module

Facial Expression Recognition Based On Global And Local Feature Fusion With Cnns

Face Expression Recognition Based on Dynamic Sparse Attention and Residual Network

Facial Expression Recognition Using Deep Neural Network

Facial Expression Recognition Based on Auxiliary Models

Expression Recognition Based on Attention-Split Convolutional Residual Network

Facial Expression Recognition with Squeeze-and-Excitation Network

EMONET: A Cross Database Progressive Deep Network for Facial Expression Recognition

Facial Expression Recognition Based on Sparse Autoencoder and Shallow Convolutional Neural Network.

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

Multi-level Spatial and Semantic Enhancement Network for Expression Recognition

Dnn-Cbam: an Enhanced Dnn Model for Facial Emotion Recognition

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Facial Expression Recognition Model Based on CNN and Data Augmentation Method

Uncertain and Biased Facial Expression Recognition Based on Depthwise Separable Convolutional Neural Network with Embedded Attention Mechanism

Scale Processing Residual Network Embedded with Attention Mechanism for Facial Expression Recognition