Abstract:The emotion perception capability could endow machines with deeper feedback evaluation of human–machine interaction (HMI), which makes the experience of HMI more harmonious and natural. Currently, how to obtain the robust surface and semantic facial representation features has not been fully explored, moreover, the parameters and computation of conventional convolutional neural network models, which are adopted in facial expression recognition (FER) are too heavy, it is not conducive to the deployment of the HMI system. Therefore, this article proposes a representation reinforcement network (RRN) and transfer self-training (TST) based efficient FER method. Our designed RRN is mainly composed of two modules, i.e., surface representation reinforcement (SurRR) module and semantic representation reinforcement (SemaRR) module. SurRR module fuses the facial features in different dimensions and highlights key feature messaging nodes in feature maps. SemaRR module establishes the deep semantic representation of global facial regions on spatial and channel dimensions. Therefore, RRN has stronger feature extraction capability when the parameters and computational complexity are significantly reduced, and TST is proposed for contributing to the training effect of RRN at the condition of insufficient training data. We tested our method on the extended Cohn-Kanade dataset (CK+), Radboud faces database (RaFD), FERPLUS, and the real-world affective faces database (RAF-DB) datasets, and the recognition accuracy are 100%, 98.62%, 89.64%, and 88.72%, respectively. Furthermore, the preliminary application experiment, which demonstrates the feasibility of our method in practice during HMI.

Effective Facial Expression Recognition via the Boosted Convolutional Neural Network

The Facial Expression Recognition Method Based on Image Fusion and CNN

Facial Expression Recognition Using Deep Neural Network

Optimizing Facial Expression Recognition: A One-Class Classification Approach Using ResNet18 and CBAM

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Ghost-based Convolutional Neural Network for Effective Facial Expression Recognition

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Facial Attention based Convolutional Neural Network for 2D+3D Facial Expression Recognition

Facial expression recognition with convolutional neural networks via a new face cropping and rotation strategy

A feature boosted deep learning method for automatic facial expression recognition

Facial expression recognition through multi-level features extraction and fusion

Facial Expression Recognition Based On Global And Local Feature Fusion With Cnns

Facial expression recognition using HOG and LBP features with convolutional neural network

Video-based Facial Expression Recognition Using Graph Convolutional Networks.

Uncertain and Biased Facial Expression Recognition Based on Depthwise Separable Convolutional Neural Network with Embedded Attention Mechanism

Facial Expression Recognition Based on Local Feature Fusion of Convolutional Neural Network

Regional Self-Attention Convolutional Neural Network for Facial Expression Recognition

Facial Expression Recognition Based on Convolutional Neural Network with Sparse Representation

A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition

Facial emotion recognition based on deep transfer learning approach

Fast and Efficient Facial Expression Recognition Using a Gabor Convolutional Network