Abstract:The emotion perception capability could endow machines with deeper feedback evaluation of human–machine interaction (HMI), which makes the experience of HMI more harmonious and natural. Currently, how to obtain the robust surface and semantic facial representation features has not been fully explored, moreover, the parameters and computation of conventional convolutional neural network models, which are adopted in facial expression recognition (FER) are too heavy, it is not conducive to the deployment of the HMI system. Therefore, this article proposes a representation reinforcement network (RRN) and transfer self-training (TST) based efficient FER method. Our designed RRN is mainly composed of two modules, i.e., surface representation reinforcement (SurRR) module and semantic representation reinforcement (SemaRR) module. SurRR module fuses the facial features in different dimensions and highlights key feature messaging nodes in feature maps. SemaRR module establishes the deep semantic representation of global facial regions on spatial and channel dimensions. Therefore, RRN has stronger feature extraction capability when the parameters and computational complexity are significantly reduced, and TST is proposed for contributing to the training effect of RRN at the condition of insufficient training data. We tested our method on the extended Cohn-Kanade dataset (CK+), Radboud faces database (RaFD), FERPLUS, and the real-world affective faces database (RAF-DB) datasets, and the recognition accuracy are 100%, 98.62%, 89.64%, and 88.72%, respectively. Furthermore, the preliminary application experiment, which demonstrates the feasibility of our method in practice during HMI.

Cross-domain Facial Expression Recognition Via an Intra-Category Common Feature and Inter-Category Distinction Feature Fusion Network

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

USTST: Unsupervised Self-Training Similarity Transfer for Cross-Domain Facial Expression Recognition

Facial Expression Recognition Through Cross-Modality Attention Fusion

Facial expression recognition through multi-level features extraction and fusion

A Multi-Scale Feature Fusion Network for Facial Expression Recognition

Facial Expression Recognition With Fusion Features Extracted From Salient Facial Areas

The Facial Expression Recognition Method Based on Image Fusion and CNN

CF-DAN: Facial-expression recognition based on cross-fusion dual-attention network

Multi-feature fusion network for facial expression recognition in the wild

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

TriCAFFNet: A Tri-Cross-Attention Transformer with a Multi-Feature Fusion Network for Facial Expression Recognition

Multi-Representation Joint Dynamic Domain Adaptation Network for Cross-Database Facial Expression Recognition

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP

Cross-Database Facial Expression Recognition With Domain Alignment And Compact Feature Learning

Cross-database facial expression recognition based on hybrid improved unsupervised domain adaptation

CFNet: Facial Expression Recognition Via Constraint Fusion under Multi-Task Joint Learning Network.

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

Cross-Domain Facial Expression Recognition via Contrastive Warm up and Complexity-Aware Self-Training