Abstract:With the development of deep learning technology, the performance of facial expression recognition (FER) has been significantly improved. The current main challenge comes from the confusion of facial expressions caused by the highly nonlinear changes of facial expressions. However, the existing FER methods based on Convolutional Neural Networks (CNN) often ignore the underlying relationship between expressions which is crucial to meliorate the performance of recognition for confusable expressions. And the methods based on Graph Convolutional Networks (GCN) can capture the relationship between vertices, but the aggregation degree of subgraphs generated by these methods is low. They are easy to include unconfident neighbors, which increases the learning difficulty of the network. To solve the above problems, this paper proposes a method to recognize facial expressions on the high aggregation subgraphs (HASs) by combing the advantages of CNN extracting features and GCN modeling complex graph patterns. Specifically, we formulate FER as a vertex prediction problem. Considering the importance of high-order neighbors and higher efficiency, we utilize vertex confidence to find high-order neighbors. Then we construct the HASs based on the top embedding features of these high-order neighbors. And we utilize the GCN to perform reasoning and infer the class of vertices for HASs without a large number of overlapping subgraphs. Our method captures the underlying relationship between expressions on the HASs and improves the accuracy and efficiency of FER. Experimental results on both the in-the-lab datasets and the in-the-wild datasets show that our method achieves higher recognition accuracy than several state-of-the-art methods. This highlights the benefit of the underlying relationship between expressions for FER.

Learning Associative Representation for Facial Expression Recognition.

Cgan Based Facial Expression Recognition for Human-Robot Interaction

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Automatic 4D Facial Expression Recognition via Collaborative Cross-domain Dynamic Image Network.

Video-based Facial Expression Recognition Using Graph Convolutional Networks.

Joint Expression Synthesis and Representation Learning for Facial Expression Recognition

Joint Pose and Expression Modeling for Facial Expression Recognition

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Geometry-Aware Facial Expression Recognition Via Attentive Graph Convolutional Networks

FG-AGR: Fine-Grained Associative Graph Representation for Facial Expression Recognition in the Wild

Two-pathway attention network for real-time facial expression recognition

Facial Expression Recognition on the High Aggregation Subgraphs

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition

GCANet: Geometry Cues-aware Facial Expression Recognition based on Graph Convolutional Networks

Joint Deep Learning of Facial Expression Synthesis and Recognition

Learning Cognitive Features as Complementary for Facial Expression Recognition

Facial expression recognition through multi-level features extraction and fusion

Adaptively Learning Facial Expression Representation via C-F Labels and Distillation

Facial Expression Recognition Based on Local–global Information Reasoning and Spatial Distribution of Landmark Features