Abstract:With the development of deep learning technology, the performance of facial expression recognition (FER) has been significantly improved. The current main challenge comes from the confusion of facial expressions caused by the highly nonlinear changes of facial expressions. However, the existing FER methods based on Convolutional Neural Networks (CNN) often ignore the underlying relationship between expressions which is crucial to meliorate the performance of recognition for confusable expressions. And the methods based on Graph Convolutional Networks (GCN) can capture the relationship between vertices, but the aggregation degree of subgraphs generated by these methods is low. They are easy to include unconfident neighbors, which increases the learning difficulty of the network. To solve the above problems, this paper proposes a method to recognize facial expressions on the high aggregation subgraphs (HASs) by combing the advantages of CNN extracting features and GCN modeling complex graph patterns. Specifically, we formulate FER as a vertex prediction problem. Considering the importance of high-order neighbors and higher efficiency, we utilize vertex confidence to find high-order neighbors. Then we construct the HASs based on the top embedding features of these high-order neighbors. And we utilize the GCN to perform reasoning and infer the class of vertices for HASs without a large number of overlapping subgraphs. Our method captures the underlying relationship between expressions on the HASs and improves the accuracy and efficiency of FER. Experimental results on both the in-the-lab datasets and the in-the-wild datasets show that our method achieves higher recognition accuracy than several state-of-the-art methods. This highlights the benefit of the underlying relationship between expressions for FER.

The Devil is in the Face: Exploiting Harmonious Representations for Facial Expression Recognition

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Cgan Based Facial Expression Recognition for Human-Robot Interaction

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

2D+3D Facial Expression Recognition via Discriminative Dynamic Range Enhancement and Multi-Scale Learning

Enhanced Dual-Level Representations for Facial Expression Recognition

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

Facial Expression Recognition by Expression-Specific Representation Swapping

Joint Expression Synthesis and Representation Learning for Facial Expression Recognition

Geometry-Aware Facial Expression Recognition Via Attentive Graph Convolutional Networks

Knowledge-Enhanced Facial Expression Recognition with Emotional-to-Neutral Transformation

Deep Representation of Facial Geometric and Photometric Attributes for Automatic 3D Facial Expression Recognition

Harmonious Mutual Learning for Facial Emotion Recognition

Learning Cognitive Features as Complementary for Facial Expression Recognition

Facial Expression Recognition on the High Aggregation Subgraphs

FG-AGR: Fine-Grained Associative Graph Representation for Facial Expression Recognition in the Wild

Joint Pose and Expression Modeling for Facial Expression Recognition

Visual Scene-Aware Hybrid and Multi-Modal Feature Aggregation for Facial Expression Recognition

Facial Expression Recognition: a Comparison with Different Classical and Deep Learning Methods