Abstract:With the development of deep learning technology, the performance of facial expression recognition (FER) has been significantly improved. The current main challenge comes from the confusion of facial expressions caused by the highly nonlinear changes of facial expressions. However, the existing FER methods based on Convolutional Neural Networks (CNN) often ignore the underlying relationship between expressions which is crucial to meliorate the performance of recognition for confusable expressions. And the methods based on Graph Convolutional Networks (GCN) can capture the relationship between vertices, but the aggregation degree of subgraphs generated by these methods is low. They are easy to include unconfident neighbors, which increases the learning difficulty of the network. To solve the above problems, this paper proposes a method to recognize facial expressions on the high aggregation subgraphs (HASs) by combing the advantages of CNN extracting features and GCN modeling complex graph patterns. Specifically, we formulate FER as a vertex prediction problem. Considering the importance of high-order neighbors and higher efficiency, we utilize vertex confidence to find high-order neighbors. Then we construct the HASs based on the top embedding features of these high-order neighbors. And we utilize the GCN to perform reasoning and infer the class of vertices for HASs without a large number of overlapping subgraphs. Our method captures the underlying relationship between expressions on the HASs and improves the accuracy and efficiency of FER. Experimental results on both the in-the-lab datasets and the in-the-wild datasets show that our method achieves higher recognition accuracy than several state-of-the-art methods. This highlights the benefit of the underlying relationship between expressions for FER.

Exploring Correlations in Multiple Facial Attributes Through Graph Attention Network

Improving Facial Attribute Recognition by Group and Graph Learning

A Novel Multi-Task Tensor Correlation Neural Network for Facial Attribute Prediction

Learning to Infer Unseen Single-/ Multi-Attribute-Object Compositions with Graph Networks.

Faceknow: Facial Expression Recognition by a Global-Local Network with a Sub-Images-Related Contextual Attention Mechanism

Grouped Multi-Task CNN for Facial Attribute Recognition

Collaborative Learning Network For Face Attribute Prediction

Joint Multi-Feature Fusion and Attribute Relationships for Facial Attribute Prediction.

A joint hierarchical cross‐attention graph convolutional network for multi‐modal facial expression recognition

Facial Attribute Classification by Deep Mining Inter‐attribute Correlations

Attention-Driven Graph Neural Network for Deep Face Super-Resolution

Face2Nodes: Learning Facial Expression Representations with Relation-Aware Dynamic Graph Convolution Networks

Learning an attention-aware parallel sharing network for facial attribute recognition

Attributes for Improved Attributes: A Multi-Task Network for Attribute Classification

Multi-task Learning of Cascaded CNN for Facial Attribute Classification.

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

Robust facial expression recognition with global-local joint representation learning

Features-Enhanced Multi-Attribute Estimation with Convolutional Tensor Correlation Fusion Network

Facial Expression Recognition on the High Aggregation Subgraphs

Geometry-Aware Facial Expression Recognition Via Attentive Graph Convolutional Networks

MVMA-GCN: Multi-view Multi-Layer Attention Graph Convolutional Networks