Abstract:With the development of deep learning technology, the performance of facial expression recognition (FER) has been significantly improved. The current main challenge comes from the confusion of facial expressions caused by the highly nonlinear changes of facial expressions. However, the existing FER methods based on Convolutional Neural Networks (CNN) often ignore the underlying relationship between expressions which is crucial to meliorate the performance of recognition for confusable expressions. And the methods based on Graph Convolutional Networks (GCN) can capture the relationship between vertices, but the aggregation degree of subgraphs generated by these methods is low. They are easy to include unconfident neighbors, which increases the learning difficulty of the network. To solve the above problems, this paper proposes a method to recognize facial expressions on the high aggregation subgraphs (HASs) by combing the advantages of CNN extracting features and GCN modeling complex graph patterns. Specifically, we formulate FER as a vertex prediction problem. Considering the importance of high-order neighbors and higher efficiency, we utilize vertex confidence to find high-order neighbors. Then we construct the HASs based on the top embedding features of these high-order neighbors. And we utilize the GCN to perform reasoning and infer the class of vertices for HASs without a large number of overlapping subgraphs. Our method captures the underlying relationship between expressions on the HASs and improves the accuracy and efficiency of FER. Experimental results on both the in-the-lab datasets and the in-the-wild datasets show that our method achieves higher recognition accuracy than several state-of-the-art methods. This highlights the benefit of the underlying relationship between expressions for FER.

Facial micro-expression recognition using stochastic graph convolutional network and dual transferred learning

Transferring Dual Stochastic Graph Convolutional Network for Facial Micro-expression Recognition

MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network

Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition

Dual-stream network with cross-layer attention and similarity constraint for micro-expression recognition

Video-based Facial Expression Recognition Using Graph Convolutional Networks.

LCBP-STGCN: A local cube binary pattern spatial temporal graph convolutional network for micro-expression recognition

SSGNN: A Macro and Microfacial Expression Recognition Graph Neural Network Combining Spatial and Spectral Domain Features

Facial Expression Recognition on the High Aggregation Subgraphs

Objective Class-based Micro-Expression Recognition through Simultaneous Action Unit Detection and Feature Aggregation

Adaptive Temporal Motion Guided Graph Convolution Network for Micro-expression Recognition

From Macro to Micro: Boosting micro-expression recognition via pre-training on macro-expression videos

Facial micro-expression recognition based on motion magnification network and graph attention mechanism

A Multi-stream Convolutional Neural Network for Micro-expression Recognition Using Optical Flow and EVM

Robust facial expression recognition with global-local joint representation learning

GCANet: Geometry Cues-aware Facial Expression Recognition based on Graph Convolutional Networks

Micro-Expression Recognition Based on Multi-task Learning and Resnet18

Micro-expression Recognition Based on Facial Graph Representation Learning and Facial Action Unit Fusion

AU-assisted Graph Attention Convolutional Network for Micro-Expression Recognition

MFDAN: Multi-level Flow-Driven Attention Network for Micro-Expression Recognition

Facial micro-expression recognition using three-stream vision transformer network with sparse sampling and relabeling