Abstract:The Facial Action Coding System (FACS) encodes the action units (AUs) in facial images, which has attracted extensive research attention due to its wide use in facial expression analysis. Many methods that perform well on automatic facial action unit (AU) detection primarily focus on modelling various AU relations between corresponding local muscle areas or mining global attention-aware facial features; however, they neglect the dynamic interactions among local-global features. We argue that encoding AU features just from one perspective may not capture the rich contextual information between regional and global face features, as well as the detailed variability across AUs, because of the diversity in expression and individual characteristics. In this paper, we propose a novel Multi-level Graph Relational Reasoning Network (termed MGRR-Net ) for facial AU detection. Each layer of MGRR-Net performs a multi-level ( i.e. , region-level, pixel-wise and channel-wise level) feature learning. On the one hand, the region-level feature learning from the local face patch features via graph neural network can encode the correlation across different AUs. On the other hand, pixel-wise and channel-wise feature learning via graph attention networks (GAT) enhance the discrimination ability of AU features by adaptively recalibrating feature responses of pixels and channels from global face features. The hierarchical fusion strategy combines features from the three levels with gated fusion cells to improve AU discriminative ability. Extensive experiments on DISFA and BP4D AU datasets show that the proposed approach achieves superior performance than the state-of-the-art methods.

Facial Action Unit recognition by relation modeling from both qualitative knowledge and quantitative data

A Novel Dual-channel Graph Convolutional Neural Network for Facial Action Unit Recognition

Facial Action Unit Classification with Hidden Knowledge under Incomplete Annotation

Facial Action Unit Recognition and Intensity Estimation Enhanced Through Label Dependencies

Attention Based Relation Network for Facial Action Units Recognition

Feature and Label Relation Modeling for Multiple-Facial Action Unit Classification and Intensity Estimation

Expression-assisted Facial Action Unit Recognition under Incomplete AU Annotation.

Capture expression-dependent AU relations for expression recognition

Knowledge Augmented Deep Neural Networks for Joint Facial Expression and Action Unit Recognition.

Multiple Facial Action Unit Recognition by Learning Joint Features and Label Relations.

Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

Capturing Global Semantic Relationships for Facial Action Unit Recognition

Facial Action Unit Detection Via Adaptive Attention and Relation.

Facial Action Unit Detection Via Hybrid Relational Reasoning

An Action Unit Co-Occurrence Constraint 3DCNN Based Action Unit Recognition Approach.

Facial Action Units Detection Aided by Global-Local Expression Embedding

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

Knowledge-Driven Self-Supervised Representation Learning for Facial Action Unit Recognition

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

AU R-CNN: Encoding Expert Prior Knowledge into R-CNN for Action Unit Detection

MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Unit Detection