Abstract:Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics as it helps to identify emotions expressed through facial expressions. Anatomically, there are innumerable correlations between AUs, which contain rich information and are vital for AU detection. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks, but it is challenging to comprehensively reflect complex correlations between AUs via hand-crafted settings. There are alternative methods that employ a fully connected graph to learn these dependencies exhaustively. However, these approaches can result in a computational explosion and high dependency with a large dataset. To address these challenges, this paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less computation for AU detection. This method adaptively learns and updates AU correlation graphs by efficiently leveraging the characteristics of different levels of AU motion and emotion representation information extracted in different stages of the network. Moreover, this paper explores the role of multi-scale learning in correlation information extraction, and design a simple yet effective multi-scale feature learning (MSFL) method to promote better performance in AU detection. By integrating AU correlation information with multi-scale features, the proposed method obtains a more robust feature representation for the final AU detection. Extensive experiments show that the proposed method outperforms the state-of-the-art methods on widely used AU detection benchmark datasets, with only 28.7\% and 12.0\% of the parameters and FLOPs of the best method, respectively. The code for this method is available at \url{<a class="link-external link-https" href="https://github.com/linuxsino/Self-adjusting-AU" rel="external noopener nofollow">this https URL</a>}.

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

A Novel Dual-channel Graph Convolutional Neural Network for Facial Action Unit Recognition

Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

Facial Action Units Detection Aided by Global-Local Expression Embedding

MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection

AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition

MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Unit Detection

Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition

Capturing Global Semantic Relationships for Facial Action Unit Recognition

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection

CaFGraph: Context-aware Facial Multi-graph Representation for Facial Action Unit Recognition.

MER-GCN: Micro Expression Recognition Based on Relation Modeling with Graph Convolutional Network

Learning facial expression-aware global-to-local representation for robust action unit detection

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

Knowledge-Driven Self-Supervised Representation Learning for Facial Action Unit Recognition

Capture expression-dependent AU relations for expression recognition

Personalized Multiple Facial Action Unit Recognition through Generative Adversarial Recognition Network.

Facial Action Unit Detection Using Attention and Relation Learning

Deep Structure Inference Network for Facial Action Unit Recognition

Facial Action Unit Detection by Exploring the Weak Relationships Between AU Labels