Abstract:Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics as it helps to identify emotions expressed through facial expressions. Anatomically, there are innumerable correlations between AUs, which contain rich information and are vital for AU detection. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks, but it is challenging to comprehensively reflect complex correlations between AUs via hand-crafted settings. There are alternative methods that employ a fully connected graph to learn these dependencies exhaustively. However, these approaches can result in a computational explosion and high dependency with a large dataset. To address these challenges, this paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less computation for AU detection. This method adaptively learns and updates AU correlation graphs by efficiently leveraging the characteristics of different levels of AU motion and emotion representation information extracted in different stages of the network. Moreover, this paper explores the role of multi-scale learning in correlation information extraction, and design a simple yet effective multi-scale feature learning (MSFL) method to promote better performance in AU detection. By integrating AU correlation information with multi-scale features, the proposed method obtains a more robust feature representation for the final AU detection. Extensive experiments show that the proposed method outperforms the state-of-the-art methods on widely used AU detection benchmark datasets, with only 28.7\% and 12.0\% of the parameters and FLOPs of the best method, respectively. The code for this method is available at \url{<a class="link-external link-https" href="https://github.com/linuxsino/Self-adjusting-AU" rel="external noopener nofollow">this https URL</a>}.

Joint Patch And Multi-Label Learning For Facial Action Unit Detection

Facial Action Unit Detection with Local Key Facial Sub-region Based Multi-label Classification for Micro-expression Analysis.

Region And Temporal Dependency Fusion For Multi-Label Action Unit Detection

Feature and Label Relation Modeling for Multiple-Facial Action Unit Classification and Intensity Estimation

Facial Action Units Detection Aided by Global-Local Expression Embedding

Facial Action Unit Recognition under Incomplete Data Based on Multi-Label Learning with Missing Labels.

Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment

Multiple Facial Action Unit Recognition by Learning Joint Features and Label Relations.

Dual Learning for Joint Facial Landmark Detection and Action Unit Recognition

Multi-label Learning with Missing Values using Combined Facial Action Unit Datasets

Deep Facial Action Unit Recognition from Partially Labeled Data.

Deep Facial Action Unit Recognition and Intensity Estimation from Partially Labelled Data

Knowledge-Spreader: Learning Facial Action Unit Dynamics with Extremely Limited Labels

Facial Action Unit Classification with Hidden Knowledge under Incomplete Annotation

Facial Action Unit Detection Using Attention and Relation Learning

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

Semi Supervised Unconstrained Action Unit Detection via Latent Feature Domain

Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction

Knowledge-Driven Self-Supervised Representation Learning for Facial Action Unit Recognition

Region Attentive Action Unit Intensity Estimation with Uncertainty Weighted Multi-task Learning