Abstract:Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics as it helps to identify emotions expressed through facial expressions. Anatomically, there are innumerable correlations between AUs, which contain rich information and are vital for AU detection. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks, but it is challenging to comprehensively reflect complex correlations between AUs via hand-crafted settings. There are alternative methods that employ a fully connected graph to learn these dependencies exhaustively. However, these approaches can result in a computational explosion and high dependency with a large dataset. To address these challenges, this paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less computation for AU detection. This method adaptively learns and updates AU correlation graphs by efficiently leveraging the characteristics of different levels of AU motion and emotion representation information extracted in different stages of the network. Moreover, this paper explores the role of multi-scale learning in correlation information extraction, and design a simple yet effective multi-scale feature learning (MSFL) method to promote better performance in AU detection. By integrating AU correlation information with multi-scale features, the proposed method obtains a more robust feature representation for the final AU detection. Extensive experiments show that the proposed method outperforms the state-of-the-art methods on widely used AU detection benchmark datasets, with only 28.7\% and 12.0\% of the parameters and FLOPs of the best method, respectively. The code for this method is available at \url{<a class="link-external link-https" href="https://github.com/linuxsino/Self-adjusting-AU" rel="external noopener nofollow">this https URL</a>}.

Multi-Scale Region with Local Relationship Learning for Facial Action Unit Detection

Facial Action Units Detection Aided by Global-Local Expression Embedding

Deep Region And Multi-Label Learning For Facial Action Unit Detection

Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition

MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Unit Detection

MGRR-Net: Multi-level Graph Relational Reasoning Network for Facial Action Units Detection

Facial Action Unit Detection Using Attention and Relation Learning

Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition

Region And Temporal Dependency Fusion For Multi-Label Action Unit Detection

Facial Action Unit Detection Via Adaptive Attention and Relation.

Facial Action Unit Detection with Local Key Facial Sub-region Based Multi-label Classification for Micro-expression Analysis.

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection

Joint Patch And Multi-Label Learning For Facial Action Unit Detection

MMA-Net: Multi-view Mixed Attention Mechanism for Facial Action Unit Detection

Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection

MRP-Net: A Light Multiple Region Perception Neural Network for Multi-label AU Detection.

Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction

Facial Action Unit Detection with Multilayer Fused Multi-Task and Multi-Label Deep Learning Network

Self-Supervised Regional and Temporal Auxiliary Tasks for Facial Action Unit Recognition

Learning facial expression-aware global-to-local representation for robust action unit detection

Detecting Facial Action Units from Global-Local Fine-grained Expressions