Abstract:Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics as it helps to identify emotions expressed through facial expressions. Anatomically, there are innumerable correlations between AUs, which contain rich information and are vital for AU detection. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks, but it is challenging to comprehensively reflect complex correlations between AUs via hand-crafted settings. There are alternative methods that employ a fully connected graph to learn these dependencies exhaustively. However, these approaches can result in a computational explosion and high dependency with a large dataset. To address these challenges, this paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less computation for AU detection. This method adaptively learns and updates AU correlation graphs by efficiently leveraging the characteristics of different levels of AU motion and emotion representation information extracted in different stages of the network. Moreover, this paper explores the role of multi-scale learning in correlation information extraction, and design a simple yet effective multi-scale feature learning (MSFL) method to promote better performance in AU detection. By integrating AU correlation information with multi-scale features, the proposed method obtains a more robust feature representation for the final AU detection. Extensive experiments show that the proposed method outperforms the state-of-the-art methods on widely used AU detection benchmark datasets, with only 28.7\% and 12.0\% of the parameters and FLOPs of the best method, respectively. The code for this method is available at \url{<a class="link-external link-https" href="https://github.com/linuxsino/Self-adjusting-AU" rel="external noopener nofollow">this https URL</a>}.

Graph-Based Multi-Modal Multi-View Fusion for Facial Action Unit Recognition

Facial Action Unit Detection with Multilayer Fused Multi-Task and Multi-Label Deep Learning Network

Region And Temporal Dependency Fusion For Multi-Label Action Unit Detection

View-Independent Facial Action Unit Detection.

Multi-view Facial Action Unit Detection Via Deep Feature Enhancement

Facial Action Units Detection with Multi-Features and -AUs Fusion

MMA-Net: Multi-view Mixed Attention Mechanism for Facial Action Unit Detection

Facial Action Units Detection Aided by Global-Local Expression Embedding

Facial Action Unit Detection Via Adaptive Attention and Relation.

A Novel Dual-channel Graph Convolutional Neural Network for Facial Action Unit Recognition

Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection

Facial Action Unit Recognition Based on Self-Attention Spatiotemporal Fusion

MultiFuser: Multimodal Fusion Transformer for Enhanced Driver Action Recognition

Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection

Action Unit Detection with Joint Adaptive Attention and Graph Relation

Multi View Facial Action Unit Detection Based on CNN and BLSTM-RNN.

Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition

A Robust Real-Time System for Multi-Intensity AU Detection in Facial Expression Recognition

Spatio-Temporal AU Relational Graph Representation Learning For Facial Action Units Detection

CaFGraph: Context-aware Facial Multi-graph Representation for Facial Action Unit Recognition.

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection