Abstract:Facial Action Unit (AU) detection is a crucial task in affective computing and social robotics as it helps to identify emotions expressed through facial expressions. Anatomically, there are innumerable correlations between AUs, which contain rich information and are vital for AU detection. Previous methods used fixed AU correlations based on expert experience or statistical rules on specific benchmarks, but it is challenging to comprehensively reflect complex correlations between AUs via hand-crafted settings. There are alternative methods that employ a fully connected graph to learn these dependencies exhaustively. However, these approaches can result in a computational explosion and high dependency with a large dataset. To address these challenges, this paper proposes a novel self-adjusting AU-correlation learning (SACL) method with less computation for AU detection. This method adaptively learns and updates AU correlation graphs by efficiently leveraging the characteristics of different levels of AU motion and emotion representation information extracted in different stages of the network. Moreover, this paper explores the role of multi-scale learning in correlation information extraction, and design a simple yet effective multi-scale feature learning (MSFL) method to promote better performance in AU detection. By integrating AU correlation information with multi-scale features, the proposed method obtains a more robust feature representation for the final AU detection. Extensive experiments show that the proposed method outperforms the state-of-the-art methods on widely used AU detection benchmark datasets, with only 28.7\% and 12.0\% of the parameters and FLOPs of the best method, respectively. The code for this method is available at \url{<a class="link-external link-https" href="https://github.com/linuxsino/Self-adjusting-AU" rel="external noopener nofollow">this https URL</a>}.

Improving Facial Attribute Recognition by Group and Graph Learning

Exploring Correlations in Multiple Facial Attributes Through Graph Attention Network

Grouped Multi-Task CNN for Facial Attribute Recognition

Learning to Infer Unseen Single-/ Multi-Attribute-Object Compositions with Graph Networks.

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

Geometry-Aware Facial Expression Recognition Via Attentive Graph Convolutional Networks

Collaborative Learning Network For Face Attribute Prediction

CaFGraph: Context-aware Facial Multi-graph Representation for Facial Action Unit Recognition.

Attributed Graph Clustering Network with Adaptive Feature Fusion

Multi-Level Adaptive Region of Interest and Graph Learning for Facial Action Unit Recognition

Improving Facial Attractiveness Prediction Via Co-attention Learning

Facial Expression Recognition on the High Aggregation Subgraphs

Facial Expression and Action Unit Recognition Augmented by Their Dependencies on Graph Convolutional Networks

Joint Multi-Feature Fusion and Attribute Relationships for Facial Attribute Prediction.

Learn to Cluster Faces with Better Subgraphs

GRNet: an Efficient Group Ranking Network for Face Attribute Recognition.

Learning Associative Representation for Facial Expression Recognition.

Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection

Facial Attribute Classification by Deep Mining Inter‐attribute Correlations

Facial Expression Recognition Method Based on a Part-Based Temporal Convolutional Network with a Graph-Structured Representation.

Robust facial expression recognition with global-local joint representation learning