Abstract:We introduce a novel automatic detection method for facial action units (AUs) that leverages both spatial and temporal data, enhancing accuracy and robustness in expression analysis and facial animation. Our approach utilizes a Temporal feature Combination and Feature Reassignment (TC&FR) module to transform and fuse features across multiple subjects and temporal sequences. Moreover, by integrating a Regional Attention (RA) encoder and a transformer model, our method refines the extraction and processing of regional features, ensuring more precise identification and analysis of AUs. This integration not only harnesses identity‐independent features but also maximizes the temporal context, significantly improving the reliability of AU predictions. Facial action units (AUs) encode the activations of facial muscle groups, playing a crucial role in expression analysis and facial animation. However, current deep learning AU detection methods primarily focus on single‐image analysis, which limits the exploitation of rich temporal context for robust outcomes. Moreover, the scale of available datasets remains limited, leading models trained on these datasets to tend to suffer from overfitting issues. This paper proposes a novel AU detection method integrating spatial and temporal data with inter‐subject feature reassignment for accurate and robust AU predictions. Our method first extracts regional features from facial images. Then, to effectively capture both the temporal context and identity‐independent features, we introduce a temporal feature combination and feature reassignment (TC&FR) module, which transforms single‐image features into a cohesive temporal sequence and fuses features across multiple subjects. This transformation encourages the model to utilize identity‐independent features and temporal context, thus ensuring robust prediction outcomes. Experimental results demonstrate the enhancements brought by the proposed modules and the state‐of‐the‐art (SOTA) results achieved by our method.

Evaluation of A Case-Based Facial Action Units Recognition Approach

Algorithm of Case-based Facial Action Units Recognition

Case-Based facial action units recognition using interactive genetic algorithm

Face Feature Extraction Method Based on CGA and ICA

Facial Action Units Recognition: A Comparative Study

Facial Expression Recognition Based on Facial Action Unit

Multiple Facial Action Unit Recognition Enhanced by Facial Expressions.

Action unit analysis enhanced facial expression recognition by deep neural network evolution

Capture expression-dependent AU relations for expression recognition

A COMPARATIVE STUDY OF HMMS AND DBNS APPLIED TO FACIAL ACTION UNITS RECOGNITION

Facial Action Units Detection Aided by Global-Local Expression Embedding

Facial Action Unit Detection with Local Key Facial Sub-region Based Multi-label Classification for Micro-expression Analysis.

A Robust Real-Time System for Multi-Intensity AU Detection in Facial Expression Recognition

Fully Automatic Upper Facial Action Recognition.

Personalized Multiple Facial Action Unit Recognition through Generative Adversarial Recognition Network.

Facial action units detection using temporal context and feature reassignment

Expression-assisted Facial Action Unit Recognition under Incomplete AU Annotation.

Facial Action Unit Recognition Enhanced by Text Descriptions of FACS

Data-aware relation learning-based graph convolution neural network for facial action unit recognition

AU recognition on 3D faces based on an extended statistical facial feature model

Relative Facial Action Unit Detection