Abstract:We introduce a novel automatic detection method for facial action units (AUs) that leverages both spatial and temporal data, enhancing accuracy and robustness in expression analysis and facial animation. Our approach utilizes a Temporal feature Combination and Feature Reassignment (TC&FR) module to transform and fuse features across multiple subjects and temporal sequences. Moreover, by integrating a Regional Attention (RA) encoder and a transformer model, our method refines the extraction and processing of regional features, ensuring more precise identification and analysis of AUs. This integration not only harnesses identity‐independent features but also maximizes the temporal context, significantly improving the reliability of AU predictions. Facial action units (AUs) encode the activations of facial muscle groups, playing a crucial role in expression analysis and facial animation. However, current deep learning AU detection methods primarily focus on single‐image analysis, which limits the exploitation of rich temporal context for robust outcomes. Moreover, the scale of available datasets remains limited, leading models trained on these datasets to tend to suffer from overfitting issues. This paper proposes a novel AU detection method integrating spatial and temporal data with inter‐subject feature reassignment for accurate and robust AU predictions. Our method first extracts regional features from facial images. Then, to effectively capture both the temporal context and identity‐independent features, we introduce a temporal feature combination and feature reassignment (TC&FR) module, which transforms single‐image features into a cohesive temporal sequence and fuses features across multiple subjects. This transformation encourages the model to utilize identity‐independent features and temporal context, thus ensuring robust prediction outcomes. Experimental results demonstrate the enhancements brought by the proposed modules and the state‐of‐the‐art (SOTA) results achieved by our method.

USAN-AAM method for fast human facial features location

A new facial feature based on the fusion of texture and shape characteristics

A Method for Accurate Localization of Facial Features

Adaptive region enhancement based facial feature localization under complex illumination

Robust Three-step Facial Landmark Localization under the Complicated Condition via ASM and POEM.

Improved Active Shape Model for Facial Feature Localization Using Poem Descriptor

Facial Feature Point Localization Algorithm Combining Improved ASM and AAM

Facial Features Extraction Based on Active Shape Model.

AAM-Based Face Modeling and Landmark Selection

Automatic 3D Face Modeling Using 2D Active Appearance Models

A New Method of Human Eye Location Based on Face Geometry Character

A System for Human Face and Facial Feature Location.

Multi-view face alignment guided by several facial feature points

Subspace Analysis And Optimization For Aam Based Face Alignment

Face recognition via AAM and multi-features fusion on riemannian manifolds

Face Anti-Spoofing with Human Material Perception

A Fast and Robust Automatic Face Alignment System

Facial action units detection using temporal context and feature reassignment

Regional Feature Based Fast Human Face Detection

Automatic Localization of the Vital Feature Points on Human Faces

A Robust Real-Time System for Multi-Intensity AU Detection in Facial Expression Recognition