Abstract:Facial expression recognition (FER) plays a crucial role in affective computing, enhancing human-computer interaction by enabling machines to understand and respond to human emotions. Despite advancements in deep learning, current FER systems often struggle with challenges such as occlusions, head pose variations, and motion blur in natural environments. These challenges highlight the need for more robust FER solutions. To address these issues, we propose the Attention-Enhanced Multi-Layer Transformer (AEMT) model, which integrates a dual-branch Convolutional Neural Network (CNN), an Attentional Selective Fusion (ASF) module, and a Multi-Layer Transformer Encoder (MTE) with transfer learning. The dual-branch CNN captures detailed texture and color information by processing RGB and Local Binary Pattern (LBP) features separately. The ASF module selectively enhances relevant features by applying global and local attention mechanisms to the extracted features. The MTE captures long-range dependencies and models the complex relationships between features, collectively improving feature representation and classification accuracy. Our model was evaluated on the RAF-DB and AffectNet datasets. Experimental results demonstrate that the AEMT model achieved an accuracy of 81.45% on RAF-DB and 71.23% on AffectNet, significantly outperforming existing state-of-the-art methods. These results indicate that our model effectively addresses the challenges of FER in natural environments, providing a more robust and accurate solution. The AEMT model significantly advances the field of FER by improving the robustness and accuracy of emotion recognition in complex real-world scenarios. This work not only enhances the capabilities of affective computing systems but also opens new avenues for future research in improving model efficiency and expanding multimodal data integration.

Attention-Rectified and Texture-Enhanced Cross-Attention Transformer Feature Fusion Network for Facial Expression Recognition

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Evaluation and analysis of visual perception using attention-enhanced computation in multimedia affective computing

Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion

Facial expression recognition through multi-level features extraction and fusion

TriCAFFNet: A Tri-Cross-Attention Transformer with a Multi-Feature Fusion Network for Facial Expression Recognition

Facial Expression Recognition Based on Fine-Tuned Channel–Spatial Attention Transformer

USTST: Unsupervised Self-Training Similarity Transfer for Cross-Domain Facial Expression Recognition

CF-DAN: Facial-expression recognition based on cross-fusion dual-attention network

The Facial Expression Recognition Method Based on Image Fusion and CNN

Adaptive multilayer perceptual attention network for facial expression recognition

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

An Enhanced Facial Expression Recognition Method Based on Multi-task Learning

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

Multi-feature fusion network for facial expression recognition in the wild

CFNet: Facial Expression Recognition Via Constraint Fusion under Multi-Task Joint Learning Network.

A facial expression recognition network based on attention double branch enhanced fusion

Facial Expression Recognition Through Cross-Modality Attention Fusion

Multi-level Spatial and Semantic Enhancement Network for Expression Recognition

A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition