Abstract:Facial expression recognition (FER) plays a crucial role in affective computing, enhancing human-computer interaction by enabling machines to understand and respond to human emotions. Despite advancements in deep learning, current FER systems often struggle with challenges such as occlusions, head pose variations, and motion blur in natural environments. These challenges highlight the need for more robust FER solutions. To address these issues, we propose the Attention-Enhanced Multi-Layer Transformer (AEMT) model, which integrates a dual-branch Convolutional Neural Network (CNN), an Attentional Selective Fusion (ASF) module, and a Multi-Layer Transformer Encoder (MTE) with transfer learning. The dual-branch CNN captures detailed texture and color information by processing RGB and Local Binary Pattern (LBP) features separately. The ASF module selectively enhances relevant features by applying global and local attention mechanisms to the extracted features. The MTE captures long-range dependencies and models the complex relationships between features, collectively improving feature representation and classification accuracy. Our model was evaluated on the RAF-DB and AffectNet datasets. Experimental results demonstrate that the AEMT model achieved an accuracy of 81.45% on RAF-DB and 71.23% on AffectNet, significantly outperforming existing state-of-the-art methods. These results indicate that our model effectively addresses the challenges of FER in natural environments, providing a more robust and accurate solution. The AEMT model significantly advances the field of FER by improving the robustness and accuracy of emotion recognition in complex real-world scenarios. This work not only enhances the capabilities of affective computing systems but also opens new avenues for future research in improving model efficiency and expanding multimodal data integration.

AVT: Au-Assisted Visual Transformer for Facial Expression Recognition

Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Facial Expression Recognition Based on Multi-Scale Convolutional Vision Transformer

Facial Action Units Detection Aided by Global-Local Expression Embedding

Facial Expression Recognition Based on Fine-Tuned Channel–Spatial Attention Transformer

FG-AGR: Fine-Grained Associative Graph Representation for Facial Expression Recognition in the Wild

Facial Expression Recognition Based on Facial Action Unit

Evaluation and analysis of visual perception using attention-enhanced computation in multimedia affective computing

MViT: Mask Vision Transformer for Facial Expression Recognition in the Wild

FER-former: Multi-modal Transformer for Facial Expression Recognition

VaBTFER: An Effective Variant Binary Transformer for Facial Expression Recognition

MFEViT: A Robust Lightweight Transformer-based Network for Multimodal 2D+3D Facial Expression Recognition

Facial expression recognition through multi-level features extraction and fusion

Robust facial expression recognition with Transformer Block Enhancement Module

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Fine-Grained Temporal-Enhanced Transformer for Dynamic Facial Expression Recognition

Attention on Emotions: A Vision Transformer Approach to Advancing Facial Expression Recognition

Comparative Analysis of Vision Transformer Models for Facial Emotion Recognition Using Augmented Balanced Datasets

Emotic Masked Autoencoder with Attention Fusion for Facial Expression Recognition