Abstract:Facial expression-based Emotion Recognition (FER) is crucial in human–computer interaction and affective computing, particularly when addressing diverse age groups. This paper introduces the Multi-Scale Vision Transformer with Contrastive Learning (MViT-CnG), an age-adaptive FER approach designed to enhance the accuracy and interpretability of emotion recognition models across different classes. The MViT-CnG model leverages vision transformers and contrastive learning to capture intricate facial features, ensuring robust performance despite diverse and dynamic facial features. By utilizing contrastive learning, the model's interpretability is significantly enhanced, which is vital for building trust in automated systems and facilitating human–machine collaboration. Additionally, this approach enriches the model's capacity to discern shared and distinct features within facial expressions, improving its ability to generalize across different age groups. Evaluations using the FER-2013 and CK + datasets highlight the model's broad generalization capabilities, with FER-2013 covering a wide range of emotions across diverse age groups and CK + focusing on posed expressions in controlled environments. The MViT-CnG model adapts effectively to both datasets, showcasing its versatility and reliability across distinct data characteristics. Performance results demonstrated that the MViT-CnG model achieved superior accuracy across all emotion recognition labels on the FER-2013 dataset with a 99.6% accuracy rate, and 99.5% on the CK + dataset, indicating significant improvements in recognizing subtle facial expressions. Comprehensive evaluations revealed that the model's precision, recall, and F1-score are consistently higher than those of existing models, confirming its robustness and reliability in facial emotion recognition tasks.

Comparative Analysis of Vision Transformer Models for Facial Emotion Recognition Using Augmented Balanced Datasets

Attention on Emotions: A Vision Transformer Approach to Advancing Facial Expression Recognition

Facial Expression Recognition Based on Multi-Scale Convolutional Vision Transformer

Facial expression-based emotion recognition across diverse age groups: a multi-scale vision transformer with contrastive learning approach

MViT: Mask Vision Transformer for Facial Expression Recognition in the Wild

Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion

MFEViT: A Robust Lightweight Transformer-based Network for Multimodal 2D+3D Facial Expression Recognition

Personalized Speech Emotion Recognition in Human-Robot Interaction using Vision Transformers

VaBTFER: An Effective Variant Binary Transformer for Facial Expression Recognition

An enhanced speech emotion recognition using vision transformer

AVT: Au-Assisted Visual Transformer for Facial Expression Recognition

CNN-Transformer Architecture Solution for Compound Facial Expression Recognition

Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers

Comparative analysis of facial emotion recognition

Variance-Aware Bi-Attention Expression Transformer for Open-Set Facial Expression Recognition in the Wild

Quaternion Orthogonal Transformer for Facial Expression Recognition in the Wild

The Extensive Usage of the Facial Image Threshing Machine for Facial Emotion Recognition Performance

Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformers

MIMIC: Mask Image Pre-training with Mix Contrastive Fine-tuning for Facial Expression Recognition

A Self-Attention Based Method for Facial Expression Recognition