Abstract:Facial expression data is characterized by a significant imbalance, with most collected data showing happy or neutral expressions and fewer instances of fear or disgust. This imbalance poses challenges to facial expression recognition (FER) models, hindering their ability to fully understand various human emotional states. Existing FER methods typically report overall accuracy on highly imbalanced test sets but exhibit low performance in terms of the mean accuracy across all expression classes. In this paper, our aim is to address the imbalanced FER problem. Existing methods primarily focus on learning knowledge of minor classes solely from minor-class samples. However, we propose a novel approach to extract extra knowledge related to the minor classes from both major and minor class samples. Our motivation stems from the belief that FER resembles a distribution learning task, wherein a sample may contain information about multiple classes. For instance, a sample from the major class surprise might also contain useful features of the minor class fear. Inspired by that, we propose a novel method that leverages re-balanced attention maps to regularize the model, enabling it to extract transformation invariant information about the minor classes from all training samples. Additionally, we introduce re-balanced smooth labels to regulate the cross-entropy loss, guiding the model to pay more attention to the minor classes by utilizing the extra information regarding the label distribution of the imbalanced training data. Extensive experiments on different datasets and backbones show that the two proposed modules work together to regularize the model and achieve state-of-the-art performance under the imbalanced FER task. Code is available at <a class="link-external link-https" href="https://github.com/zyh-uaiaaaa" rel="external noopener nofollow">this https URL</a>.

Facial Expression Recognition with Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing.

Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing for Facial Expression Recognition

Cgan Based Facial Expression Recognition for Human-Robot Interaction

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Enhanced Dual-Level Representations for Facial Expression Recognition

Facial Expression Recognition with Contrastive Learning and Uncertainty-Guided Relabeling

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Exploring Large-scale Unlabeled Faces to Enhance Facial Expression Recognition

Enhancing Facial Expression Recognition Under Data Uncertainty Based on Embedding Proximity

FG-AGR: Fine-Grained Associative Graph Representation for Facial Expression Recognition in the Wild

Towards A Robust Group-level Emotion Recognition via Uncertainty-Aware Learning

Facial Expression Recognition by Expression-Specific Representation Swapping

Geometric Graph Representation with Learnable Graph Structure and Adaptive AU Constraint for Micro-Expression Recognition

Leave No Stone Unturned: Mine Extra Knowledge for Imbalanced Facial Expression Recognition

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

The Devil is in the Face: Exploiting Harmonious Representations for Facial Expression Recognition

Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition.

Multi Loss-based Feature Fusion and Top Two Voting Ensemble Decision Strategy for Facial Expression Recognition in the Wild

Subtle expression recognition-a comprehensive approach

Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognition