Abstract:Learning discriminative and robust representations is important for facial expression recognition (FER) due to subtly different emotional faces and their subjective annotations. Previous works usually address one representation solely because these two goals seem to be contradictory for optimization. Their performances inevitably suffer from challenges from the other representation. In this article, by considering this problem from two novel perspectives, we demonstrate that discriminative and robust representations can be learned in a unified approach, i.e., DR-FER, and mutually benefit each other. Moreover, we make it with the supervision from only original annotations. Specifically, to learn discriminative representations, we propose performing masked image modeling (MIM) as an auxiliary task to force our network to discover expression-related facial areas. This is the first attempt to employ MIM to explore discriminative patterns in a self-supervised manner. To extract robust representations, we present a category-aware self-paced learning schedule to mine high-quality annotated (easy) expressions and incorrectly annotated (hard) counterparts. We further introduce a retrieval similarity-based relabeling strategy to correct hard expression annotations, exploiting them more effectively. By enhancing the discrimination ability of the FER classifier as a bridge, these two learning goals significantly strengthen each other. Extensive experiments on several popular benchmarks demonstrate the superior performance of our DR-FER. Moreover, thorough visualizations and extra experiments on manually annotation-corrupted datasets show that our approach successfully accomplishes learning both discriminative and robust representations simultaneously.

Representation ensemble learning applied to facial expression recognition

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

A Novel Multi-Feature Joint Learning Ensemble Framework for Multi-Label Facial Expression Recognition

Visual-Audio Emotion Recognition Based on Multi-Task and Ensemble Learning with Multiple Features

Efficient Facial Feature Learning with Wide Ensemble-Based Convolutional Neural Networks

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Emotic Masked Autoencoder with Attention Fusion for Facial Expression Recognition

AU-Expression Knowledge Constrained Representation Learning for Facial Expression Recognition

Adaptively Learning Facial Expression Representation via C-F Labels and Distillation

Learning Associative Representation for Facial Expression Recognition.

Facial Expression Recognition Based on Ensemble of Mulitple CNNs.

Multi Loss-based Feature Fusion and Top Two Voting Ensemble Decision Strategy for Facial Expression Recognition in the Wild

Improving Ensemble Learning Performance With Complementary Neural Networks For Facial Expression Recognition

Facial Emotion Detection Using Convolutional Neural Networks and Representational Autoencoder Units

The Devil is in the Face: Exploiting Harmonious Representations for Facial Expression Recognition

Multi-model Ensemble Learning Method for Human Expression Recognition

Facial expression recognition and generation using sparse autoencoder

Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition.

Global and Local Fusion Ensemble Network for Facial Expression Recognition

Regional Self-Attention Convolutional Neural Network for Facial Expression Recognition

Facial Expression Recognition by Expression-Specific Representation Swapping