Abstract:Facial expression recognition (FER) holds significant practical implications in real-world scenarios such as human–computer interaction, fatigue driving detection, and learning engagement analysis. Nonetheless, acquiring large-scale and high-quality annotated facial expression datasets is profoundly challenging due to the inherent ambiguity of facial images and concerns over privacy. Consequently, this paper introduces a self-supervised facial expression recognition method based on mask image modeling. This method can learn multi-level facial feature representations without expensive labels and achieves commendable facial expression recognition performance through further fine-grained feature selection. Specifically, we propose the multi-level feature selector (MFS). The MFS comprises two pivotal components: the multi-level feature combiner and the feature selector. During the pre-training stage, the multi-level feature combiner is employed to integrate multi-level features, effectively addressing the vision transformer's deficiencies in capturing high-frequency facial semantics. Subsequently, in the fine-tuning stage, the feature selector can automatically differentiate highly discriminative regions, extracting fine-grained features. Subsequently, we use graph convolutional networks to further mine the latent connections among fine-grained features, ultimately deriving an integrated feature with enhanced discriminative capabilities. Through such fine-grained facial feature selection, we can mitigate performance degradation induced by inter-class similarities and intra-class variations. Experimental results on the RAF-DB, AffectNet, and FER + datasets demonstrate that our approach significantly outperforms other self-supervised methods in recognition performance and closely approaches the state-of-the-art methods in supervised learning. The code is available at https://github.com/Greysahy/MFS

AN ADAPTIVE BOOSTING ALGORITHM WITH SEMI-SUPERVISED LEARNING FOR FACIAL EXPRESSION RECOGNITION

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

Multiclass AdaBoost ELM and Its Application in LBP Based Face Recognition

Multi-class AdaBoost ELM

Bridging the Gaps: Utilizing Unlabeled Face Recognition Datasets to Boost Semi-Supervised Facial Expression Recognition

Boosting Facial Expression Recognition by A Semi-Supervised Progressive Teacher

Adaptively Learning Facial Expression Representation via C-F Labels and Distillation

Boosting Semi-Supervised Face Recognition with Raw Faces

Exploring Facial Expression Recognition through Semi-Supervised Pretraining and Temporal Modeling

Multiple source domain adaptation in micro-expression recognition

Real Time Facial Expression Recognition with AdaBoost

Pose-robust personalized facial expression recognition through unsupervised multi-source domain adaptation

Conditional Mutual Infomation Based Boosting for Facial Expression Recognition.

Facial Expression Recognition Based on Gabor Feature and Adaboost

Cross-database facial expression recognition based on hybrid improved unsupervised domain adaptation

Boosting Semi-Supervised Face Recognition With Noise Robustness

A feature boosted deep learning method for automatic facial expression recognition

MaskMatch: Boosting Semi-Supervised Learning Through Mask Autoencoder-Driven Feature Learning

Facial Expression Recognition from a Single Face Image Based on Deep Learning and Broad Learning

Exploring the Boundaries of Semi-Supervised Facial Expression Recognition using In-Distribution, Out-of-Distribution, and Unconstrained Data

Self-supervised facial expression recognition with fine-grained feature selection