Abstract:Smiling has often been incorrectly interpreted as "happy" in the popular facial expression datasets (AffectNet, RAF-DB, FERPlus). Smiling is the most complex human expression, with positive, neutral, and negative smiles. We focused on fine-grained facial expression recognition (FER) and built a new smiling face dataset, named Facial Expression Emotions. This dataset categorizes smiles into six classes of smiles, containing a total of 11,000 images labeled with corresponding fine-grained facial expression classes. We propose Smile Transformer, a network architecture for FER based on the Swin Transformer, to enhance the local perception capability of the model and improve the accuracy of fine-grained face recognition. Moreover, a convolutional block attention module (CBAM) was designed, to focus on important features of the face image and suppress unnecessary regional responses. For better classification results, an image quality evaluation module was used to assign different labels to images with different qualities. Additionally, a dynamic weight loss function was designed, to assign different learning strategies according to the labels during training, focusing on hard yet recognizable samples and discarding unidentifiable samples, to achieve better recognition. Overall, we focused on (a) creating a novel dataset of smiling facial images from online annotated images, and (b) developing a method for improved FER in smiling images. Facial Expression Emotions achieved an accuracy of 88.56% and could serve as a new benchmark dataset for future research on fine-grained FER.

Realistic Smile Expression Recognition Using Biologically Inspired Features

Real-Time facial expression recognition system based on HMM and feature point localization

How Do You Smile? Towards a Comprehensive Smile Analysis System

Methods of Recognizing True and Fake Smiles by Using Au6 and Au12 in A Holistic Way

Real-Time Facial Expression Recognition Based on the Improved Brain Emotional Learning Model

Fine-Grained Facial Expression Recognition in Multiple Smiles

Spontaneous Versus Posed Smile Recognition Via Region-Specific Texture Descriptor and Geometric Facial Dynamics

Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models

Distinguishing Posed and Spontaneous Smiles by Facial Dynamics

Face Recognition Using Early Biologically Inspired Features

Smile detection in the wild based on transfer learning

Smile: Spiking Multi-Modal Interactive Label-Guided Enhancement Network for Emotion Recognition

Casme Ii: an Improved Spontaneous Micro-Expression Database and the Baseline Evaluation

Smile detection in unconstrained scenarios using self-similarity of gradients features

Real Time Facial Expression Recognition with AdaBoost

Facial Expression Recognition with LBP and ORB Features

Deep Convolutional Neural Networks for Smile Recognition

Facial Expression Recognition from a Single Face Image Based on Deep Learning and Broad Learning

Automatic Smile Detection of Infants in Mother-Infant Interaction Via CNN-based Feature Learning

An Efficient Approach to Smile Detection

Region Based Robust Facial Expression Analysis