Abstract:Automatic facial expression recognition is an emerging field. Moreover, the interest has been increased with the transition from laboratory-controlled conditions to in the wild scenarios. Most of the research has been done over nonoccluded faces under the constrained environment, while automatic facial expression is less understood/implemented for partial occlusion in the real world conditions. Apart from that, our research aims to tackle the issues of overfitting (caused by the shortage of adequate training data) and to alleviate the expression-unrelated/intraclass/nonlinear facial variations, such as head pose estimation, eye gaze estimation, intensity and microexpressions. In our research, we control the magnitude of each Action Unit (AU) and combine several of the Action Unit combinations to leverage learning from the generative and discriminative representations for automatic FER. We have also addressed the problem of diversification of expressions from lab controlled to real-world scenarios from our cross-database study and proposed a model for enhancement of the discriminative power of deep features while increasing the interclass scatters, by preserving the locality closeness. Furthermore, facial expression consists of an expressive component as well as neutral component, so we proposed a generative model which is capable of generating neutral expression from an input image using cGAN. The expressive component is filtered and passed to the intermediate layers and the process is called De-expression Residue Learning. The residue in the intermediate/middle layers is very important for learning through expressive components. Finally, we validate the effectiveness of our method (DLP-DeRL) through qualitative and quantitative experimental results using four databases. Our method is more accurate and robust, and outperforms all the existing methods (hand crafted features and deep learning) while dealing the images in the wild.

Enhanced Dual-Level Representations for Facial Expression Recognition

DR-FER: Discriminative and Robust Representation Learning for Facial Expression Recognition

Efficient Facial Expression Recognition with Representation Reinforcement Network and Transfer Self-Training for Human–Machine Interaction

Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing for Facial Expression Recognition

Facial expression recognition through multi-level features extraction and fusion

Combining 2D Gabor and Local Binary Pattern for Facial Expression Recognition Using Extreme Learning Machine

Multi-level Spatial and Semantic Enhancement Network for Expression Recognition

Facial Expression Recognition of Nonlinear Facial Variations Using Deep Locality De-Expression Residue Learning in the Wild

Facial Expression Recognition by Expression-Specific Representation Swapping

Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition

Facial Expression Recognition with Mid-level Representation Enhancement and Graph Embedded Uncertainty Suppressing.

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Facial Expression Recognition Based on Dual-Feature Fusion and Improved Random Forest Classifier

Adaptively Learning Facial Expression Representation via C-F Labels and Distillation

Real-Time Facial Expression Recognition Based on Attention Mechanism

Facial Expression Recognition with Contrastive Learning and Uncertainty-Guided Relabeling

A Dual-Direction Attention Mixed Feature Network for Facial Expression Recognition

A Fine-Grained Facial Expression Database for End-to-End Multi-Pose Facial Expression Recognition

Facial Expression Recognition Methods in the Wild Based on Fusion Feature of Attention Mechanism and LBP

Facial expression recognition based on deep convolution long short-term memory networks of double-channel weighted mixture

Facial Expression Recognition Considering Individual Differences in Facial Structure and Texture.