Abstract:The interest in the facial expression recognition (FER) is increasing day by day due to its practical and potential applications, such as human physiological interaction diagnosis and mental diseases detection. This area has received much attention from the research community in recent years and achieved remarkable results; however, a significant improvement is required in spatial problems. This research work presents a novel framework and proposes an effective and robust solution for FER under an unconstrained environment. Face detection is performed using the supervision of facial attributes. Faceness-Net is used for deep facial part responses for the detection of faces under severe unconstrained variations. In order to improve the generalization problems and avoid insufficient data regime, Deep Convolutional Graphical Adversarial Network (DC-GAN) is utilized. Due to the challenging environmental factors faced in the wild, a large number of noises disrupt feature extraction, thus making it hard to capture ground truth. We leverage different multimodal sensors with a camera that aids in data acquisition, by extracting the features more accurately and improve the overall performance of FER. These intelligent sensors are used to tackle the significant challenges like illumination variance, subject dependence, and head pose. Dual-enhanced capsule network is used which is able to handle the spatial problem. The traditional capsule networks are unable to sufficiently extract the features, as the distance varies greatly between facial features. Therefore, the proposed network is capable of spatial transformation due to action unit aware mechanism and thus forward most desiring features for dynamic routing between capsules. Squashing function is used for the classification function. We have elaborated the effectiveness of our method by validating the results on four popular and versatile databases that outperform all state-of-the-art methods.

Facial Expression Recognition from Occluded Images Using Deep Convolution Neural Network with Vision Transformer

TFE: A Transformer Architecture for Occlusion Aware Facial Expression Recognition.

Collaborative Attention Transformer on Facial Expression Recognition under Partial Occlusion

Facial Expression Recognition With Visual Transformers and Attentional Selective Fusion

A convolution-transformer dual branch network for head-pose and occlusion facial expression recognition

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

Facial Expression Recognition Based on Multi-Scale Convolutional Vision Transformer

Facial Expression Recognition with Grid-Wise Attention and Visual Transformer

Interrelated Fusion CNN with Statistical Grouping among Multipatches for Occluded Facial Expression Recognition

Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism

Facial Expression Recognition Based on Fine-Tuned Channel–Spatial Attention Transformer

Facial Expression Recognition Via Region-Based Convolutional Fusion Network.

Patch Attention Convolutional Vision Transformer for Facial Expression Recognition with Occlusion

Empirical Investigation of Multimodal Sensors in Novel Deep Facial Expression Recognition In-the-Wild

Hybrid Attention-Aware Learning Network for Facial Expression Recognition in the Wild

Facial Expression Recognition Using Deep Neural Network

Uncertain and Biased Facial Expression Recognition Based on Depthwise Separable Convolutional Neural Network with Embedded Attention Mechanism

Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields

Facial expression recognition in facial occlusion scenarios: A path selection multi-network

Latent-OFER: Detect, Mask, and Reconstruct with Latent Vectors for Occluded Facial Expression Recognition