Abstract:The interest in the facial expression recognition (FER) is increasing day by day due to its practical and potential applications, such as human physiological interaction diagnosis and mental diseases detection. This area has received much attention from the research community in recent years and achieved remarkable results; however, a significant improvement is required in spatial problems. This research work presents a novel framework and proposes an effective and robust solution for FER under an unconstrained environment. Face detection is performed using the supervision of facial attributes. Faceness-Net is used for deep facial part responses for the detection of faces under severe unconstrained variations. In order to improve the generalization problems and avoid insufficient data regime, Deep Convolutional Graphical Adversarial Network (DC-GAN) is utilized. Due to the challenging environmental factors faced in the wild, a large number of noises disrupt feature extraction, thus making it hard to capture ground truth. We leverage different multimodal sensors with a camera that aids in data acquisition, by extracting the features more accurately and improve the overall performance of FER. These intelligent sensors are used to tackle the significant challenges like illumination variance, subject dependence, and head pose. Dual-enhanced capsule network is used which is able to handle the spatial problem. The traditional capsule networks are unable to sufficiently extract the features, as the distance varies greatly between facial features. Therefore, the proposed network is capable of spatial transformation due to action unit aware mechanism and thus forward most desiring features for dynamic routing between capsules. Squashing function is used for the classification function. We have elaborated the effectiveness of our method by validating the results on four popular and versatile databases that outperform all state-of-the-art methods.

Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

Deep learning based efficient emotion recognition technique for facial images

Facial Expression Recognition Using Deep Neural Network

Facial Expression Recognition: a Comparison with Different Classical and Deep Learning Methods

Image-based facial emotion recognition using convolutional neural network on emognition dataset

Improved facial emotion recognition model based on a novel deep convolutional structure

Facial emotion recognition based on deep transfer learning approach

Improvement of emotion recognition from facial images using deep learning and early stopping cross validation

Spatio-Temporal Facial Expression Recognition Using Convolutional Neural Networks and Conditional Random Fields

Facial expression recognition using adapted residual based deep neural network

DTL-I-ResNet18: facial emotion recognition based on deep transfer learning and improved ResNet18

Methods for Facial Expression Recognition with Applications in Challenging Situations

Automatic Facial Expression Recognition Based on a Deep Convolutional-Neural-network Structure

The application of deep learning in computer vision: Facial emotion recognition based on convolutional neural network

A novel modular deep fully convolutional network for efficient low resolution facial expression recognition

Uncertain and Biased Facial Expression Recognition Based on Depthwise Separable Convolutional Neural Network with Embedded Attention Mechanism

Facial Expression Recognition Using Enhanced Deep 3D Convolutional Neural Networks

Extended Deep Neural Network for Facial Emotion Recognition

Empirical Investigation of Multimodal Sensors in Novel Deep Facial Expression Recognition In-the-Wild

Facial Emotion Recognition in Verbal Communication Based on Deep Learning

Deep Facial Expression Recognition Using Transfer Learning and Fine-Tuning Techniques