Abstract:The automatic facial expression tracking method has become an emergent topic during the last few decades. It is a challenging problem that impacts many fields such as virtual reality, security surveillance, driver safety, homeland security, human-computer interaction, medical applications. A remarkable cost-efficiency can be achieved by considering some areas of a face. These areas are termed Active Regions of Interest (AROIs). This work proposes a facial expression recognition framework that investigates five types of facial expressions, namely neutral, happiness, fear, surprise, and disgust. Firstly, a pose estimation method is incorporated and to go along with an approach to rotate the face to achieve a normalized pose. Secondly, the whole face-image is segmented into four classes and eight regions. Thirdly, only four AROIs are identified from the segmented regions. The four AROIs are the nose-tip, right eye, left eye, and lips respectively. Fourthly, an info-image-data-mask database is maintained for classification and it is used to store records of images. This database is the mixture of all the images that are gained after introducing a ten-fold cross-validation technique using the Convolutional Neural Network. Correlations of variances and standard deviations are computed based on identified images. To minimize the required processing time in both training and testing the data set, a parallelism technique is introduced, in which each region of the AROIs is classified individually and all of them run in parallel. Fifthly, a decision-tree-level synthesis-based framework is proposed to coordinate the results of parallel classification, which helps to improve the recognition accuracy. Finally, experimentation on both independent and synthesis databases is voted for calculating the performance of the proposed technique. By incorporating the proposed synthesis method, we gain 94.499%, 95.439%, and 98.26% accuracy with the CK+ image sets and 92.463%, 93.318%, and 94.423% with the JAFFE image sets. The overall accuracy is 95.27% in recognition. We gain 2.8% higher accuracy by introducing a decision-level synthesis method. Moreover, with the incorporation of parallelism, processing time speeds up three times faster. This accuracy proves the robustness of the proposed scheme.

Feature fusion for human compound emotion recognition: a fusion of facial expression texture and action unit data

Emotion Recognition in Videos via Fusing Multimodal Features.

Facial Expression Recognition Using Visible, IR, and MSX Images by Early and Late Fusion of Deep Learning Models

Compound facial expressions recognition approach using DCGAN and CNN

Highly Effective End-to-End Single-to-Multichannel Feature Fusion and Ensemble Classification to Decode Emotional Secretes from Small-Scale Spontaneous Facial Micro-Expressions

A Robust Facial Expression Recognition Algorithm Based on Multi-Rate Feature Fusion Scheme

Fusion in Context: A Multimodal Approach to Affective State Recognition

Human Emotion Recognition Based on Spatio-Temporal Facial Features Using HOG-HOF and VGG-LSTM

3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network

Facial Expression Recognition Through Cross-Modality Attention Fusion

Automated facial expression recognition using exemplar hybrid deep feature generation technique

Deep-Emotion: Facial Expression Recognition Using Attentional Convolutional Network

Multimodal Gated Information Fusion for Emotion Recognition from EEG Signals and Facial Behaviors

Facial Expression Recognition Using Hybrid Features of Pixel and Geometry

Facial Expression Recognition Based on Dual-Feature Fusion and Improved Random Forest Classifier

Objective Classes for Micro-Facial Expression Recognition

A deep-learning-based facial expression recognition method using textural features

Facial expression recognition based on active region of interest using deep learning and parallelism

Hierarchical fusion of visual and physiological signals for emotion recognition

A Robust Lightweight Compound Emotion Recognition Approach Using Depthwise Separable CNN

Multimodal Emotion Recognition Using Different Fusion Techniques