Abstract:Emotion estimation from face expression analysis is nowadays a widely-explored computer vision task. In turn, the classification of expressions relies on relevant facial features and their dynamics. Despite the promising accuracy results achieved in controlled and favorable conditions, the processing of faces acquired at a distance, entailing low-quality images, still suffers from a significant performance decrease. In particular, most approaches and related computational models become extremely unstable in the case of the very small amount of useful pixels that is typical in these conditions. Therefore, their behavior should be investigated more carefully. On the other hand, real-time emotion recognition at a distance may play a critical role in smart video surveillance, especially when controlling particular kinds of events, e.g., political meetings, to try to prevent adverse actions. This work compares facial expression recognition at a distance by: 1) a deep learning architecture based on state-of-the-art (SOTA) proposals, which exploits the whole images to autonomously learn the relevant embeddings; 2) a machine learning approach that relies on hand-crafted features, namely the facial landmarks preliminarily extracted using the popular Mediapipe framework. Instead of using either the complete sequence of frames or only the final still image of the expression, like current SOTA approaches, the two proposed methods are designed to use rich temporal information to identify three different stages of emotion. Expressions are time-split accordingly into four phases to better exploit their temporal-dependent dynamics. Experiments were conducted on the popular Extended Cohn-Kanade dataset (CK+). It was chosen for its wide use in related literature, and because it includes videos of facial expressions and not only still images. The results show that the approach relying on machine learning via hand-crafted features is more suitable for classifying the initial phases of the expression and does not decay in terms of accuracy when images are at a distance (only 0.08% of decay). On the contrary, deep learning not only has difficulties classifying the initial phases of the expressions but also suffers from relevant performance decay when considering images at a distance (52.68% accuracy decay).

Sentiment Analysis from Face Expressions Based on Image Processing Using Deep Learning Methods

A real time face emotion classification and recognition using deep learning model

Facial expression recognition using lightweight deep learning modeling

Facial Emotions Recognition Using Deep Learning Technology

Real-Time Human Profiling: Unveiling Age, Gender and Emotions Using Deep Learning

An efficient facial emotion recognition system using novel deep learning neural network-regression activation classifier

Facial expression recognition based on active region of interest using deep learning and parallelism

Facial emotion recognition based on deep transfer learning approach

Real-time Facial Expression Recognition "In The Wild'' by Disentangling 3D Expression from Identity

A novel real time system for facial expression recognition

Deep Learning Based Human Emotion Exposure Detection Using Vocal and Countenance

Automated facial expression recognition using exemplar hybrid deep feature generation technique

Methods for Facial Expression Recognition with Applications in Challenging Situations

Contextual emotion detection in images using deep learning

Emotion recognition at a distance: The robustness of machine learning based on hand-crafted facial features vs deep learning models

A feature boosted deep learning method for automatic facial expression recognition

Human Facial Emotion Recognition using Deep Neural Networks

Human Face Emotion in 3D Using Machine Learning

Facial Sentiment Analysis Using AI Techniques: State-of-the-Art, Taxonomies, and Challenges

A deep-learning-based facial expression recognition method using textural features

Deep learning-based facial emotion recognition for human–computer interaction applications