Abstract:Facial expressions are a universally recognised means of conveying internal emotional states across diverse human cultural and ethnic groups. Recent advances in understanding people’s emotions expressed through verbal and non-verbal communication are particularly noteworthy in the clinical context for the assessment of patients’ health and well-being. Facial expression recognition (FER) plays an important and vital role in health care, providing communication with a patient’s feelings and allowing the assessment and monitoring of mental and physical health conditions. This paper shows that automatic machine learning methods can predict health deterioration accurately and robustly, independent of human subjective assessment. The prior work of this paper is to discover the early signs of deteriorating health that align with the principles of preventive reactions, improving health outcomes and human survival, and promoting overall health and well-being. Therefore, methods are developed to create a facial database mimicking the underlying muscular structure of the face, whose Action Unit motions can then be transferred to human face images, thus displaying animated expressions of interest. Then, building and developing an automatic system based on convolution neural networks (CNN) and long short-term memory (LSTM) to recognise patterns of facial expressions with a focus on patients at risk of deterioration in hospital wards. This research presents state-of-the-art results on generating and modelling synthetic database and automated deterioration prediction through FEs with 99.89% accuracy. The main contributions to knowledge from this paper can be summarized as (1) the generation of visual datasets mimicking real-life samples of facial expressions indicating health deterioration, (2) improvement of the understanding and communication with patients at risk of deterioration through facial expression analysis, and (3) development of a state-of-the-art model to recognize such facial expressions using a ConvLSTM model.

Learning Grimaces by Watching TV

A Deep Learning Perspective on the Origin of Facial Expressions

AI in Pursuit of Happiness, Finding Only Sadness: Multi-Modal Facial Emotion Recognition Challenge

Building and validation of a set of facial expression images to detect emotions: a transcultural study

EmotioNet Challenge: Recognition of facial expressions of emotion in the wild

An Ensemble Approach for Facial Expression Analysis in Video

Interpretable Explainability in Facial Emotion Recognition and Gamification for Data Collection

Automatic Recognition of Facial Displays of Unfelt Emotions

Smile Like You Mean It: Driving Animatronic Robotic Face with Learned Models

Discrimination between the facial gestures of vocalising and non-vocalising lemurs and small apes using deep learning

Portable Facial Expression System Based on EMG Sensors and Machine Learning Models

Learning Face Expression Features from Video Using Spatio-Temporal Feature Extractor and CNN-LSTM

Real-time Facial Expression Recognition "In The Wild'' by Disentangling 3D Expression from Identity

AI-Based Visual Early Warning System

Implementation of an adaptive E-learning platform with facial emotion recognition

Human Face Emotion in 3D Using Machine Learning

Implicit Media Tagging and Affect Prediction from video of spontaneous facial expressions, recorded with depth camera

Associating Facial Expressions and Upper-Body Gestures with Learning Tasks for Enhancing Intelligent Tutoring Systems

Analysis of Micro Facial Expression by Machine and Deep Learning Methods: Haar, CNN, and RNN

Human Facial Emotion Recognition using Deep Neural Networks

Human and machine validation of 14 databases of dynamic facial expressions