Abstract:Automatic recognition of human emotions is not a trivial process. There are many factors affecting emotions internally and externally. Expressing emotions could also be performed in many ways such as text, speech, body gestures or even physiologically by physiological body responses. Emotion detection enables many applications such as adaptive user interfaces, interactive games, and human robot interaction and many more. The availability of advanced technologies such as mobiles, sensors, and data analytics tools led to the ability to collect data from various sources, which enabled researchers to predict human emotions accurately. Most current research uses them in the lab experiments for data collection. In this work, we use direct and real time sensor data to construct a subject-independent (generic) multi-modal emotion prediction model. This research integrates both on-body physiological markers, surrounding sensory data, and emotion measurements to achieve the following goals: (1) Collecting a multi-modal data set including environmental, body responses, and emotions. (2) Creating subject-independent Predictive models of emotional states based on fusing environmental and physiological variables. (3) Assessing ensemble learning methods and comparing their performance for creating a generic subject-independent model for emotion recognition with high accuracy and comparing the results with previous similar research. To achieve that, we conducted a real-world study "in the wild" with physiological and mobile sensors. Collecting the data-set is coming from participants walking around Minia university campus to create accurate predictive models. Various ensemble learning models (Bagging, Boosting, and Stacking) have been used, combining the following base algorithms (K Nearest Neighbor KNN, Decision Tree DT, Random Forest RF, and Support Vector Machine SVM) as base learners and DT as a meta-classifier. The results showed that, the ensemble stacking learner technique gave the best accuracy of 98.2% compared with other variants of ensemble learning methods. On the contrary, bagging and boosting methods gave (96.4%) and (96.6%) accuracy levels respectively.

Bi-Model Engagement Emotion Recognition Based on Facial and Upper-Body Landmarks and Machine Learning Approaches

Human Emotion Recognition Based on Spatio-Temporal Facial Features Using HOG-HOF and VGG-LSTM

Face Emotion Recognition Based on Machine Learning: A Review

Evaluating Ensemble Learning Methods for Multi-Modal Emotion Recognition Using Sensor Data Fusion

Emotion Detection through Body Gesture and Face

Emotion recognition framework using multiple modalities for an effective human–computer interaction

Deploying Machine Learning Techniques for Human Emotion Detection

Emotion Recognition System Based on Facial Expressions Using SVM

A real time face emotion classification and recognition using deep learning model

Real-time Automatic Emotion Recognition from Body Gestures

Multimodal emotion recognition based on the fusion of vision, EEG, ECG, and EMG signals

Multimodal Real-Time patient emotion recognition system using facial expressions and brain EEG signals based on Machine learning and Log-Sync methods

Towards Context-Aware Facial Emotion Reaction Database for Dyadic Interaction Settings

Facial Human Emotion Recognition by Using YOLO Faces Detection Algorithm

Human Facial Emotion Recognition using Deep Neural Networks

Automated Facial Emotion Recognition Using the Pelican Optimization Algorithm with a Deep Convolutional Neural Network

Human facial emotion recognition using improved black hole based extreme learning machine

Using Facial Micro-Expressions in Combination With EEG and Physiological Signals for Emotion Recognition

Novel multimodal emotion detection method using Electroencephalogram and Electrocardiogram signals

Portable Facial Expression System Based on EMG Sensors and Machine Learning Models

Multimodal Emotion Recognition based on Facial Expressions, Speech, and EEG