Abstract:Automatic recognition of human emotions is not a trivial process. There are many factors affecting emotions internally and externally. Expressing emotions could also be performed in many ways such as text, speech, body gestures or even physiologically by physiological body responses. Emotion detection enables many applications such as adaptive user interfaces, interactive games, and human robot interaction and many more. The availability of advanced technologies such as mobiles, sensors, and data analytics tools led to the ability to collect data from various sources, which enabled researchers to predict human emotions accurately. Most current research uses them in the lab experiments for data collection. In this work, we use direct and real time sensor data to construct a subject-independent (generic) multi-modal emotion prediction model. This research integrates both on-body physiological markers, surrounding sensory data, and emotion measurements to achieve the following goals: (1) Collecting a multi-modal data set including environmental, body responses, and emotions. (2) Creating subject-independent Predictive models of emotional states based on fusing environmental and physiological variables. (3) Assessing ensemble learning methods and comparing their performance for creating a generic subject-independent model for emotion recognition with high accuracy and comparing the results with previous similar research. To achieve that, we conducted a real-world study "in the wild" with physiological and mobile sensors. Collecting the data-set is coming from participants walking around Minia university campus to create accurate predictive models. Various ensemble learning models (Bagging, Boosting, and Stacking) have been used, combining the following base algorithms (K Nearest Neighbor KNN, Decision Tree DT, Random Forest RF, and Support Vector Machine SVM) as base learners and DT as a meta-classifier. The results showed that, the ensemble stacking learner technique gave the best accuracy of 98.2% compared with other variants of ensemble learning methods. On the contrary, bagging and boosting methods gave (96.4%) and (96.6%) accuracy levels respectively.

Cross-Modal Diversity-Based Active Learning for Multi-Modal Emotion Estimation.

Cross-Culture Multimodal Emotion Recognition With Adversarial Learning

Cross-Task Inconsistency Based Active Learning (CTIAL) for Emotion Recognition

Bridging the Emotional Semantic Gap via Multimodal Relevance Estimation

Emotion Dictionary Learning with Modality Attentions for Mixed Emotion Exploration

Multimodal Emotion Distribution Learning.

Learning Fine-Grained Cross Modality Excitement for Speech Emotion Recognition

Multimodal Emotion Recognition by Extracting Common and Modality-Specific Information.

Early Joint Learning of Emotion Information Makes MultiModal Model Understand You Better

Evaluating Ensemble Learning Methods for Multi-Modal Emotion Recognition Using Sensor Data Fusion

Hybrid Multi-Task Learning for End-To-End Multimodal Emotion Recognition

Multi-View Common Space Learning For Emotion Recognition In The Wild

Fine-grained Disentangled Representation Learning for Multimodal Emotion Recognition

Enhancing Emotion Recognition in Incomplete Data: A Novel Cross-Modal Alignment, Reconstruction, and Refinement Framework

Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition

Multi-modal Sentiment and Emotion Joint Analysis with a Deep Attentive Multi-task Learning Model

Correlation-Driven Multi-Modality Graph Decomposition for Cross-Subject Emotion Recognition

Multimodal Knowledge-enhanced Interactive Network with Mixed Contrastive Learning for Emotion Recognition in Conversation

Multi-Classifier Interactive Learning for Ambiguous Speech Emotion Recognition

Cross-modality reinforcement for unaligned sequences sentiment analysis