A Framework for Emotion Recognition from Human Computer Interaction in Natural Setting

Layale Constantine,Gilbert Badaro,Hazem M. Hajj,W. El-Hajj,Mohamed BenSaleh

Abstract:Since human's emotions play a central role in everyday decisions and well-being, developing systems for recognizing and managing human's emotions captured significant research interest in the last decade. However, there is limited research on studying emotion recognition from human-computer interaction (HCI) in natural settings. This work aims at providing a comprehensive study of emotion recognition from HCI, while addressing several remaining challenges in the context of HCI systems. The first challenge incudes the development of HCI emotion recognition models in natural settings instead of lab-controlled settings. The second challenge is to provide a comprehensive collection of potential humans’ interactions with their computers. The third challenge is to provide a meaningful mapping from digital interactions to human related activities, where the mapped activities can then be used as a feature set for accurate emotion recognition models. Hence, the objective of this work is to develop a framework to address these challenges. A robust ground-truth system is defined for the natural capture of a person’s emotion in the context of computer usage while having unobtrusive and seamless data collection. A ground-truth model is designed for emotion recognition by combining facial expressions analysis and selfassessment. New rules are then defined for capturing the digital activity, and then mapping it to human activity that reflects the person’s context and behavior. Finally, the inferred features are used to derive personalized machine learning models for emotion recognition from digital activity. This work also includes a study from real life experiments, where participants were conducting their activity in their natural settings. The inferred features were annotated using the proposed class labels extraction strategy. Finally, a Bayesian Network was used for the emotion recognition model. Results show evidence that it is indeed feasible to sense the user’s emotions through implicit monitoring of everyday computer interactions.