Exploring Emotions in Multi-componential Space using Interactive VR Games

Rukshani Somarathna,Gelareh Mohammadi
2024-04-04
Abstract:Emotion understanding is a complex process that involves multiple components. The ability to recognise emotions not only leads to new context awareness methods but also enhances system interaction's effectiveness by perceiving and expressing emotions. Despite the attention to discrete and dimensional models, neuroscientific evidence supports those emotions as being complex and multi-faceted. One framework that resonated well with such findings is the Component Process Model (CPM), a theory that considers the complexity of emotions with five interconnected components: appraisal, expression, motivation, physiology and feeling. However, the relationship between CPM and discrete emotions has not yet been fully explored. Therefore, to better understand emotions underlying processes, we operationalised a data-driven approach using interactive Virtual Reality (VR) games and collected multimodal measures (self-reports, physiological and facial signals) from 39 participants. We used Machine Learning (ML) methods to identify the unique contributions of each component to emotion differentiation. Our results showed the role of different components in emotion differentiation, with the model including all components demonstrating the most significant contribution. Moreover, we found that at least five dimensions are needed to represent the variation of emotions in our dataset. These findings also have implications for using VR environments in emotion research and highlight the role of physiological signals in emotion recognition within such environments.
Human-Computer Interaction,Artificial Intelligence,Machine Learning
What problem does this paper attempt to address?
This paper aims to explore the complex process of emotion understanding, particularly by using interactive virtual reality (VR) games to study the multi-component space of emotions. Currently, the understanding of emotions is mainly divided into discrete models and dimensional models, but neuroscientific evidence suggests that emotions are complex and multifaceted. The paper proposes a data-driven approach based on the Component Process Model (CPM), which includes five interconnected components: evaluation, motivation, expression, physiology, and sensation. Although the relationship between CPM and discrete emotions has not been fully explored, the paper identifies the unique contributions of each component to emotional differentiation by collecting multimodal data (self-report, physiological, and facial signals) from 39 participants using machine learning (ML) methods. The research findings show that all components make significant contributions to emotional differentiation, and at least five dimensions are needed to represent the variation of emotions in the dataset. Furthermore, the paper emphasizes the importance of using physiological signals for emotion recognition in VR environments. Through emotion induction in VR games, the study reveals the roles of different components in emotional differences and highlights the need for more actively engaging research using enhanced data-driven methods to better understand emotions. The main contributions of the paper include: exploring the relationship between CPM and discrete emotions in interactive VR environments, providing new insights into the relationship between discrete emotions and CPM, and investigating the roles of each CPM component in capturing discrete emotions. These findings have potential applications in improving situational awareness, system interaction, and system adaptability, particularly in the fields of healthcare, VR environments, education, games, and interfaces.