Biomarkers of Immersion in Virtual Reality Based on Features Extracted from the EEG Signals: A Machine Learning Approach

Hamed Tadayyoni,Michael S. Ramirez Campos,Alvaro Joffre Uribe Quevedo,Bernadette A. Murphy

DOI: https://doi.org/10.3390/brainsci14050470

IF: 3.333

2024-05-08

Brain Sciences

Abstract:Virtual reality (VR) enables the development of virtual training frameworks suitable for various domains, especially when real-world conditions may be hazardous or impossible to replicate because of unique additional resources (e.g., equipment, infrastructure, people, locations). Although VR technology has significantly advanced in recent years, methods for evaluating immersion (i.e., the extent to which the user is engaged with the sensory information from the virtual environment or is invested in the intended task) continue to rely on self-reported questionnaires, which are often administered after using the virtual scenario. Having an objective method to measure immersion is particularly important when using VR for training, education, and applications that promote the development, fine-tuning, or maintenance of skills. The level of immersion may impact performance and the translation of knowledge and skills to the real-world. This is particularly important in tasks where motor skills are combined with complex decision making, such as surgical procedures. Efforts to better measure immersion have included the use of physiological measurements including heart rate and skin response, but so far they do not offer robust metrics that provide the sensitivity to discriminate different states (idle, easy, and hard), which is critical when using VR for training to determine how successful the training is in engaging the user's senses and challenging their cognitive capabilities. In this study, electroencephalography (EEG) data were collected from 14 participants who completed VR jigsaw puzzles with two different levels of task difficulty. Machine learning was able to accurately classify the EEG data collected during three different states, obtaining accuracy rates of 86% and 97% for differentiating easy versus hard difficulty states and baseline vs. VR states. Building on these results may enable the identification of robust biomarkers of immersion in VR, enabling real-time recognition of the level of immersion that can be used to design more effective and translative VR-based training. This method has the potential to adjust aspects of VR related to task difficulty to ensure that participants are immersed in VR.

neurosciences

What problem does this paper attempt to address?

The problem this paper attempts to address is: how to measure immersion in Virtual Reality (VR) through objective methods, particularly in applications such as training, education, and skill development. Current methods for assessing immersion primarily rely on user self-reported questionnaires, which are typically administered after using the virtual environment and lack real-time and objective qualities. Therefore, researchers aim to develop a technology that can measure immersion in real-time and objectively through electrophysiological signals (such as EEG) and machine learning methods.

Specifically, the goals of the paper include:
1. **Developing an objective measurement method based on EEG signals**: By extracting features from EEG signals and using machine learning algorithms to classify different states of immersion.
2. **Identifying biomarkers of immersion**: By analyzing EEG data to find biomarkers that can distinguish between different task difficulties and baseline states.
3. **Designing more effective VR training systems**: Using real-time immersion data to adjust the difficulty of VR tasks, ensuring that users maintain a high level of immersion in the virtual environment, thereby improving training effectiveness.

The research background emphasizes the potential of current VR technology in training and education but also points out its limitations, particularly the lack of effective immersion assessment methods. By addressing these issues, researchers hope to improve the performance of VR systems, making them more effective in practical applications.