Current State of User Experience Evaluation in Virtual Reality: A Systematic Review from an Ergonomic Perspective
Yong Min Kim,Ilsun Rhiu,Minshik Rhie,Hye Seon Choi,Myung Hwan Yun
DOI: https://doi.org/10.1177/1071181319631080
2019-11-01
Proceedings of the Human Factors and Ergonomics Society Annual Meeting
Abstract:Virtual reality (VR) is receiving enough attention to be regarded as a revival era and technologies related to the implementation of VR systems continue to evolve. VR systems are applied not only in entertainment but also in various fields such as medicine, rehabilitation, education, engineering, and military (Aïm, Lonjon, Hannouche, & Nizard, 2016; Howard, 2017; Lele, 2013). In particular, low-cost and immersive VR systems are commercialized to the general public, accelerating the revival of VR (Wang & Lindeman, 2015). In VR system, the research from the viewpoint of human–computer interaction and user experience (UX) is required to provide a high sense of immersion to the user. Therefore, the purpose of this study is to provide a structural methodology for classifying current VR researches and to review UX evaluation of VR systems systematically to identify research trends and to clarify future research directions. This study followed systematic review protocol of (PRISMA) (Liberati et al., 2009). To cover a broad spectrum of perspectives of engineering and medical fields, six web databases were selected: Scopus, Web of Science, Science direct, IEEE Xplore, EBSCO, and ProQuest. The main search keywords were virtual reality and user experience. These two words can be used in acronyms or other words. As a result, four and three words were chosen for virtual reality and user experience, respectively (‘virtual reality’, ‘virtual environment’, ‘VR,’ and ‘VE’ were chosen as keywords for virtual reality, ‘user experience’, ‘UX,’ and ‘human experience’). In addition, the journal articles in English were searched only. After the screening process was completed, final articles were selected based on the full-text. In this process, there were two essential selection conditions. The selected articles should use VR system and evaluate UX component. No restrictions other than these conditions were made. As a result, 78 articles were found to be consistent with the purpose of this study. As a result, there were two main points of discussion about UX studies in a VR system. The first is related to the implementation of equipment and technology including input devices, output devices, feedback forms, platforms, and applications. The other is related to research methods including user characteristics, interactions, and evaluation method. With respect to hand input devices, conventional input devices such as keyboards and game pads were used in many cases compared to trackable devices. However, as implementation techniques for natural interaction such as gesture recognition or real-time tracking of the body parts have been extensively developed, UX research needs to be conducted on VR systems that apply these techniques. In relation to feedback, stimuli other than visual stimuli were not frequently provided. Since providing multiple types of stimuli simultaneously may increase the user’s immersion and sense of reality, it is necessary to intensively study the effect of multi-sensory feedback in the future. In addition, there is a lack of academic research on CAVE and motion platforms. Though CAVE and motion platforms are difficult to set up for experimentation because they are expensive to build and require large space, there is a need to continually expand the UX research on this platform since the public will have more opportunities to access these platforms. Regarding research methods, most of the studies have focused on subjective measurements, quantitative research, laboratory experiments, and episode UX. To comprehensively understand the overall UX, it is necessary to conduct a qualitative study such as observation of subjects experiencing a VR system, think aloud, or deep interview with them, rather than evaluating UX only through a questionnaire. In addition, there was no case in which UX was evaluated in terms of momentary UX. However, there is a limit to evaluating the subjective measurement such as immersion, presence, and motion sickness during usage by directly asking the user, since the VR system provides an immersive environment to the user. Thus, behavioral characteristic or physiological signal of users can be used as one of the evaluation indicators of these measurements. Today, new VR systems are emerging and VR-related technologies are expected to evolve steadily. In this context, the findings can contribute to future research directions and provide insights into conducting UX evaluation in VR system.