Multimodal interaction in virtual reality supported education: a systematic review

Hanlin Hu,Geping Liu,Tao Xie
DOI: https://doi.org/10.1080/10494820.2024.2342993
2024-01-01
Interactive Learning Environments
Abstract:This study reviews empirical studies on applying Multimodal Interactions in Virtual Reality (MIVR) environments in education. We selected 25 papers and reviewed them based on (1) educational research domains, (2) MIVR modalities, (3) MIVR devices/tools, and (4) the methodology and major findings. The results suggest that MIVR focused on language and mathematical science. The common modalities were Visual + Haptic and Visual + Auditory. The mature haptic feedback devices and Virtual Reality (VR) app have advantages in devices. In methodology and key findings, the participants were primarily from middle school and university; the duration of the studies commonly less than one day, and the number of participants were mostly less than 50; the research design more used quasi-experimental; the research methods were quantitative and mixed; the common data collection approaches were tests and questionnaires; scholars evaluated learners' knowledge, skills development, learning experience, learning attitude, and opinion about the MIVR system. The advantages of MIVR in education include natural and real learning experiences, flexibility and efficiency in obtaining information etc., disadvantages were availability issues, cognitive load and personalized insufficiency.
What problem does this paper attempt to address?