Abstract:Research shows that XR experiences activate portions of the brain that facilitate learning. Case studies in industry bear this out, showing enormous benefits of XR when used to train employees, including reduced time to learn and complete a task, increased ability to perform the task correctly the first time, and better knowledge retention than with traditional training methods. The emerging technologies that fall under the umbrella term XR—virtual reality, augmented reality, mixed reality, and 360-degree video—can facilitate or enhance learning through different affordances. The degree of immersion and the ability to exercise agency through interaction also enhances learning. 360-degree video is highly immersive. But it is not inherently interactive. It is theorized that by inserting interactive 3D objects into a 360-degree video environment, both immersion and interactivity can be achieved, with the result of enhanced learning. The solution presented here also has the advantage of being more easily produced and cheaper than virtual reality. The objectives of this preliminary study were to create a simple and relatively inexpensive prototype virtual environment very close to reality and test the technical effectiveness of learning and situational awareness using 360-degree videos together with 3D objects. An iterative human-centered design (HCD) process that integrates airplane pilots and training personnel as end users and participants in the cocreation of an aviation training system; actively involves users; provides a clear understanding of tasks; facilitates good interaction between user and technology; and includes well-designed interactions was used. A few scripts were created in the prototype to improve the user experience (UX) when interacting with the user interface (UI). The prototype app was presented to flight instructors and flight pilots using a Samsung S7 smartphone with a Samsung Gear VR by Oculus headset. The prototype also allows the experience to be viewed on a mobile device without a VR headset. The experience is based on a 360-degree video taken within a Cessna 150, with a view of the flight panel. 3D objects were inserted and were made interactive so the user can control them. A survey that asked for suggestions to improve the next version of this prototype was given to the test subjects. Preliminary findings suggest system user acceptance, improved pilot confidence before the actual flight, decreased errors, and better situational awareness. In conclusion, the combination of 360-degree video and 3D virtual objects can help train pilots better, more safely, in less time, and with more confidence in the use of the aircraft checklist. The technology provides an alternative to costly and time-consuming VR solutions for training those without access to aircraft, and permits learning with the use of their smartphone.

Exploring educational immersive videogames: an empirical study with a 3D multimodal interaction prototype

Exploitation of Multiplayer Interaction and Development of Virtual Puppetry Storytelling Using Gesture Control and Stereoscopic Devices.

Multi-modal natural interaction in game design: a comparative analysis of player experience in a large scale role-playing game

Digital Game for Teaching and Learning: an Analysis of Usability and Experience of Educational Games.

Evaluating the Impact of Highly Immersive Technologies and Natural Interaction on Player Engagement and Flow Experience in Games

Exploitation Of Novel Multiplayer Gesture-Based Interaction And Virtual Puppetry For Digital Storytelling To Develop Children'S Narrative Skills

Mixed Assessment of Virtual Serious Games Applied in Architectural and Urban Design Education

Exploring Virtual Reality in the Higher Education Classroom: Using VR to Build Knowledge and Understanding

Immersive virtual-reality computer-assembly serious game to enhance autonomous learning

An Extended Reality Platform for Cultural Gaming: Enabling Interactive Narratives in Spatial Contexts

Using virtual reality to assess user experience.

Educational Seismology through an Immersive Virtual Reality Game: Design, Development and Pilot Evaluation of User Experience

ImCasting: Nonverbal Behaviour Reinforcement Learning of Virtual Humans through Adaptive Immersive Game

Using Multimodal Interaction in a Virtual Reality Thoracic Diagnostic Scenario

Learning experience design with immersive virtual reality in physics education

Exploring the Perception of the Effect of Three-Dimensional Interaction Feedback Types on Immersive Virtual Reality Education

Development of a Smart Learning Application in Multi-person Virtual Reality Using Biometric Measures of Neuroimaging, Eye Tracking, and Haptic Interactions

Combining immersion and interaction in XR training with 360-degree video and 3D virtual objects

Designing Experiences: A Virtual Reality Video Game to Enhance Immersion

User Experience Evaluation in Shared Interactive Virtual Reality

Immersive virtual reality game for cognitive-empathy education: Implementation and formative evaluation