Abstract:The proposed research introduces an innovative Virtual Reality (VR) and Large Language Model (LLM) architecture to enhance the learning process across diverse educational contexts, ranging from school to industrial settings. everaging the capabilities of LLMs and Retrieval-Augmented Generation (RAG), the architecture centers around an immersive VR application. This application empowers students of all backgrounds to interactively engage with their environment by posing questions and receiving informative responses in text format and with visual hints in VR, thereby fostering a dynamic learning experience. LLMs with RAG act as the backbones of this architecture, facilitating the integration of private or domain-specific data into the learning process. By seamlessly connecting various data sources through data connectors, RAG overcomes the challenge of disparate and siloed information repositories, including APIs, PDFs, SQL databases, and more. The data indexes provided by RAG solutions further streamline this process by structuring the ingested data into formats optimized for consumption by LLMs. An empirical study was conducted to evaluate the effectiveness of this VR and LLM architecture. Twenty participants, divided into Experimental and Control groups, were selected to assess the impact on their learning process. The Experimental group utilized the immersive VR application, which allowed interactive engagement with the educational environment, while the Control group followed traditional learning methods. The study revealed significant improvements in learning outcomes for the Experimental group, demonstrating the potential of integrating VR and LLMs in enhancing comprehension and engagement in learning contexts. This study presents an innovative approach that capitalizes on the synergy between LLMs and immersive VR technology, opening avenues for a transformative learning experience that transcends traditional boundaries and empowers learners across a spectrum of educational landscapes.

Immersed in my Ideas: Using Virtual Reality and Multimodal Interactions to Visualize Users' Ideas and Thoughts

Assessing Creativity and User Experience in Immersive Virtual Reality with Cultural Heritage Learning

Virtual Reality and Language Models, a New Frontier in Learning

VR-GPT: Visual Language Model for Intelligent Virtual Reality Applications

DesignMinds: Enhancing Video-Based Design Ideation with Vision-Language Model and Context-Injected Large Language Model

VRFromX: from Scanned Reality to Interactive Virtual Experience with Human-in-the-Loop.

A Study on the Application of Virtual Reality Technology in Engish Language Education Relying on Immersive Learning Environments and Interactive Experience Design

Examining Creativity Through a Virtual Reality Support System

The use of Virtual Reality in Enhancing Interdisciplinary Research and Education

Immersive virtual reality (VR) for digital media making: transmediation is key

Immersive Virtual Reality Implementations in Developmental Psychology

Real-time multimodal interaction in virtual reality - a case study with a large virtual interface

Immersive Virtual Reality: A Novel Approach to Second Language Vocabulary Acquisition in K-12 Education

Virtual Empathy: Usability and Immersivity of a VR System for Enhancing Social Cohesion through Cultural Heritage

Exploring Virtual Reality in the Higher Education Classroom: Using VR to Build Knowledge and Understanding

Immersive virtual reality for science learning: Design, implementation, and evaluation

Evaluating User Engagement and Preference in Virtual Reality and Augmented Virtuality for Interactive Storytelling

Exploration of Visual Communication Design Methods Combining Virtual Reality Technology

”Paint that object yellow”: Multimodal Interaction to Enhance Creativity During Design Tasks in VR

An evaluation of intelligent and immersive digital applications in eliciting cognitive states in humans through the utilization of Emotiv Insight

Development of an Interactive Live Streaming System for Language Learning