The Rise of the AI Scientist: Unleashing the Potential of Chat-GPT Powered Avatars in Virtual Reality Digital-twin Laboratories

Stephen Hilton,Mae Taylor,Zaid Muwaffak,Matthew Penny,Blanka Szulc,Andy Merritt,Steven Brown

DOI: https://doi.org/10.26434/chemrxiv-2023-t4vg7

2023-11-28

Abstract:Digital twin laboratories, accessible via the use of low-cost and portable virtual reality (VR) headsets, have emerged as an immensely powerful tool for chemical education and research collaboration. Having an immersive environment identical to that of a laboratory can provide scientists with a unique platform in which to plan future experiments, conduct laboratory tours, and train on specialist equipment. However, what digital twin laboratories currently lack is on-hand support of co-workers to assist with tasks such as locating chemicals, aiding with machine set-up, and issuing reminders regarding local laboratory health and safety rules Here we show how this key gap can be overcome with the use of knowledge-loaded Chat-GPT avatars in VR. We trained three different chat avatars to perform specialist functions crucial to working in a laboratory and obtained accurate and useful responses in up to 95% of cases, using a range of evaluation metrics including Human Evaluation, Set-Based F1 Scoring, and BERTScore. Our findings demonstrate the vast potential of this technology in harnessing the capabilities of AI assistants for scientists and enhancing immersive digital twin environments within VR settings.

Chemistry

What problem does this paper attempt to address?

This paper discusses how to use Chat-GPT driven virtual reality (VR) avatars to address the limitations of existing digital twin laboratories. These laboratories provide a powerful platform for chemical education and research collaboration through low-cost VR headsets, but lack the support of onsite colleagues, such as assistance in finding chemicals, setting up machines, and reminding of safety rules. The paper proposes training Chat-GPT avatars with domain knowledge to deliver accurate and useful information in 95% of cases, to perform professional tasks in the laboratory. These avatars can serve as sources of information in the virtual reality environment, enhancing immersive experiences and improving the effectiveness of education and collaboration. Three avatars were created for inventory navigation, health and safety guidance, and locating laboratory consumables and glassware. Through a series of tests and evaluations, the study demonstrates the significant potential of this technology in leveraging AI assistants to enhance scientist capabilities and enhance the VR digital twin environment.