Abstract:Learning surgical skills require critical visual-spatial motor skills. Current learning methods employ costly and limited in-person teaching in addition to supplementation by videos, textbooks, and cadaveric labs. Increasingly limited healthcare resources and in-person training has led to growing concerns for skills acquisition of trainees. Recent Mixed Reality (MR) devices offer an attractive solution to these resource barriers by providing three-dimensional holographic representations of reality that mimic in-person experiences in a portable, individualized, and cost-effective form. We developed and evaluated two holographic MR models to explore the feasibility of visual-spatial motor skill acquisition from a technical development, learning, and usability perspective. In our first, a pair of holographic hands were created and projected in front of the trainee, and participants were evaluated on their ability to learn complex hand motions in comparison to traditional methods of video and apprenticeship-based learning. The second model displayed a 3D holographic model of the middle and inner ear with labeled anatomical structures which users could explore and user experience feedback was obtained. Our studies demonstrated that scores between MR and apprenticeship learning were comparable. All felt MR was an effective learning tool and most noted that the MR models were better than existing didactic methods of learning. Identified advantages of MR included the ability to provide true 3D spatial representation, improved visualization of smaller structures in detail by upscaling the models, and improved interactivity. Our results demonstrate that holographic learning is able to mimic in-person learning for visual-spatial motor skills and could be a new effective form of self-directed apprenticeship learning.

Opportunities and Limitations of Mixed Reality Holograms in Industrial Robotics

Mixed Reality as Communication Medium for Human-Robot Collaboration

Closed-Loop Robotic Arm Manipulation Based on Mixed Reality

A Mixed Reality Interface for a Digital Twin Based Crane

Investigating Mixed Reality for Communication Between Humans and Mobile Manipulators

Digital Twin for Human–Robot Interactions by Means of Industry 4.0 Enabling Technologies

Immersive Assistance System for Intuitive Robot Programming using Mixed-Reality and Digital Twin

Application of mixed reality in robot manipulator programming

Toward Programming a Collaborative Robot by Interacting with Its Digital Twin in a Mixed Reality Environment

Mixed Reality Applications in Industry: Challenges and Research Areas

Using Mixed Reality for Control and Monitoring of Robot Model Based on Robot Operating System 2

Mobile Mixed-Reality Interfaces That Enhance Human–Robot Interaction in Shared Spaces

A 3D Mixed Reality Interface for Human-Robot Teaming

Visualizing Robot Intent for Object Handovers with Augmented Reality

Mixed Reality: The Interface of the Future

An augmented reality software suite enabling seamless human robot interaction

Building Your Future Holographic Mentor: Can We Use Mixed Reality Holograms for Visual Spatial Motor Skills Acquisition in Surgical Education?

The case study of new approach to robot programming and layout design by supporting virtual and augmented reality

A Real-Time Distance Measurement System for a Digital Twin Using Mixed Reality Goggles

Extended Reality for Enhanced Human-Robot Collaboration: a Human-in-the-Loop Approach

Augmented Reality for Human–Robot Collaboration and Cooperation in Industrial Applications: A Systematic Literature Review