Abstract:With the rapid development of mixed reality (MR) technology, many compact, lightweight, and powerful devices suitable for remote collaboration, such as MR headsets, hand trackers, and 3D cameras, become readily available, providing hardware and software support for remote collaboration. Consequently, exploring MR technologies for remote collaboration on physical industry tasks is becoming increasingly worthwhile. In many complex production scenarios, such as assembly tasks, significant gains can be achieved by having remote experts assist local workers to manipulate objects in local workspaces. However, it can be challenging for a remote expert to carry out effective spatial reference and action demonstration in a local scene. Sharing 3D stereoscopic scenes can provide depth perception and support remote experts to move and explore a local user's environment freely. Previous studies have demonstrated that gesture-based interaction is natural and intuitive, and interaction based on virtual replicas can provide clear guidance, especially for industrial physical tasks. In this study, we develop an MR remote collaboration system that shares the stereoscopic scene of the local workspace by using real-time 3D video. This system combines gesture cues and virtual replicas in a complementary manner to support the remote expert to create augmented reality (AR) guidance for the local worker naturally and intuitively in the virtual reality immersive space. A formal user study was performed to explore the effects of two different modalities interface in industrial assembly tasks: our novel method of using the combination of virtual replicas and gesture cues in the 3D video (VG3DV), and a method similar to the popular method currently of using gesture cues in the 3D video (G3DV). We found that using the VG3DV can significantly improve the performance and user experience of MR remote collaboration in industrial assembly tasks. Finally, some conclusions and future research directions were given.

Collaborative Mixed Reality Annotations System for Science and History Education Based on UWB Positioning and Low-Cost AR Glasses

A Bare Hand Controlled Ar Map Navigation System

Enhancing Visualisation of Anatomical Presentation and Education Using Marker-based Augmented Reality Technology on Web-based Platform

Multi-User Augmented Reality with Infrastructure-free Collaborative Localization

A Web-based Augmented Reality Plat-form Using Pictorial QR Code for Educational Purposes and Beyond.

A mixed reality platform for assembly assistance based on gaze interaction in industry

Wearable Biosensor Smart Glasses Based on Augmented Reality and Eye Tracking

Multi-View AR Streams for Interactive 3D Remote Teaching

MagLoc-AR: Magnetic-Based Localization for Visual-Free Augmented Reality in Large-Scale Indoor Environments

Exploring Interactive Learning Environments Based on Augmented Reality Technology

A gesture- and head-based multimodal interaction platform for MR remote collaboration

Hand-Controlled User Interfacing for Head-Mounted Augmented Reality Learning Environments

Real-time Collaboration Between Mixed Reality Users in Geo-referenced Virtual Environment

Collaborative MR Workspace with Shared 3D Vision Based on Stereo Video Transmission

Teaching Mode of Augmented Reality College English Listening and Speaking Supported by Wearable Technology

Rationale and Clinical Results of Inhibiting Interleukin-5 for the Treatment of Severe Asthma

35.1: A novel in‐situ interactive 3D floating autostereoscopic display system with aerial imaging plate

A novel AR remote collaborative platform for sharing 2.5D gestures and gaze

Multi-sensor Fusion for Interactive Visual Computing in Mixed Environment.

Collaborative System Design of Mixed Reality Communication for Medical Training

Real-time 3D video-based MR remote collaboration using gesture cues and virtual replicas