Abstract:In mixed reality (MR) remote collaborative assembly, remote experts can guide local users to complete the assembly of physical tasks by sharing user cues (eye gazes, gestures, etc.) and spatial visual cues (such as AR annotations, virtual replicas). At present, remote experts need to carry out complex operations to transfer information to local users, but the fusion of virtual and real information makes the display of information in the MR collaborative interaction interface appear messy and redundant, and local users sometimes find it difficult to pay attention to the focus of information transferred by experts. Our research aims to simplify the operation of remote experts in MR remote collaborative assembly and to enhance the expression of visual cues that reflect experts' attention, so as to promote the expression and communication of collaborative intention that user has and improve assembly efficiency. We developed a system (EaVAS) through a method that is based on the assembly semantic association model and the expert operation visual enhancement mechanism that integrates gesture, eye gaze, and spatial visual cues. EaVAS can give experts great freedom of operation in MR remote collaborative assembly, so that experts can strengthen the visual expression of the information they want to convey to local users. EaVAS was tested for the first time in an engine physical assembly task. The experimental results show that the EaVAS has better time performance, cognitive performance, and user experience than that of the traditional MR remote collaborative assembly method (3DGAM). Our research results have certain guiding significance for the research of user cognition in MR remote collaborative assembly, which expands the application of MR technology in collaborative assembly tasks.

A multi-modal augmented reality based virtual assembly system

An Integrated Virtual Design And Virtual Assembly Environment

MIVAS: A Multi-Modal Immersive Virtual Assembly System

Virtual Assembly In A Cave (Tm) Environment

Interactive task planning in virtual assembly.

Research on Component-Based Collaborative Assembly Technology for Virtual Prototype Unit

Heterogeneous CAD Supported Virtual Assembly System for Distributed Collaborative Design

A Natural Bare-Hand Interaction Method With Augmented Reality for Constraint-Based Virtual Assembly

Augmented Reality for Collaborative Assembly Design in Manufacturing Sector

Development and Application of Parts Assembly Guidance System Based on Augmented Reality

Research and Implementation on Key Technologies of Assembly Assistance System based on Augmented Reality

Virtual Manipulation in an Immersive Virtual Environment: Simulation of Virtual Assembly

Critical Techniques of Virtual Assembly for Combine Harvester Chassis Based on Virtual Reality

Research on Immersive Virtual Assembly Technology

A multimodal interface to situated assembly robot systems

A touchable virtual screen interaction system with handheld Kinect camera

Constrained Behavior Manipulation for Interactive Assembly in a Virtual Environment

Research and Implementation of Virtual Assembly System

Scene-Planning System Based on Augmented Reality Technique

A natural bare-hand interface-enabled interactive AR assembly guidance

Can you notice my attention? A novel information vision enhancement method in MR remote collaborative assembly