Abstract:Purpose Aiming at presenting an interaction-free assembly assistance tool, the purpose of this paper is to propose a synchronous augmented reality (AR) assembly assistance and monitoring system. The system monitors operator’s hands activity and process completeness to recognize the assembly state, then display the AR contents contextually. Design/methodology/approach An assembly behavior recognition method is proposed based on gesture recognition. An assembly completeness inspection method is proposed based on SURF feature matching. Assembly state and AR display state are solved by a novel sequential hybrid AR display control strategy. A synchronous multi-channel AR view output strategy is proposed based on QR matrix decomposition. Findings A prototype system has been developed, and case study is performed on an industrial product. Experiments are performed to verify the feasibility, efficiency and recognition accuracy of the proposed methods. Research limitations/implications The proposed system assists users to perform assembly tasks with automatic visual guidance and vision monitoring, avoiding distractions caused by redundant human–computer interactions. Practical implications All methods are integrated to work on only one head-worn device, making the proposed system portable and cheaper. The vision processing pipelines and the view output channels are reconfigurable for customization. Originality/value This paper proposes an interaction-free AR assembly assistance and monitoring system. Assembly behavior recognition and assembly completeness inspection methods are integrated to monitor the assembly state. A sequential hybrid AR display control strategy is proposed to contextually update the AR contents. A synchronous multi-channel AR view output strategy is proposed to fulfill different visualization needs.

Seeing is Believing: AR-assisted Blind Area Assembly to Support Hand–eye Coordination

First- and Third-person Perspectives in the AR-assisted Blind Area Assembly

Integrate Augmented Reality and Force Sensing Devices to Assist Blind Area Assembly

An AR-Enabled See-Through System for Vision Blind Areas.

Visual Guidance Method for Artificial Assembly in Visual Blind Areas Based on Augmented Reality

A Blind Area Information Perception and AR Assembly Guidance Method Based on RGBD Data for Dynamic Environments and User Study

Bare-hand Gesture Occlusion-Aware Interactive Augmented Reality Assembly

Learning-Based Hand Motion Capture and Understanding in Assembly Process

A natural bare-hand interface-enabled interactive AR assembly guidance

Augmented Reality Based Product Invisible Area Assembly Assistance

Product Blind Area Assembly Method Based on Augmented Reality and Machine Vision

Multi-modal augmented-reality assembly guidance based on bare-hand interface

Occlusion Handling in Augmented Reality System for Human-Assisted Assembly Task

AR-Assisted Guidance for Assembly and Maintenance of Avionics Equipment

Synchronous AR Assembly Assistance and Monitoring System Based on Ego-Centric Vision

Efficient evaluation of misalignment between real and virtual objects for HMD-Based AR assembly assistance system

A Natural Bare-Hand Interaction Method With Augmented Reality for Constraint-Based Virtual Assembly

Integrating Knowledge and Augmented Reality: A Novel Framework for Intelligent Assembly Assistance

ARCoA: Using the AR-Assisted Cooperative Assembly System to Visualize Key Information about the Occluded Partner.

A Multi-Modal Context-Aware Sequence Stage Validation for Human-Centric AR Assembly

The Effects of Complex Assembly Task Type and Assembly Experience on Users’ Demands for Augmented Reality Instructions