Abstract:As an auxiliary assembly guidance method, the augmented assembly has been gradually applied in modern manufacturing. However, with current product assembly developing toward customization and diversification, static augmented assembly guidance instructions that mainly depend on position measurement are hard to meet the needs of complex assembly cases. The position of assembly elements such as tools and essential parts may change frequently. This paper proposes a novel approach for detecting and annotating assembly elements using AR mobile devices in an offline environment to solve the problem. An improved deep learning object detection network YOLOv4-Lite is proposed. It achieves further lightweight while improving detection accuracy than YOLOv4-Tiny, running well in offline AR mobile devices with limited computing power. A spatial 3D annotation algorithm is presented to calculate the location of the center of the target object and generate markers for annotation. To verify the feasibility of the proposed approach, an offline detection system for assembly elements named ARODAS (augmented reality offline detection and annotation system) is developed and deployed into HoloLens2. Experimental results show that the proposed approach can complete detection and annotation with reasonable accuracy, assist the operator in detecting key assembly elements, and has a significant advantage in operation speed. To verify the usability and usefulness of the ARODAS, we performed quantitative and qualitative analyses by conducting a user study: searching for several kinds of small parts in a real environment. Compared with the traditional marker-based AR system, the ARODAS system shows a better overall performance in the user test.

Research and Implementation on Key Technologies of Assembly Assistance System based on Augmented Reality

Deep learning-based augmented reality work instruction assistance system for complex manual assembly

A Robot Assisted Assembly System for Small Components in Aircraft Assembly

Development and Application of Parts Assembly Guidance System Based on Augmented Reality

A posture detection method for augmented reality–aided assembly based on YOLO-6D

AR-Assisted Guidance for Assembly and Maintenance of Avionics Equipment

Integrated Registration and Occlusion Handling Based on Deep Learning for Augmented Reality Assisted Assembly Instruction

Digital Assembly Assistance System in Industry 4.0 Era: A Case Study with Projected Augmented Reality

A Novel Method of Multitarget Augmented Reality Assembly Result Inspection for Large Complex Scenes

Augmented Reality for Collaborative Assembly Design in Manufacturing Sector

AR-assisted assembly method based on instance segmentation

A comprehensive review of augmented reality-based instruction in manual assembly, training and repair

A multi-modal augmented reality based virtual assembly system

Augmented Reality Assisted Assembly Training Oriented Dynamic Gesture Recognition and Prediction

A smart assistance system for cable assembly by combining wearable augmented reality with portable visual inspection

SHARIDEAS: a smart collaborative assembly platform based on augmented reality supporting assembly intention recognition

Equipment Assembly Recognition for Augmented Reality Guidance

A comprehensive survey of augmented reality assembly research

Rapid Offline Detection and 3D Annotation of Assembly Elements in the Augmented Assembly

Guided assembly using deep learning-based object recognition and augmented reality instructions

A 3D Tracking and Registration Method Based on Point Cloud and Visual Features for Augmented Reality Aided Assembly System