Abstract:The application of Augmented Reality (AR) can effectively reduce the cognitive load of operators by presenting complex assembly work instruction with virtual augmented contents. If the virtual augmented contents superimposed on physical product without handling occlusion, the distortion may appear in AR guidance video. This paper proposes a monocular image based real-time occlusion handling method for virtual part model and physical product. In AR assistance assembly process, the virtual part model can be occluded by assembly scene or assembly objects. First, the monocular SLAM is used to reconstruct the assembly scene, and the reconstructed sparse 3D points are converted into depth points in the depth map. We use the color information of the assembly scene to control the propagation of parse depth points to the remaining pixels. With GPU acceleration calculation, a densification depth map of the assembly scene can be obtained. Then, the 3D models of the assembly objects are rendered as depth map according to the registration coordinates in AR assistance assembly system. The depth map of the assembly scene and assembly objects are synthesized as the final depth map for AR occlusion handling. Finally, the depth relationship between real assembly scenes, assembly objects, and virtual augmented contents are compared to determine the display pixels for AR effect. The results show that the proposed occlusion handling method shows high accuracy and fast speed in comparison with conventional methods, which demonstrates that our method can effectively solve the problem of occlusion handing for AR assistance assembly systems.

Robust Planar Surface Extraction from Noisy and Semi-Dense 3D Point Cloud for Augmented Reality

A New Approach of Point Cloud Processing and Scene Segmentation for Guiding the Visually Impaired

A Submap Joining Algorithm for 3D Reconstruction Using an RGB-D Camera Based on Point and Plane Features

Two-Stage Multi-Camera Constrain Mapping Pipeline for Large-Scale 3D Reconstruction

Outdoor Scene 3D Reconstruction from Multiple Point Cloud

3D Reconstruction Approach for Outdoor Scene Based on Multiple Point Cloud Fusion

Real-Time Dense Monocular SLAM for Augmented Reality

Geometrically consistent plane extraction for dense indoor 3D maps segmentation

Robust Keyframe-Based Monocular SLAM for Augmented Reality

Mobile3DRecon: Real-time Monocular 3D Reconstruction on a Mobile Phone

Scene-Planning System Based on Augmented Reality Technique

Noise Model Based Line Segmentation for Plane Extraction in Sparse 3D LiDAR Data

Real-time occlusion handling for augmented reality assistance assembly systems with monocular images

Robust Extraction of 3D Line Segment Features from Unorganized Building Point Clouds

Resolving Occlusion Between Virtual And Real Scenes For Augmented Reality Applications

Real-time dense 3D reconstruction and camera tracking via embedded planes representation

Learning to Match 2D Images and 3D LiDAR Point Clouds for Outdoor Augmented Reality

Image-based 3D scene reconstruction and exploration in augmented reality

Robust 3D Reconstruction of Building Surfaces from Point Clouds Based on Structural and Closed Constraints

Registration Using Multiplanar Structures for Augmented Reality Systems.

Advanced Mapping Using Planar Features Segmented from 3D Point Clouds