Integrated Registration and Occlusion Handling Based on Deep Learning for Augmented Reality Assisted Assembly Instruction
Wang Li,Junfeng Wang,Maoding Liu,Shiwen Zhao,Xintao Ding
DOI: https://doi.org/10.1109/tii.2022.3189428
IF: 12.3
2022-01-01
IEEE Transactions on Industrial Informatics
Abstract:Augmented reality (AR) can convert complex work instructions into virtualreality fusion contents for assembly guidance. In the past, AR registration and occlusion were usually implemented separately, with low robustness and poor timeliness. This article proposes a novel deep learning scheme, named AR-CenterNet, to integrate AR registration and occlusion handling. The proposed method mainly includes two stages, i.e., the neural network prediction stage and the AR processing stage. In the first stage, AR-CenterNet is designed for keypoint detection and depth map prediction. In the second stage, the pose matrix of the physical camera is solved with the predicted keypoints and the depth map of the virtual scene is compared with the predicted depth map for occlusion handling. The experiments demonstrate that our method is robust against different conditions for assisted assembly. This article can provide a new solution method for AR virtualreality fusion based on monocular images.
automation & control systems,computer science, interdisciplinary applications,engineering, industrial