Guided assembly using deep learning-based object recognition and augmented reality instructions

José Mario De Martino,Prof. Dr Mariana Zaninelo Reis

Abstract:– Efficient assembly is vital to guarantee quality and celerity in manufacturing processes. Usually, the assembly tasks in a non-robotic process are expressed by assembly instructions that provide text and drawing information, both in 2D and 3D, about the form and order of the execution of tasks. However, these instructions are rarely interactive and, depending on their complexity, hard to follow. The main objective of this master’s work is to develop an approach capable of providing the user with interactive step-by-step guidance for assembly tasks. Deep learning techniques will be applied to recognize, in real-time, equipment parts, indicate the assembly order, and the coupling to other parts. The assembly instructions will be provided exploring augmented reality. We plan to implement the solution on a mobile device and assess and compare our solution with more traditional ones, considering assembly speed-up and perceptual feedback from users.

Engineering,Computer Science