Atae Jafari-Tabrizi,Dieter P. Gruber,Andrej Gams
Abstract:Abstract Currently, the standard method of programming industrial robots is to perform it manually, which is cumbersome and time-consuming. Thus, it can be a burden for the flexibility of inspection systems when a new component with a different design needs to be inspected. Therefore, developing a way to automate the task of generating a robotic trajectory offers a substantial improvement in the field of automated manufacturing and quality inspection. This paper proposes and evaluates a methodology for automatizing the process of scanning a 3D surface for the purpose of quality inspection using only visual feedback. The paper is divided into three sub-tasks in the same general setting: (1) autonomously finding the optimal distance of the camera on the robot’s end-effector from the surface, (2) autonomously generating a trajectory to scan an unknown surface, and (3) autonomous localization and scan of a surface with a known shape, but with an unknown position. The novelty of this work lies in the application that only uses visual feedback, through the image focus measure, for determination and optimization of the motion. This reduces the complexity and the cost of such a setup. The methods developed have been tested in simulation and in real-world experiments and it was possible to obtain a precision in the optimal pose of the robot under 1 mm in translational, and 0.1 $$^\circ $$ ∘ in angular directions. It took less than 50 iterations to generate a trajectory for scanning an unknown free-form surface. Finally, with less than 30 iterations during the experiments it was possible to localize the position of the surface. Overall, the results of the proposed methodologies show that they can bring substantial improvement to the task of automatic motion generation for visual quality inspection.
What problem does this paper attempt to address?
The paper attempts to address the problem of how to automatically generate robot trajectories to scan unknown-shaped 3D surfaces in automated manufacturing and quality inspection. Specifically, the paper proposes a method that uses visual feedback (through image focus metrics) to automatically determine the optimal distance and angle between the camera and the surface being inspected, and generate the optimal scanning trajectory. Additionally, the paper explores how to determine the position and orientation of a known-shaped object in an unknown location to execute a known inspection trajectory.
### Background and Problem Definition
In modern industrial environments, there is an increasing demand to reduce manual labor and replace it with machines to achieve automated tasks. Particularly in quality inspection processes, manual inspection is a highly repetitive and tedious task, and prolonged manual labor can easily lead to operator judgment errors. Therefore, a fast, flexible, and reliable quality inspection process is needed. The application of industrial robots and AI-based quality inspection methods has become key to solving this problem.
### Main Challenges
1. **Unknown-shaped surfaces**: If the CAD data of the component is unavailable or unprocessable, the surface shape is unknown.
2. **Surfaces in unknown locations**: Even if a robot scanning trajectory is available, in many inspection scenarios, the surface may not always be in a fixed predefined location.
### Solution
The paper proposes a method that uses visual feedback (through image focus metrics) to automatically determine the optimal distance and angle between the camera and the surface being inspected, and generate the optimal scanning trajectory. The specific steps are as follows:
1. **Determine the optimal distance and angle**: Using the Fibonacci search method, the optimal distance and angle between the camera and the surface are automatically determined using the image focus metric (FM) as the optimization target.
2. **Generate scanning trajectories for unknown surfaces**: Using Dynamic Movement Primitives (DMPs) to encode motion trajectories, optimized through CMA-ES and a simplified version of the PoWER algorithm, to generate the optimal scanning trajectory.
3. **Determine the position and orientation of known-shaped objects**: In unknown locations, the position and orientation of known-shaped objects are determined using the Nelder-Mead simplex method and a simple linear search algorithm, to execute the known inspection trajectory.
### Experimental Results
- **Optimal distance and angle**: Experimental results show that after about 10 iterations, the optimal distance and angle between the camera and the surface can be stably determined.
- **Scanning trajectory generation**: In both simulated and actual experiments, the time required to generate scanning trajectories for unknown freeform surfaces is less than 50 iterations.
- **Positioning known-shaped objects**: In experiments, the time required to determine the position and orientation of known-shaped objects is less than 30 iterations.
### Conclusion
The automatic trajectory generation method proposed in the paper performs excellently in terms of accuracy and efficiency, and can be applied in actual industrial environments, improving the flexibility and reliability of automated manufacturing and quality inspection.