Research on Multi-Hole Localization Tracking Based on a Combination of Machine Vision and Deep Learning

Rong Hou,Jianping Yin,Yanchen Liu,Huijuan Lu
DOI: https://doi.org/10.3390/s24030984
IF: 3.9
2024-02-03
Sensors
Abstract:In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot's work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand–eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs.
engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
What problem does this paper attempt to address?
### Problems Addressed by the Paper This paper aims to address the issues of low efficiency, high intensity, and certain dangers associated with manual assembly of workpieces in industrial production processes. Traditional machine learning algorithms struggle to adapt to the complexity of the current industrial environment, and environmental changes can significantly impact the accuracy of robot operations. Therefore, this paper proposes a method that combines machine vision with the YOLOv5 deep learning model to obtain positioning information for disc-shaped multi-holes. It uses ROS communication to control the robotic arm, enhancing environmental interference resistance and work efficiency while reducing risks to humans. ### Main Contributions 1. **Multi-hole Positioning and Tracking Method**: A method combining machine vision and deep learning for multi-hole positioning and tracking is proposed, reducing the impact of a single method. 2. **YOLOv5 Network Model**: The YOLOv5 network model is used for data training and detection, improving the target recognition's resistance to environmental interference. 3. **ROS Communication**: Communication between the upper and lower computers uses ROS communication, providing good real-time performance for robot control. 4. **Research Approach**: The research approach in this paper provides important insights for future researchers to improve their studies in large-scale applications. ### System Implementation Process 1. **Image Acquisition**: Images are obtained through a camera, and high-quality target area images are acquired through good light distribution. 2. **Image Processing and Data Model Training**: Image processing and data model training are performed on a PC to extract the feature values of the disc-shaped multi-holes and obtain the multi-hole coordinate information. 3. **System Calibration**: The extracted coordinate information is converted into the robotic arm coordinate system to control the robotic arm for real-time tracking. 4. **Upper and Lower Computer Communication**: Data communication between the PC and the robotic arm industrial control machine is realized. ### Target Recognition 1. **YOLOv5 Framework**: Considering the various interference factors in the actual environment, this paper uses the YOLOv5 (You Only Look Once version 5) deep learning framework to accurately capture target objects. The characteristics of YOLOv5 include fast detection speed, high precision, and good small object detection capability. 2. **Model Training**: Images are collected using a high-resolution Realsense D415 camera, and a dataset suitable for YOLOv5 training is generated. The accuracy of target recognition is evaluated by analyzing relevant indicators. The main indicators used include Precision, Recall, and F1 Score. ### Summary By combining machine vision and deep learning technology, this paper proposes an efficient multi-hole positioning and tracking method, addressing the shortcomings of traditional methods in complex environments and improving the efficiency and reliability of industrial automated production lines.