Magnetic-Controlled Microrobot: Real-Time Detection and Tracking through Deep Learning Approaches

Hao Li,Xin Yi,Zhaopeng Zhang,Yuan Chen
DOI: https://doi.org/10.3390/mi15060756
IF: 3.4
2024-06-06
Micromachines
Abstract:As one of the most significant research topics in robotics, microrobots hold great promise in biomedicine for applications such as targeted diagnosis, targeted drug delivery, and minimally invasive treatment. This paper proposes an enhanced YOLOv5 (You Only Look Once version 5) microrobot detection and tracking system (MDTS), incorporating a visual tracking algorithm to elevate the precision of small-target detection and tracking. The improved YOLOv5 network structure is used to take magnetic bodies with sizes of 3 mm and 1 mm and a magnetic microrobot with a length of 2 mm as the pretraining targets, and the training weight model is used to obtain the position information and motion information of the microrobot in real time. The experimental results show that the accuracy of the improved network model for magnetic bodies with a size of 3 mm is 95.81%, representing an increase of 2.1%; for magnetic bodies with a size of 1 mm, the accuracy is 91.03%, representing an increase of 1.33%; and for microrobots with a length of 2 mm, the accuracy is 91.7%, representing an increase of 1.5%. The combination of the improved YOLOv5 network model and the vision algorithm can effectively realize the real-time detection and tracking of magnetically controlled microrobots. Finally, 2D and 3D detection and tracking experiments relating to microrobots are designed to verify the robustness and effectiveness of the system, which provides strong support for the operation and control of microrobots in an in vivo environment.
chemistry, analytical,nanoscience & nanotechnology,instruments & instrumentation,physics, applied
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: how to achieve real - time detection and tracking of magnetically - controlled microrobots in the biomedical environment. Specifically, the research aims to improve the detection accuracy and tracking ability of small targets (such as magnets with a size of 1 - 3 millimeters and microrobots 2 millimeters long) by improving the YOLOv5 object detection algorithm and combining it with visual algorithms. ### Main problems and challenges of the paper 1. **Challenges in small - target detection**: - The size of the microrobot is very small (for example, 1 - 2 millimeters), which makes it difficult for traditional detection methods to accurately identify and track. - Small targets are easily confused with the background, increasing the difficulty of feature extraction. 2. **Requirements for real - time and accuracy**: - In practical applications, especially in the biomedical field, real - time, high - precision detection and tracking need to be achieved to ensure that the microrobot can accurately reach the target location and perform tasks. 3. **Robustness in complex environments**: - Microrobots usually work in complex in - vivo environments, such as in blood vessels, so they need to maintain stable detection and tracking performance in dynamic and changeable environments. ### Solutions To solve the above problems, the author proposes the following improvement measures: 1. **Improve the YOLOv5 network model**: - Introduce the Swin Transformer module to enhance the feature extraction ability of small targets and reduce the loss of feature information. - Introduce the ODConv module in the final feature fusion part, and adaptively adjust the convolution kernel weights through the multi - dimensional attention mechanism to improve the detection accuracy and robustness. 2. **Combination of visual algorithms**: - Use the monocular vision algorithm to calculate the world coordinates of the target, and obtain three - dimensional depth information through the binocular stereo vision algorithm, so as to achieve more accurate positioning and tracking. 3. **Experimental verification**: - Design two - dimensional and three - dimensional epoxy resin vascular models for experiments to verify the effectiveness and robustness of the system. ### Results The experimental results show that the improved YOLOv5 model has an accuracy rate of 95.81% when detecting 3 - millimeter magnets, which is 2.1% higher than the original model; for 1 - millimeter magnets, the accuracy rate is 91.03%, an increase of 1.33%; for 2 - millimeter - long microrobots, the accuracy rate is 91.7%, an increase of 1.5%. These results indicate that this system can effectively achieve real - time detection and tracking of magnetically - controlled microrobots. ### Summary This paper successfully solves the problem of small - target detection and tracking by improving the YOLOv5 object detection algorithm and combining it with visual algorithms, providing strong support for the application of microrobots in the biomedical field.