Abstract:The development of artificial microscopic robots, like synthetic microswimmers, is one of the state-of-the-art research topics due to their promising biomedical applications. The movement of microswimmers is affected by stringent constraints because of the low Reynolds number of the surrounding environment. Researchers have been working on enhancing microrobots’ propulsion tactics and figuring out new approaches to find optimal propulsion strategies. In this research, we employ a Reinforcement Learning (RL) algorithm, specifically Q-Learning, to train linear-shaped microrobots, comprised of spheres and rods, by introducing an innovative and pioneer coding approach, termed “Basic Coding”, which is utilized to specify states and actions within the RL framework. Basic Coding is a powerful general method that can be employed for different agents in any discrete RL environment. We show how to apply Basic Coding to various microrobots with different geometrical configurations, like a triangular one. Our smart microswimmers, with different numbers of spheres, acquire the knowledge of the optimal propulsion cycle to accomplish large net mechanical displacement without relying on any pre-existing locomotion expertise. The three-sphere linear microrobot recovers the cycle Najafi and Golestanian suggested. The N-sphere microrobots with higher degrees of freedom can find the optimal propulsion cycle within a reasonable number of learning steps and low computational cost utilizing our RL and Basic Coding approach, while the learning step number significantly increases using other methods like Brute-force search. For example, we show this number for the 5-sphere microrobot is 1.19E03 and 8.97E10 steps using our methodology and Brute-force, respectively. Furthermore, our intelligent microrobots can successfully and adaptively find new optimal strategies in indeterministic environments in the presence of uncertainty. Moreover, the effects of learning parameters on our RL agents are investigated in this work.

Model-Based Reinforcement Learning for Ultrasound-Driven Autonomous Microrobots

Programmable Control of Ultrasound Swarmbots through Reinforcement Learning

Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment

Autonomous 3D positional control of a magnetic microrobot using reinforcement learning

Simulation of Nanorobots with Artificial Intelligence and Reinforcement Learning for Advanced Cancer Cell Detection and Tracking

Multi-degree-of-freedom Intelligent Ultrasound Robot System Based on Reinforcement Learning

An Automatic Control Model for Rat-Robot

Residual Model Learning for Microrobot Control

Hierarchical Planning with Deep Reinforcement Learning for 3D Navigation of Microrobots in Blood Vessels

Hierarchical deep reinforcement learning controlled three-dimensional navigation of microrobots in blood vessels

Multi-Modal Autonomous Ultrasound Scanning for Efficient Human–Machine Fusion Interaction

AI-enhanced biomedical micro/nanorobots in microfluidics

Autonomous Tissue Manipulation via Surgical Robot Using Learning Based Model Predictive Control

Enhancing Surgical Robots with Embodied Intelligence for Autonomous Ultrasound Scanning

Autonomic Robotic Ultrasound Imaging System Based on Reinforcement Learning

Machine Learning in Robotic Ultrasound Imaging: Challenges and Perspectives

Motion Planning and Robust Control for the Endovascular Navigation of a Microrobot

Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

A Reinforcement Learning Approach to Find Optimal Propulsion Strategy for Microrobots Swimming at Low Reynolds Number

Enabling microrobotic chemotaxis via reset-free hierarchical reinforcement learning

Micro/Nano Motor Navigation and Localization via Deep Reinforcement Learning