Adaptable navigation of magnetic microrobots

Charlotte Allard

DOI: https://doi.org/10.1038/s41578-024-00656-3

IF: 83.5

2024-01-27

Nature Reviews Materials

Abstract:An article in Nature Machine Intelligence presents an adaptable method to control magnetic microrobots' navigation using reinforcement learning.

materials science, multidisciplinary,nanoscience & nanotechnology

What problem does this paper attempt to address?

Navigating micro- and nano-motors/swimmers with machine learning: Challenges and future directions

Jueyi Xue,Hamid Alinejad-Rokny,Kang Liang

DOI: https://doi.org/10.1016/j.chphma.2024.06.001

2024-07-01

ChemPhysMater

Abstract:Micro-/nano-motors (MNMs) or swimmers are minuscule machines that can convert various forms of energy, such as chemical, electrical, or magnetic energy, into motion. These devices have attracted significant attention owing to their potential application in a wide range of fields such as drug delivery, sensing, and microfabrication. However, owing to their diverse shapes, sizes, and structural/chemical compositions, the development of MNMs faces several challenges, such as understanding their structure-function relationships, which is crucial for achieving precise control over their motion within complex environments. In recent years, machine learning techniques have shown promise in addressing these challenges and improving the performance of MNMs. Machine learning techniques can analyze large amounts of data, learn from patterns, and make predictions, thereby enabling MNMs to navigate complex environments, avoid obstacles, and perform tasks with higher efficiency and reliability. This review introduces the current state-of-the-art machine learning techniques in MNM research, with a particular focus on employing machine learning to understand and manipulate the navigation and locomotion of MNMs. Finally, we discuss the challenges and opportunities in this field and suggest future research directions.
Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment

Amar Salehi,Soleiman Hosseinpour,Nasrollah Tabatabei,Mahmoud Soltani Firouz,Tingting Yu

DOI: https://doi.org/10.3390/mi15010112

IF: 3.4

2024-01-10

Micromachines

Abstract:Microrobotics has opened new horizons for various applications, especially in medicine. However, it also witnessed challenges in achieving maximum optimal performance. One key challenge is the intelligent, autonomous, and precise navigation control of microrobots in fluid environments. The intelligence and autonomy in microrobot control, without the need for prior knowledge of the entire system, can offer significant opportunities in scenarios where their models are unavailable. In this study, two control systems based on model-free deep reinforcement learning were implemented to control the movement of a disk-shaped magnetic microrobot in a real-world environment. The training and results of an off-policy SAC algorithm and an on-policy TRPO algorithm revealed that the microrobot successfully learned the optimal path to reach random target positions. During training, the TRPO exhibited a higher sample efficiency and greater stability. The TRPO and SAC showed 100% and 97.5% success rates in reaching the targets in the evaluation phase, respectively. These findings offer basic insights into achieving intelligent and autonomous navigation control for microrobots to advance their capabilities for various applications.

chemistry, analytical,nanoscience & nanotechnology,instruments & instrumentation,physics, applied
Autonomous 3D positional control of a magnetic microrobot using reinforcement learning

Sarmad Ahmad Abbasi,Awais Ahmed,Seungmin Noh,Nader Latifi Gharamaleki,Seonhyoung Kim,A. M. Masum Bulbul Chowdhury,Jin-young Kim,Salvador Pané,Bradley J. Nelson,Hongsoo Choi

DOI: https://doi.org/10.1038/s42256-023-00779-2

IF: 23.8

2024-01-11

Nature Machine Intelligence

Abstract:Magnetic microrobots have shown promise in the field of biomedical engineering, facilitating precise drug delivery, non-invasive diagnosis and cell-based therapy. Current techniques for controlling the motion of such microrobots rely on the assumption of homogenous magnetic fields and are significantly influenced by a microrobot's properties and surrounding environment. These strategies lack a sense of generality and adaptability when changing the environment or microrobot and exhibit a moderate delay due to independent control of the electromagnetic actuation system and microrobot's position. To address these issues, we propose a machine learning-based positional control of magnetic microrobots via gradient fields generated by electromagnetic coils. We use reinforcement learning and a gradual training approach to control the three-dimensional position of a microrobot within a defined working area by directly managing the coil currents. We develop a simulation environment for initial exploration to reduce the overall training time. After simulation training, the learning process is transferred to a physical electromagnetic actuation system that reflects real-world intricacies. We compare our method to conventional proportional-integral-derivative control; our system is more accurate and efficient. The proposed method was combined with path planning algorithms to allow fully autonomous control. The presented approach is an alternative to complex mathematical models, which are sensitive to variations in microrobot design, the environment and the nonlinearity of magnetic systems.

computer science, artificial intelligence, interdisciplinary applications
Micro/Nano Motor Navigation and Localization via Deep Reinforcement Learning

Yuguang Yang,Michael A. Bevan,Bo Li

DOI: https://doi.org/10.1002/adts.202000034

2020-04-28

Advanced Theory and Simulations

Abstract:Efficient navigation and precise localization of Brownian micro/nano self‐propelled motor particles within complex landscapes could enable future high‐tech applications involving for example drug delivery, precision surgery, oil recovery, and environmental remediation. Here, a model‐free deep reinforcement learning algorithm based on bio‐inspired neural networks is employed to enable different types of micro/nano motors to be continuously controlled to carry out complex navigation and localization tasks. Micro/nano motors with either tunable self‐propelling speeds or orientations or both, are found to exhibit strikingly different dynamics. In particular, distinct control strategies are required to achieve effective navigation in free space and obstacle environments, as well as under time constraints. The findings provide fundamental insights into active dynamics of Brownian particles controlled using artificial intelligence and could guide the design of motor and robot control systems to meet diverse application requirements. A generic deep reinforcement learning algorithm is developed to compute navigation and localization control policies for micro/nano motors of different translational and rotational dynamics. This artificial intelligence approach provides a new route toward the design of motor and robot control systems to meet diverse application requirements.

multidisciplinary sciences
Efficient Model Learning and Adaptive Tracking Control of Magnetic Micro-Robots for Non-Contact Manipulation

Yongyi Jia,Shu Miao,Junjian Zhou,Niandong Jiao,Lianqing Liu,Xiang Li

2024-03-21

Abstract:Magnetic microrobots can be navigated by an external magnetic field to autonomously move within living organisms with complex and unstructured environments. Potential applications include drug delivery, diagnostics, and therapeutic interventions. Existing techniques commonly impart magnetic properties to the target object,or drive the robot to contact and then manipulate the object, both probably inducing physical damage. This paper considers a non-contact formulation, where the robot spins to generate a repulsive field to push the object without physical contact. Under such a formulation, the main challenge is that the motion model between the input of the magnetic field and the output velocity of the target object is commonly unknown and difficult to analyze. To deal with it, this paper proposes a data-driven-based solution. A neural network is constructed to efficiently estimate the motion model. Then, an approximate model-based optimal control scheme is developed to push the object to track a time-varying trajectory, maintaining the non-contact with distance constraints. Furthermore, a straightforward planner is introduced to assess the adaptability of non-contact manipulation in a cluttered unstructured environment. Experimental results are presented to show the tracking and navigation performance of the proposed scheme.

Robotics
Enabling microrobotic chemotaxis via reset-free hierarchical reinforcement learning

Tongzhao Xiong,Zhaorong Liu,Chong Jin Ong,Lailai Zhu

2024-08-14

Abstract:Microorganisms have evolved diverse strategies to propel in viscous fluids, navigate complex environments, and exhibit taxis in response to stimuli. This has inspired the development of synthetic microrobots, where machine learning (ML) is playing an increasingly important role. Can ML endow these robots with intelligence resembling that developed by their natural counterparts over evolutionary timelines? Here, we demonstrate chemotactic navigation of a multi-link articulated microrobot using two-level hierarchical reinforcement learning (RL). The lower-level RL allows the robot -- featuring either a chain or ring topology -- to acquire topology-specific swimming gaits: wave propagation characteristic of flagella or body oscillation akin to an ameboid. Such flagellar and ameboid microswimmers, further enabled by the higher-level RL, accomplish chemotactic navigation in prototypical biologically-relevant scenarios that feature conflicting chemoattractants, pursuing a swimming bacterial mimic, steering in vortical flows, and squeezing through tight constrictions. Additionally, we achieve reset-free, partially observable RL, where the robot observes only its joint angles and local scalar quantities. This advancement illuminates solutions for overcoming the persistent challenges of manual resets and partial observability in real-world microrobotic RL.

Soft Condensed Matter,Biological Physics
Towards Independent Control of Multiple Magnetic Mobile Microrobots

Sagar Chowdhury,Wuming Jing,David J Cappelleri

DOI: https://doi.org/10.3390/mi7010003

2015-12-29

Abstract:In this paper, we have developed an approach for independent autonomous navigation of multiple microrobots under the influence of magnetic fields and validated it experimentally. We first developed a heuristics based planning algorithm for generating collision-free trajectories for the microrobots that are suitable to be executed by an available magnetic field. Second, we have modeled the dynamics of the microrobots to develop a controller for determining the forces that need to be generated for the navigation of the robots along the trajectories at a suitable control frequency. Next, an optimization routine is developed to determine the input currents to the electromagnetic coils that can generate the required forces for the navigation of the robots at the controller frequency. We then validated our approach by simulating an electromagnetic system that contains an array of sixty-four magnetic microcoils designed for generating local magnetic fields suitable for simultaneous independent actuation of multiple microrobots. Finally, we prototyped an m m -scale version of the system and present experimental results showing the validity of our approach.
3D Navigation Control of Untethered Magnetic Microrobot in Centimeter-Scale Workspace Based on Field-of-View Tracking Scheme

Liushuai Zheng,Yuanjun Jia,Dingran Dong,Wahshing Lam,Dongfang Li,Haibo Ji,Dong Sun

DOI: https://doi.org/10.1109/tro.2021.3118205

IF: 7.8

2021-01-01

IEEE Transactions on Robotics

Abstract:Automatic 3D navigation control of microrobot in large microenvironment is one of the primary challenges hindering its applications. In this article, we present a systematic approach to use an electromagnetic manipulation system to deal with this challenge. Two movable orthogonal microscopic cameras scan the microenvironment on the top view and side view to build a stereo occupancy map automatically. The initial position of microrobot can be located on the basis of the similarity curves generated by the scanning sequences. In accordance with the distribution characteristic of electromagnetic fields in workspace, an enhanced rapidly exploring random tree algorithm is proposed to avoid obstacles in the complex environment. To ensure continuous visual servo in wide range, a field-of-view tracking method is developed using a specific image definition evaluation algorithm. A prescribed performance controller with disturbance observer is designed, which guarantees that the microrobot can remain in the view of microscopic cameras, and the transient and steady-state performance of the system can satisfy the expected requirements. A set of experiments are performed to verify the effectiveness of the proposed automatic 3D navigation strategy. Experimental results show that the microrobot can navigate automatically in a $\boldsymbol{12}\,\times 10\,\times \text{10-mm}$ microenvironment containing obstacles, and achieve reliable field-of-view tracking and path following at high speed.

robotics
Multiple Magneto-Optical Microrobotic Collectives with Selective Control in Three Dimensions Under Water

Mengmeng Sun,Shihao Yang,Jialin Jiang,Qianqian Wang,Li Zhang

DOI: https://doi.org/10.1002/smll.202310769

IF: 13.3

Small

Abstract:Inspired by natural swarms, various methods are developed to create artificial magnetic microrobotic collectives. However, these magnetic collectives typically receive identical control inputs from a common external magnetic field, limiting their ability to operate independently. And they often rely on interfaces or boundaries for controlled movement, posing challenges for independent, three-dimensional(3D) navigation of multiple magnetic collectives. To address this challenge, self-assembled microrobotic collectives are proposed that can be selectively actuated in a combination of external magnetic and optical fields. By harnessing both actuation methods, the constraints of single actuation approaches are overcome. The magnetic field excites the self-assembly of colloids and maintains the self-assembled microrobotic collectives without disassembly, while the optical field drives selected microrobotic collectives to perform different tasks. The proposed magnetic-photo microrobotic collectives can achieve independent position and path control in the two-dimensional (2D) plane and 3D space. With this selective control strategy, the microrobotic collectives can cooperate in convection and mixing the dye in a confined space. The results present a systematic approach for realizing selective control of multiple microrobotic collectives, which can address multitasking requirements in complex environments.
Closed Loop Control of Bubble-Propelled Microrobots

Yanda Yang,David Rivas,Max Sokolich,Sambeeta Das

DOI: https://doi.org/10.1109/marss58567.2023.10294166

Abstract:Bubble-propelled microrobots have an advantage of relatively swift movement compared to most other types of microrobots, which makes them well suited for applications such as micromanipulation or movement in flows, but their high speed also poses challenges in precisely controlling their motion. This study proposes automated control of the microrobots using visual feedback and steering with uniform magnetic fields to constrain the microrobot's moving direction. The implementation of a closed-loop control mechanism ensures precise autonomous navigation along prescribed trajectories. Experimental results demonstrate that this approach achieves satisfactory tracking performance, with an average error of 6. 7 μm for a microrobot with a diameter of 24 μm.
Magnetically Driven Micro and Nanorobots

Huaijuan Zhou,Carmen C Mayorga-Martinez,Salvador Pané,Li Zhang,Martin Pumera

DOI: https://doi.org/10.1021/acs.chemrev.0c01234

2021-04-28

Abstract:Manipulation and navigation of micro and nanoswimmers in different fluid environments can be achieved by chemicals, external fields, or even motile cells. Many researchers have selected magnetic fields as the active external actuation source based on the advantageous features of this actuation strategy such as remote and spatiotemporal control, fuel-free, high degree of reconfigurability, programmability, recyclability, and versatility. This review introduces fundamental concepts and advantages of magnetic micro/nanorobots (termed here as "MagRobots") as well as basic knowledge of magnetic fields and magnetic materials, setups for magnetic manipulation, magnetic field configurations, and symmetry-breaking strategies for effective movement. These concepts are discussed to describe the interactions between micro/nanorobots and magnetic fields. Actuation mechanisms of flagella-inspired MagRobots (i.e., corkscrew-like motion and traveling-wave locomotion/ciliary stroke motion) and surface walkers (i.e., surface-assisted motion), applications of magnetic fields in other propulsion approaches, and magnetic stimulation of micro/nanorobots beyond motion are provided followed by fabrication techniques for (quasi-)spherical, helical, flexible, wire-like, and biohybrid MagRobots. Applications of MagRobots in targeted drug/gene delivery, cell manipulation, minimally invasive surgery, biopsy, biofilm disruption/eradication, imaging-guided delivery/therapy/surgery, pollution removal for environmental remediation, and (bio)sensing are also reviewed. Finally, current challenges and future perspectives for the development of magnetically powered miniaturized motors are discussed.
Adaptive Pattern and Motion Control of Magnetic Microrobotic Swarms

Jiangfan Yu,Lidong Yang,Xingzhou Du,Hui Chen,Tiantian Xu,Li Zhang

DOI: https://doi.org/10.1109/tro.2021.3130432

IF: 7.8

2021-01-01

IEEE Transactions on Robotics

Abstract:Reconfigurable microrobotic swarms and controllable active matter systems have drawn extensive attention recently. Developing effective actuation strategies and control schemes that enable embodied intelligence of microscopic swarms are both major challenges. In this work, we realize the generation of an elliptical paramagnetic nanoparticle swarm (EPNS) with enhanced dexterity for adaptive locomotion, and subsequently a fuzzy control strategy is developed for automatically tuning pattern deformation, orientation, and position of the swarm. By adjusting the input field, the aspect ratio of the EPNS will change accordingly, and we demonstrate its adaptive navigation through curved and narrowed channel by performing pattern reconfigurations. Moreover, using the proposed control strategy, precise matches can be reached between the controlled swarms and the desired patterns. Finally, to show the high compatibility of the control strategy, we employ ribbon-like colloidal swarms driven by oscillating magnetic field, and the results also validate the effectiveness of the strategy.

robotics
Anisotropic Magnetized Tubular Microrobots for Bioinspired Adaptive Locomotion

Guanghui Yan,Borui Xu,Xiaojie Shi,Yang Zong,Yue Wu,Jinrun Liu,Yi Ouyang,Guoxiang Chen,Jizhai Cui,Yongfeng Mei

DOI: https://doi.org/10.1016/j.apmt.2022.101457

IF: 8.663

2022-01-01

Applied Materials Today

Abstract:Mimicry of motile microorganisms offers new concepts and strategies to design and manipulate microrobots. As a representative instance, Escherichia coli switches its motility between swimming and tumbling to adapt to surroundings. Inspired by this, we present a tubular microrobot behaving rolling and tumbling motion modes analogous to the motion of Escherichia coli, as a result of the anisotropic magnetized status with radially magnetized status at rolling and longitudinally magnetized one at tumbling. The motion behavior of microrobot as rolling or tumbling is determined by the frequency of rotating magnetic field. The threshold frequency for switching is closely related to the amplitude of magnetic field, viscosity of solution and size of microrobot. The net locomotion near substrate comes from the friction and pressure, and the motion switching is due to the magnetized statuses and discrepancy of drag force. The switch from rolling to tumbling endows the microrobot with adaptive locomotion to cross obstacles. Furthermore, microfluidic manipulation through rotational microrobot is achieved based on the local microvortex, which is generated from the hydrodynamic interaction between microrobot and substrate. This manipulation is applied in cargo delivery and release by altering surrounding microfluid via switchable motion modes. This work opens up new possibilities in improving moving ability and extending functions of microrobots through bioinspired motion behaviors. (C) 2022 Elsevier Ltd. All rights reserved.
Learning automatic navigation control skills for miniature helical robots from human demonstrations

Mengde Li,Xutian Deng,Fuqiang Zhao,Mingchang Li,Sheng Liu,Miao Li

DOI: https://doi.org/10.1016/j.engappai.2024.109187

IF: 8

2024-09-03

Engineering Applications of Artificial Intelligence

Abstract:Magnetic micro-robotic technology holds immense potential for revolutionizing minimally invasive procedures, particularly in the realm of interventional medicine. The ability to effectively and precisely control micro-scale, magnetically actuated robots in real-world scenarios is important. Mastering this capability promises to elevate the precision and efficacy of medical interventions, thereby enhancing patient outcomes. Numerous methods were proposed in previous work and achieved significant progress. However, these efforts primarily focused on model-based strategies and control improvements, with little attention given to the manipulation of the surgeon. This paper presents an exploration of an imitation learning approach, leveraging extensive manual operation experiments, to replicate the task of robotic navigation in simulated vascular environments. The control strategies are directly acquired from experimental observations and encapsulated within a high-dimensional neural network, specifically a tailored variant of the Residual Network (ResNet). The robustness and effectiveness of our proposed methodology are validated through comprehensive experimentation. In automatic navigation trials, the average error spanned from 2.29 mm to 3.32 mm, leading to a mean trajectory deviation of approximately 2.92 mm. The average error rate is 31% lower than that observed in traditional model-based Proportional Integral Derivative (PID) controller (approximately 3.81 mm). In addition, the maximum error (4.87 mm) is 83% of that of the traditional method (5.85 mm). Our findings emphasize the viability and benefits of learning-based techniques in micro-robot control, paving the way for innovative control strategies in interventional surgeries applications.

automation & control systems,computer science, artificial intelligence,engineering, electrical & electronic, multidisciplinary
Closed-loop Control of Catalytic Janus Microrobots

Max Sokolich,David Rivas,Zameer Hussain Shah,Sambeeta Das

DOI: https://doi.org/10.48550/arXiv.2210.11460

2022-10-21

Abstract:We report a closed-loop control system for paramagnetic catalytically self-propelled Janus microrobots. We achieve this control by employing electromagnetic coils that direct the magnetic field in a desired orientation to steer the microrobots. The microrobots move due to the catalytic decomposition of hydrogen peroxide, during which they align themselves to the magnetic torques applied to them. Because the angle between their direction of motion and their magnetic orientation is a priori unknown, an algorithm is used to determine this angular offset and adjust the magnetic field appropriately. The microrobots are located using real-time particle tracking that integrates with a video camera. A target location or desired trajectory can be drawn by the user for the microrobots to follow.

Robotics,Chemical Physics
Vision-Based Automated Control of Magnetic Microrobots

Xiaoqing Tang,Yuke Li,Xiaoming Liu,Dan Liu,Zhuo Chen,Tatsuo Arai

DOI: https://doi.org/10.3390/mi13020337

IF: 3.4

2022-02-21

Micromachines

Abstract:Magnetic microrobots are vital tools for targeted therapy, drug delivery, and micromanipulation on cells in the biomedical field. In this paper, we report an automated control and path planning method of magnetic microrobots based on computer vision. Spherical microrobots can be driven in the rotating magnetic field generated by electromagnetic coils. Under microscopic visual navigation, robust target tracking is achieved using PID–based closed–loop control combined with the Kalman filter, and intelligent obstacle avoidance control can be achieved based on the dynamic window algorithm (DWA) implementation strategy. To improve the performance of magnetic microrobots in trajectory tracking and movement in complicated environments, the magnetic microrobot motion in the flow field at different velocities and different distribution obstacles was investigated. The experimental results showed that the vision-based controller had an excellent performance in a complex environment and that magnetic microrobots could be controlled to move to the target position smoothly and accurately. We envision that the proposed method is a promising opportunity for targeted drug delivery in biological research.

chemistry, analytical,nanoscience & nanotechnology,instruments & instrumentation,physics, applied
Hierarchical deep reinforcement learning controlled three-dimensional navigation of microrobots in blood vessels

Yuguang Yang,Michael A. Bevan,Bo Li

DOI: https://doi.org/10.48550/arXiv.2103.12966

2021-03-24

Soft Condensed Matter

Abstract:Designing intelligent microrobots that can autonomously navigate and perform instructed routines in blood vessels, a complex and crowded environment with obstacles including dense cells, different flow patterns and diverse vascular geometries, can offer enormous possibilities in biomedical applications. Here we report a hierarchical control scheme that enables a microrobot to efficiently navigate and execute customizable routines in blood vessels. The control scheme consists of two highly decoupled components: a high-level controller setting short-ranged dynamic targets to guide the microrobot to follow a preset path and a low-level deep reinforcement learning (DRL) controller responsible for maneuvering microrobots towards these dynamic guiding targets. The proposed DRL controller utilizes three-dimensional (3D) convolutional neural networks and is capable of learning control policy directly from a coarse raw 3D sensory input. In blood vessels with rich configurations of red blood cells and vessel geometry, the control scheme enables efficient navigation and faithful execution of instructed routines. The control scheme is also robust to adversarial perturbations including blood flows. This study provides a proof-of-principle for designing data-driven control systems for autonomous navigation in vascular networks; it illustrates the great potential of artificial intelligence for broad biomedical applications such as target drug delivery, blood clots clear, precision surgery, disease diagnosis, and more.
Data‐Driven Navigation of Ferromagnetic Soft Continuum Robots Based on Machine Learning

Yangyang Ni,Yuxuan Sun,Huajian Zhang,Xingxiang Li,Shiwu Zhang,Mujun Li

DOI: https://doi.org/10.1002/aisy.202200167

IF: 7.298

2023-01-01

Advanced Intelligent Systems

Abstract:Ferromagnetic soft continuum robots (FSCRs) have great potential in biomedical applications due to their miniaturization and remote control capabilities. However, to direct the FSCR accurately and effectively, it is critical to realize inverse kinematics control in navigation, which is difficult for existing mechanical models. Herein, with the path segmentation strategy, an automatic method to navigate the FSCR in different paths based on machine learning is developed. A data‐driven artificial neural network (ANN) model to guide the steering of the magnetically responsive tip is presented. Using parametric simulations as the training data, the ANN model shows good generalization performance to predict control parameters. Moreover, the basic framework of the learning model remains effective when the FSCR materials change, which shows high scalability and is important for adapting to various environments. The study presents a promising strategy for guiding FSCRs in the narrow and tortuous vasculature, which is essential for many biomedical operations.
Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

Yuguang Yang,Michael A. Bevan,Bo Li

DOI: https://doi.org/10.1002/aisy.201900106

IF: 7.298

2019-11-04

Advanced Intelligent Systems

Abstract:Equipping micro‐/nanoscale colloidal robots with artificial intelligence (AI) such that they can efficiently navigate in unknown complex environments can dramatically impact their use in emerging applications such as precision surgery and targeted nanodrug delivery. Herein, a model‐free deep reinforcement learning algorithm is developed that trains colloidal robots to efficiently navigate in unknown environments with random obstacles. A deep neural network architecture is used that enables the colloidal robots to mimic animal navigation decision‐making by directly processing raw sensor input and decomposing long‐range navigations to short‐range ones. The trained robot agents learn to make navigation decisions regarding both obstacle avoidance and travel time minimization, based solely on local sensory inputs without prior knowledge of the global environment. Such agents with biologically inspired mechanisms can acquire competitive navigation capabilities in large‐scale, complex environments containing obstacles of diverse shapes, sizes, and configurations. Herein, the potential of AI to enable colloidal robots in extensive applications is illustrated. A bioinspired deep reinforcement learning architecture is developed to equip self‐propelled colloidal robots with artificial intelligence. The intelligent robots are capable of efficiently navigating in unknown environments with random obstacles.
Magnetic-Field-Inspired Navigation for Robots in Complex and Unknown Environments

Ahmad Ataka,Hak-Keung Lam,Kaspar Althoefer

DOI: https://doi.org/10.3389/frobt.2022.834177

2022-02-18

Abstract:Over the course of the past decade, we have witnessed a huge expansion in robotic applications, most notably from well-defined industrial environments into considerably more complex environments. The obstacles that these environments often contain present robotics with a new challenge - to equip robots with a real-time capability of avoiding them. In this paper, we propose a magnetic-field-inspired navigation method that significantly has several advantages over alternative systems. Most importantly, 1) it guarantees obstacle avoidance for both convex and non-convex obstacles, 2) goal convergence is still guaranteed for point-like robots in environments with convex obstacles and non-maze concave obstacles, 3) no prior knowledge of the environment, such as the position and geometry of the obstacles, is needed, 4) it only requires temporally and spatially local environmental sensor information, and 5) it can be implemented on a wide range of robotic platforms in both 2D and 3D environments. The proposed navigation algorithm is validated in simulation scenarios as well as through experimentation. The results demonstrate that robotic platforms, ranging from planar point-like robots to robot arm structures such as the Baxter robot, can successfully navigate toward desired targets within an obstacle-laden environment.

Adaptable navigation of magnetic microrobots

Navigating micro- and nano-motors/swimmers with machine learning: Challenges and future directions

Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment

Autonomous 3D positional control of a magnetic microrobot using reinforcement learning

Micro/Nano Motor Navigation and Localization via Deep Reinforcement Learning

Efficient Model Learning and Adaptive Tracking Control of Magnetic Micro-Robots for Non-Contact Manipulation

Enabling microrobotic chemotaxis via reset-free hierarchical reinforcement learning

Towards Independent Control of Multiple Magnetic Mobile Microrobots

3D Navigation Control of Untethered Magnetic Microrobot in Centimeter-Scale Workspace Based on Field-of-View Tracking Scheme

Multiple Magneto-Optical Microrobotic Collectives with Selective Control in Three Dimensions Under Water

Closed Loop Control of Bubble-Propelled Microrobots

Magnetically Driven Micro and Nanorobots

Adaptive Pattern and Motion Control of Magnetic Microrobotic Swarms

Anisotropic Magnetized Tubular Microrobots for Bioinspired Adaptive Locomotion

Learning automatic navigation control skills for miniature helical robots from human demonstrations

Closed-loop Control of Catalytic Janus Microrobots

Vision-Based Automated Control of Magnetic Microrobots

Hierarchical deep reinforcement learning controlled three-dimensional navigation of microrobots in blood vessels

Data‐Driven Navigation of Ferromagnetic Soft Continuum Robots Based on Machine Learning

Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

Magnetic-Field-Inspired Navigation for Robots in Complex and Unknown Environments