Abstract:Significant progress has been made in enhancing the motion capabilities of quadruped robots in unstructured environments due to advancements in hardware and control algorithms. However, limited research has been conducted on the fault-tolerant control of quadruped robots, which is crucial for their operation in remote or extreme environments like disaster sites. In this paper, we primarily focus on fault-tolerant strategies for common joint-stuck situations. By leveraging the static stability of quadruped robots, it becomes possible to adjust their control policies and enable them to continue following predetermined trajectories. We introduce a contextual meta-reinforcement learning (Meta-RL) method to design fault-tolerant policies. This method infers task-related latent vectors from the context to assist in training the policy network, ensuring both conciseness and optimality in various situations. Additionally, to expedite algorithm training, we propose a reference action generator (RAG). To validate the proposed algorithm, extensive simulations and physical experiments are conducted. The results demonstrate that our method allows the robot to maintain its trajectory even when faced with motor locking. Furthermore, our method outperforms all baseline algorithms, highlighting its superiority in terms of fault tolerance. Note to Practitioners —The motivation of this article is to provide fault-tolerant policies for quadruped robots, specifically referring to the policies for joint-stuck situations. Previous fault-tolerant strategies either require individually designing control strategies for each joint stuck task, which brings a significant workload to designers, or adopting a unified strategy that cannot provide the optimal strategy for each task. In this article, we utilize the Meta-RL method to handle the joint stuck issue in robots for the first time. By combining the context encoder and RAG, we can provide more suitable policies for various motor-stuck tasks. Both the simulation and physical experiments validate the effectiveness and applicability of this method.

Enabling microrobotic chemotaxis via reset-free hierarchical reinforcement learning

Navigating micro- and nano-motors/swimmers with machine learning: Challenges and future directions

Chemotaxis of an elastic flagellated microrobot

Meta Reinforcement Learning of Locomotion Policy for Quadruped Robots with Motor Stuck

Emergence of Chemotactic Strategies with Multi-Agent Reinforcement Learning

Active particles using reinforcement learning to navigate in complex motility landscapes

A Reinforcement Learning Approach to Find Optimal Propulsion Strategy for Microrobots Swimming at Low Reynolds Number

Efficient Navigation of Colloidal Robots in an Unknown Environment via Deep Reinforcement Learning

Micro/Nano Motor Navigation and Localization via Deep Reinforcement Learning

Intelligent Navigation of a Magnetic Microrobot with Model-Free Deep Reinforcement Learning in a Real-World Environment

Navigation of micro-robot swarms for targeted delivery using reinforcement learning

Hierarchical Planning with Deep Reinforcement Learning for 3D Navigation of Microrobots in Blood Vessels

Hierarchical deep reinforcement learning controlled three-dimensional navigation of microrobots in blood vessels

Model-Based Reinforcement Learning for Ultrasound-Driven Autonomous Microrobots

Counterfactual rewards promote collective transport using individually controlled swarm microrobots

Training microrobots to swim by a large language model

Adaptable navigation of magnetic microrobots

Anisotropic Magnetized Tubular Microrobots for Bioinspired Adaptive Locomotion

Autonomous robotic nanofabrication with reinforcement learning

Residual Model Learning for Microrobot Control

Programmable Control of Ultrasound Swarmbots through Reinforcement Learning