Abstract:Significant progress has been made in enhancing the motion capabilities of quadruped robots in unstructured environments due to advancements in hardware and control algorithms. However, limited research has been conducted on the fault-tolerant control of quadruped robots, which is crucial for their operation in remote or extreme environments like disaster sites. In this paper, we primarily focus on fault-tolerant strategies for common joint-stuck situations. By leveraging the static stability of quadruped robots, it becomes possible to adjust their control policies and enable them to continue following predetermined trajectories. We introduce a contextual meta-reinforcement learning (Meta-RL) method to design fault-tolerant policies. This method infers task-related latent vectors from the context to assist in training the policy network, ensuring both conciseness and optimality in various situations. Additionally, to expedite algorithm training, we propose a reference action generator (RAG). To validate the proposed algorithm, extensive simulations and physical experiments are conducted. The results demonstrate that our method allows the robot to maintain its trajectory even when faced with motor locking. Furthermore, our method outperforms all baseline algorithms, highlighting its superiority in terms of fault tolerance. Note to Practitioners —The motivation of this article is to provide fault-tolerant policies for quadruped robots, specifically referring to the policies for joint-stuck situations. Previous fault-tolerant strategies either require individually designing control strategies for each joint stuck task, which brings a significant workload to designers, or adopting a unified strategy that cannot provide the optimal strategy for each task. In this article, we utilize the Meta-RL method to handle the joint stuck issue in robots for the first time. By combining the context encoder and RAG, we can provide more suitable policies for various motor-stuck tasks. Both the simulation and physical experiments validate the effectiveness and applicability of this method.

Obtaining fault tolerance avoidance behavior using deep reinforcement learning

Meta Reinforcement Learning of Locomotion Policy for Quadruped Robots with Motor Stuck

A Residual Meta-Reinforcement Learning Method for Training Fault-Tolerant Policies for Quadruped Robots

Failure-aware Policy Learning for Self-assessable Robotics Tasks

Meta-Reinforcement Learning of Hierarchical Fault-Tolerant Controller for Multiple Leg Failures in Hexapod Robots

Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation

Learning Resilient Behaviors for Navigation Under Uncertainty

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees

Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation

Learning Agile Locomotion on Risky Terrains

Learning Push Recovery Behaviors for Humanoid Walking Using Deep Reinforcement Learning

Reactive navigation under a fuzzy rules-based scheme and reinforcement learning for mobile robots

Resilient Legged Local Navigation: Learning to Traverse with Compromised Perception End-to-End

Reinforcement Learning with Adaptive Curriculum Dynamics Randomization for Fault-Tolerant Robot Control

Safe Reinforcement Learning of Robot Trajectories in the Presence of Moving Obstacles

Training a Non-Cooperator to Identify Vulnerabilities and Improve Robustness for Robot Navigation

Self-Learning Robot Autonomous Navigation with Deep Reinforcement Learning Techniques

A Deep Safe Reinforcement Learning Approach for Mapless Navigation.

Intelligent mobile robot navigation in unknown and complex environment using reinforcement learning technique

Dynamic Obstacle Avoidance Technique for Mobile Robot Navigation Using Deep Reinforcement Learning