Abstract:Significant progress has been made in enhancing the motion capabilities of quadruped robots in unstructured environments due to advancements in hardware and control algorithms. However, limited research has been conducted on the fault-tolerant control of quadruped robots, which is crucial for their operation in remote or extreme environments like disaster sites. In this paper, we primarily focus on fault-tolerant strategies for common joint-stuck situations. By leveraging the static stability of quadruped robots, it becomes possible to adjust their control policies and enable them to continue following predetermined trajectories. We introduce a contextual meta-reinforcement learning (Meta-RL) method to design fault-tolerant policies. This method infers task-related latent vectors from the context to assist in training the policy network, ensuring both conciseness and optimality in various situations. Additionally, to expedite algorithm training, we propose a reference action generator (RAG). To validate the proposed algorithm, extensive simulations and physical experiments are conducted. The results demonstrate that our method allows the robot to maintain its trajectory even when faced with motor locking. Furthermore, our method outperforms all baseline algorithms, highlighting its superiority in terms of fault tolerance. Note to Practitioners —The motivation of this article is to provide fault-tolerant policies for quadruped robots, specifically referring to the policies for joint-stuck situations. Previous fault-tolerant strategies either require individually designing control strategies for each joint stuck task, which brings a significant workload to designers, or adopting a unified strategy that cannot provide the optimal strategy for each task. In this article, we utilize the Meta-RL method to handle the joint stuck issue in robots for the first time. By combining the context encoder and RAG, we can provide more suitable policies for various motor-stuck tasks. Both the simulation and physical experiments validate the effectiveness and applicability of this method.

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Meta Reinforcement Learning of Locomotion Policy for Quadruped Robots with Motor Stuck

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Reinforcement Learning based Control of a Quadruped Robot

Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

Hierarchical Reinforcement Learning for Quadruped Locomotion

RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion

Manipulator as a Tail: Promoting Dynamic Stability for Legged Locomotion

Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC

Robot Learning of Mobile Manipulation with Reachability Behavior Priors

Multi-Agent Reinforcement Learning Tracking Control of a Bionic Wheel-Legged Quadruped

Learning of Long-Horizon Sparse-Reward Robotic Manipulator Tasks With Base Controllers

A Hierarchical Framework for Quadruped Robots Gait Planning Based on DDPG

Learning Visual Quadrupedal Loco-Manipulation from Demonstrations

Learning Force Control for Legged Manipulation

Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks

HiLMa-Res: A General Hierarchical Framework via Residual RL for Combining Quadrupedal Locomotion and Manipulation

Bridging Locomotion and Manipulation Using Reconfigurable Robotic Limbs via Reinforcement Learning

Zero-Shot Retargeting of Learned Quadruped Locomotion Policies Using Hybrid Kinodynamic Model Predictive Control