Abstract:Modular robots have the potential for an unmatched ability to perform versatile and robust locomotion. However, designing effective and adaptive locomotion controllers for modular robots is challenging, resulting in a number of model-based methods that typically require various forms of prior knowledge. Deep reinforcement learning (DRL) provides a promising model-free approach for locomotion control by trial-and-error. However, current DRL methods often require extensive interaction data, hindering many possible applications. In this letter, a novel two-level hierarchical locomotion framework for modular quadrupedal robots is proposed. The approach combines a low-level central pattern generator (CPG)-based controller with a high-level neural network to learn a variety of locomotion tasks using DRL. The low-level CPG controller is pre-optimized to generate stable rhythmic walking gaits, while the high-level network is trained to modulate the CPG parameters for achieving task goals based on high-dimensional inputs, including the robot states and user commands. The proposed approach is employed on a simulated modular quadruped. With a limited amount of prior knowledge, the proposed method is demonstrated to be capable of learning a variety of locomotion skills such as velocity tracking, path following, and navigating to a target. Simulation results show that the proposed method can achieve higher sample efficiency than the model-free DRL method and are substantially more robust than the baseline methods to external disturbances and irregular terrain.

Motion Simulation of Flying Quadruped Robot Based on Deep Reinforcement Learning

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Model-free Maneuvering Control of Fixed-Wing UAVs Based on Deep Reinforcement Learning

A Motion Planning and Control Method of Quadruped Robot Based on Deep Reinforcement Learning

Reinforcement Learning based Control of a Quadruped Robot

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks

Human Motion Control of Quadrupedal Robots using Deep Reinforcement Learning

A Hierarchical Framework for Quadruped Robots Gait Planning Based on DDPG

Modeling and Control of Quadruped Robot with Flying Capability

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Deep Reinforcement Learning-based Quadcopter Controller: A Practical Approach and Experiments

Quadrupedal Locomotion in an Energy-efficient Way Based on Reinforcement Learning

Learning Agile Flight Maneuvers: Deep SE(3) Motion Planning and Control for Quadrotors

Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning

Learning the Quadruped Robot by Reinforcement Learning (RL)

Zero-Shot Retargeting of Learned Quadruped Locomotion Policies Using Hybrid Kinodynamic Model Predictive Control

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

Saving the Limping: Fault-tolerant Quadruped Locomotion via Reinforcement Learning

Motion Control of a Hybrid Quadruped-Quadrotor Robot

A Real-Time Planning and Control Framework for Robust and Dynamic Quadrupedal Locomotion

Motion Planning of a Quadrotor Robot Game Using a Simulation-Based Projected Policy Iteration Method