Abstract:Recently reinforcement learning (RL) has emerged as a promising approach for quadrupedal locomotion, which can save the manual effort in conventional approaches such as designing skill-specific controllers. However, due to the complex nonlinear dynamics in quadrupedal robots and reward sparsity, it is still difficult for RL to learn effective gaits from scratch, especially in challenging tasks such as walking over the balance beam. To alleviate such difficulty, we propose a novel RL-based approach that contains an evolutionary foot trajectory generator. Unlike prior methods that use a fixed trajectory generator, the generator continually optimizes the shape of the output trajectory for the given task, providing diversified motion priors to guide the policy learning. The policy is trained with reinforcement learning to output residual control signals that fit different gaits. We then optimize the trajectory generator and policy network alternatively to stabilize the training and share the exploratory data to improve sample efficiency. As a result, our approach can solve a range of challenging tasks in simulation by learning from scratch, including walking on a balance beam and crawling through the cave. To further verify the effectiveness of our approach, we deploy the controller learned in the simulation on a 12-DoF quadrupedal robot, and it can successfully traverse challenging scenarios with efficient gaits. We provide a video to show the learned gaits in different tasks in YouTube.11[Online]. Available: youtube.com/watch?vhgBLR09MEOw, and code is available in Github: github.com/PaddlePaddle/PaddleRobotics [Online]. Available: youtube.com/watch?vhgBLR09MEOw, and code is available in Github: github.com/PaddlePaddle/PaddleRobotics

A Hierarchical Framework for Quadruped Locomotion Based on Reinforcement Learning

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Hierarchical Reinforcement Learning for Quadruped Locomotion

A Novel Framework for Adaptive Quadruped Robot Locomotion Learning in Uncertain Environments.

A Hierarchical Framework for Quadruped Robots Gait Planning Based on DDPG

Adaptive Gait Generation for Hexapod Robots Based on Reinforcement Learning and Hierarchical Framework

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

Learning and Reusing Quadruped Robot Movement Skills from Biological Dogs for Higher-Level Tasks

Environmental Features Assessment Network Aided Deep Reinforcement Learning for Quadrupedal Locomotion in Tough Terrain

Quadrupedal Locomotion in an Energy-efficient Way Based on Reinforcement Learning

Agile Control for Quadruped Robot in Complex Environment Based on Deep Reinforcement Learning Method.

Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning

Learning Smooth and Omnidirectional Locomotion for Quadruped Robots

Economical Quadrupedal Multi-Gait Locomotion via Gait-Heuristic Reinforcement Learning

Reinforcement Learning With Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion

Hierarchical Vision Navigation System for Quadruped Robots with Foothold Adaptation Learning

Reinforcement Learning based Control of a Quadruped Robot

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Robust High-speed Running for Quadruped Robots via Deep Reinforcement Learning

A Real-Time Planning and Control Framework for Robust and Dynamic Quadrupedal Locomotion

Hybrid Bipedal Locomotion Based on Reinforcement Learning and Heuristics