Abstract:Recently reinforcement learning (RL) has emerged as a promising approach for quadrupedal locomotion, which can save the manual effort in conventional approaches such as designing skill-specific controllers. However, due to the complex nonlinear dynamics in quadrupedal robots and reward sparsity, it is still difficult for RL to learn effective gaits from scratch, especially in challenging tasks such as walking over the balance beam. To alleviate such difficulty, we propose a novel RL-based approach that contains an evolutionary foot trajectory generator. Unlike prior methods that use a fixed trajectory generator, the generator continually optimizes the shape of the output trajectory for the given task, providing diversified motion priors to guide the policy learning. The policy is trained with reinforcement learning to output residual control signals that fit different gaits. We then optimize the trajectory generator and policy network alternatively to stabilize the training and share the exploratory data to improve sample efficiency. As a result, our approach can solve a range of challenging tasks in simulation by learning from scratch, including walking on a balance beam and crawling through the cave. To further verify the effectiveness of our approach, we deploy the controller learned in the simulation on a 12-DoF quadrupedal robot, and it can successfully traverse challenging scenarios with efficient gaits. We provide a video to show the learned gaits in different tasks in YouTube.11[Online]. Available: youtube.com/watch?vhgBLR09MEOw, and code is available in Github: github.com/PaddlePaddle/PaddleRobotics [Online]. Available: youtube.com/watch?vhgBLR09MEOw, and code is available in Github: github.com/PaddlePaddle/PaddleRobotics

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Skill Latent Space Based Multigait Learning for a Legged Robot

Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion

Learning Agile, Robust Locomotion Skills for Quadruped Robot.

Learning Multiple Gaits within Latent Space for Quadruped Robots

Economical Quadrupedal Multi-Gait Locomotion via Gait-Heuristic Reinforcement Learning

Terrain-Aware Quadrupedal Locomotion via Reinforcement Learning

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Bio-Inspired Rhythmic Locomotion for Quadruped Robots

Deep Reinforcement Learning Based Co-Optimization of Morphology and Gait for Small-Scale Legged Robot

Behavior evolution-inspired approach to walking gait reinforcement training for quadruped robots

Adaptive Gait Acquisition through Learning Dynamic Stimulus Instinct of Bipedal Robot

Lifelike Agility and Play in Quadrupedal Robots using Reinforcement Learning and Generative Pre-trained Models

Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior

Animal-Like Eye Vision Assisted Locomotion of a Quadruped Based on Reinforcement Learning.

Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animals

Learning Agile Locomotion on Risky Terrains

Reinforcement Learning With Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion

Learning Quadrupedal Locomotion on Tough Terrain Using an Asymmetric Terrain Feature Mining Network