Abstract:Learning locomotion skills is a challenging problem. To generate realistic and smooth locomotion, existing methods use motion capture, finite state machines or morphology-specific knowledge to guide the motion generation algorithms. Deep reinforcement learning (DRL) is a promising approach for the automatic creation of locomotion control. Indeed, a standard benchmark for DRL is to automatically create a running controller for a biped character from a simple reward function [Duan et al. 2016]. Although several different DRL algorithms can successfully create a running controller, the resulting motions usually look nothing like a real runner. This paper takes a minimalist learning approach to the locomotion problem, without the use of motion examples, finite state machines, or morphology-specific knowledge. We introduce two modifications to the DRL approach that, when used together, produce locomotion behaviors that are symmetric, low-energy, and much closer to that of a real person. First, we introduce a new term to the loss function (not the reward function) that encourages symmetric actions. Second, we introduce a new curriculum learning method that provides modulated physical assistance to help the character with left/right balance and forward movement. The algorithm automatically computes appropriate assistance to the character and gradually relaxes this assistance, so that eventually the character learns to move entirely without help. Because our method does not make use of motion capture data, it can be applied to a variety of character morphologies. We demonstrate locomotion controllers for the lower half of a biped, a full humanoid, a quadruped, and a hexapod. Our results show that learned policies are able to produce symmetric, low-energy gaits. In addition, speed-appropriate gait patterns emerge without any guidance from motion examples or contact planning.

Symmetry-Guided Reinforcement Learning for Versatile Quadrupedal Gait Generation

Learning Accurate and Robust Velocity Tracking for Quadrupedal Robots

Breaking Symmetries Leads to Diverse Quadrupedal Gaits

Economical Quadrupedal Multi-Gait Locomotion via Gait-Heuristic Reinforcement Learning

Synthesizing the optimal gait of a quadruped robot with soft actuators using deep reinforcement learning

Behavior evolution-inspired approach to walking gait reinforcement training for quadruped robots

Leveraging Symmetry in RL-based Legged Locomotion Control

Reinforcement Learning With Evolutionary Trajectory Generator: A General Approach for Quadrupedal Locomotion

Automated Gait Generation For Walking, Soft Robotic Quadrupeds

Geometrically Modulable Gait Design for Quadrupeds

AllGaits: Learning All Quadruped Gaits and Transitions

Trot Gait Design and CPG Method for a Quadruped Robot

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

SYNLOCO: Synthesizing Central Pattern Generator and Reinforcement Learning for Quadruped Locomotion

Reliable gait planning for a quadruped walking robot

A Unified Motion Generation Approach for Quadruped L-S Walk and Trot Gaits Based on Linear Model Predictive Control

Harnessing Natural Oscillations for High-Speed, Efficient Asymmetrical Locomotion in Quadrupedal Robots

Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion

ManyQuadrupeds: Learning a Single Locomotion Policy for Diverse Quadruped Robots

A Learning Framework for Diverse Legged Robot Locomotion Using Barrier-Based Style Rewards

Learning symmetric and low-energy locomotion