Abstract:Learning locomotion skills is a challenging problem. To generate realistic and smooth locomotion, existing methods use motion capture, finite state machines or morphology-specific knowledge to guide the motion generation algorithms. Deep reinforcement learning (DRL) is a promising approach for the automatic creation of locomotion control. Indeed, a standard benchmark for DRL is to automatically create a running controller for a biped character from a simple reward function [Duan et al. 2016]. Although several different DRL algorithms can successfully create a running controller, the resulting motions usually look nothing like a real runner. This paper takes a minimalist learning approach to the locomotion problem, without the use of motion examples, finite state machines, or morphology-specific knowledge. We introduce two modifications to the DRL approach that, when used together, produce locomotion behaviors that are symmetric, low-energy, and much closer to that of a real person. First, we introduce a new term to the loss function (not the reward function) that encourages symmetric actions. Second, we introduce a new curriculum learning method that provides modulated physical assistance to help the character with left/right balance and forward movement. The algorithm automatically computes appropriate assistance to the character and gradually relaxes this assistance, so that eventually the character learns to move entirely without help. Because our method does not make use of motion capture data, it can be applied to a variety of character morphologies. We demonstrate locomotion controllers for the lower half of a biped, a full humanoid, a quadruped, and a hexapod. Our results show that learned policies are able to produce symmetric, low-energy gaits. In addition, speed-appropriate gait patterns emerge without any guidance from motion examples or contact planning.

Exciting Action: Investigating Efficient Exploration for Learning Musculoskeletal Humanoid Locomotion

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Learning Control Policies for Imitating Human Gaits

Learning Responsive Humanoid Motion Skills from Graph-Powered Motion Matching

Learning Human-like Locomotion Based on Biological Actuation and Rewards

Natural and Robust Walking using Reinforcement Learning without Demonstrations in High-Dimensional Musculoskeletal Models

Learning to Exploit Elastic Actuators for Quadruped Locomotion

HumanMimic: Learning Natural Locomotion and Transitions for Humanoid Robot via Wasserstein Adversarial Imitation

A Biped Robot Learning to Walk like Human by Reinforcement Learning.

Learning symmetric and low-energy locomotion

Learning to Control Emulated Muscles in Real Robots: Towards Exploiting Bio-Inspired Actuator Morphology

Modeling and reinforcement learning-based locomotion control for a humanoid robot with kinematic loop closures

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning Agile Locomotion via Adversarial Training

Inertia-Constrained Reinforcement Learning to Enhance Human Motor Control Modeling

Sample Efficient Optimization for Learning Controllers for Bipedal Locomotion

Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control

Experience-Learning Inspired Two-Step Reward Method for Efficient Legged Locomotion Learning Towards Natural and Robust Gaits

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

DEP-RL: Embodied Exploration for Reinforcement Learning in Overactuated and Musculoskeletal Systems