Abstract:Learning locomotion skills is a challenging problem. To generate realistic and smooth locomotion, existing methods use motion capture, finite state machines or morphology-specific knowledge to guide the motion generation algorithms. Deep reinforcement learning (DRL) is a promising approach for the automatic creation of locomotion control. Indeed, a standard benchmark for DRL is to automatically create a running controller for a biped character from a simple reward function [Duan et al. 2016]. Although several different DRL algorithms can successfully create a running controller, the resulting motions usually look nothing like a real runner. This paper takes a minimalist learning approach to the locomotion problem, without the use of motion examples, finite state machines, or morphology-specific knowledge. We introduce two modifications to the DRL approach that, when used together, produce locomotion behaviors that are symmetric, low-energy, and much closer to that of a real person. First, we introduce a new term to the loss function (not the reward function) that encourages symmetric actions. Second, we introduce a new curriculum learning method that provides modulated physical assistance to help the character with left/right balance and forward movement. The algorithm automatically computes appropriate assistance to the character and gradually relaxes this assistance, so that eventually the character learns to move entirely without help. Because our method does not make use of motion capture data, it can be applied to a variety of character morphologies. We demonstrate locomotion controllers for the lower half of a biped, a full humanoid, a quadruped, and a hexapod. Our results show that learned policies are able to produce symmetric, low-energy gaits. In addition, speed-appropriate gait patterns emerge without any guidance from motion examples or contact planning.

DeepWalk: Omnidirectional Bipedal Gait by Deep Reinforcement Learning

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Bipedal Walking Robot using Deep Deterministic Policy Gradient

A Multi-Agent Reinforcement Learning Method for Omnidirectional Walking of Bipedal Robots

Learning Bipedal Walking for Humanoids with Current Feedback

Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey

Dexterous Legged Locomotion in Confined 3D Spaces with Reinforcement Learning

Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control

Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning

Learning Bipedal Walking for Humanoid Robots in Challenging Environments with Obstacle Avoidance

CPG-Based Hierarchical Locomotion Control for Modular Quadrupedal Robots Using Deep Reinforcement Learning.

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Learning 3D Bipedal Walking with Planned Footsteps and Fourier Series Periodic Gait Planning

Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains

Quadruped Robot Locomotion in Unknown Terrain Using Deep Reinforcement Learning

Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Learning symmetric and low-energy locomotion

Intelligent Controller For Passivity-Based Biped Robot Using Deep Q Network

Obstacle-Aware Quadrupedal Locomotion With Resilient Multi-Modal Reinforcement Learning