Abstract:We propose a robust dynamic walking controller consisting of a dynamic locomotion planner, a reinforcement learning process for robustness, and a novel whole-body locomotion controller (WBLC). Previous approaches specify either the position or the timing of steps, however, the proposed locomotion planner simultaneously computes both of these parameters as locomotion outputs. Our locomotion strategy relies on devising a reinforcement learning (RL) approach for robust walking. The learned policy generates multi step walking patterns, and the process is quick enough to be suitable for real-time controls. For learning, we devise an RL strategy that uses a phase space planner (PSP) and a linear inverted pendulum model to make the problem tractable and very fast. Then, the learned policy is used to provide goal-based commands to the WBLC, which calculates the torque commands to be executed in full-humanoid robots. The WBLC combines multiple prioritized tasks and calculates the associated reaction forces based on practical inequality constraints. The novel formulation includes efficient calculation of the time derivatives of various Jacobians. This provides high-fidelity dynamic control of fast motions. More specifically, we compute the time derivative of the Jacobian for various tasks and the Jacobian of the centroidal momentum task by utilizing Lie group operators and operational space dynamics respectively. The integration of RL-PSP and the WBLC provides highly robust, versatile, and practical locomotion including steering while walking and handling push disturbances of up to 520 N during an interval of 0.1 sec. Theoretical and numerical results are tested through a 3D physics-based simulation of the humanoid robot Valkyrie.

Revisiting Reward Design and Evaluation for Robust Humanoid Standing and Walking

A Biped Robot Learning to Walk like Human by Reinforcement Learning.

Benchmarking Potential Based Rewards for Learning Humanoid Locomotion

HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulation

Deriving Rewards for Reinforcement Learning from Symbolic Behaviour Descriptions of Bipedal Walking

A Heuristics-Based Reinforcement Learning Method to Control Bipedal Robots

Robust Dynamic Locomotion via Reinforcement Learning and Novel Whole Body Controller

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

Teach Biped Robots to Walk via Gait Principles and Reinforcement Learning with Adversarial Critics

Benchmarking the Full-Order Model Optimization Based Imitation in the Humanoid Robot Reinforcement Learning Walk

Learning Bipedal Walking for Humanoids with Current Feedback

Adaptive Energy Regularization for Autonomous Gait Transition and Energy-Efficient Quadruped Locomotion

Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion

Hybrid Zero Dynamics Inspired Feedback Control Policy Design for 3D Bipedal Locomotion using Reinforcement Learning

Creation and Evaluation of Human Models with Varied Walking Ability from Motion Capture for Assistive Device Development

Video2Reward: Generating Reward Function from Videos for Legged Robot Behavior Learning

A Disturbance Rejection Control Method Based on Deep Reinforcement Learning for a Biped Robot

Development of a New Robust Stable Walking Algorithm for a Humanoid Robot Using Deep Reinforcement Learning with Multi-Sensor Data Fusion

A Multi-Stage Approach for Efficiently Learning Humanoid Robot Stand-Up Behavior

Research of Reinforcement Learning Based Share Control of Walking-Aid Robot

Robust Walking and Sim-to-Real Optimization for Quadruped Robots via Reinforcement Learning