Abstract:We propose a robust dynamic walking controller consisting of a dynamic locomotion planner, a reinforcement learning process for robustness, and a novel whole-body locomotion controller (WBLC). Previous approaches specify either the position or the timing of steps, however, the proposed locomotion planner simultaneously computes both of these parameters as locomotion outputs. Our locomotion strategy relies on devising a reinforcement learning (RL) approach for robust walking. The learned policy generates multi step walking patterns, and the process is quick enough to be suitable for real-time controls. For learning, we devise an RL strategy that uses a phase space planner (PSP) and a linear inverted pendulum model to make the problem tractable and very fast. Then, the learned policy is used to provide goal-based commands to the WBLC, which calculates the torque commands to be executed in full-humanoid robots. The WBLC combines multiple prioritized tasks and calculates the associated reaction forces based on practical inequality constraints. The novel formulation includes efficient calculation of the time derivatives of various Jacobians. This provides high-fidelity dynamic control of fast motions. More specifically, we compute the time derivative of the Jacobian for various tasks and the Jacobian of the centroidal momentum task by utilizing Lie group operators and operational space dynamics respectively. The integration of RL-PSP and the WBLC provides highly robust, versatile, and practical locomotion including steering while walking and handling push disturbances of up to 520 N during an interval of 0.1 sec. Theoretical and numerical results are tested through a 3D physics-based simulation of the humanoid robot Valkyrie.

A Reinforcement Learning Based Dynamic Walking Control

Reinforcement learning for passive dynamic walking robot

Stable walking control and strategy of under-actuated biped robot

A Zmp and Com Online Replanning Method for Stable Walking of Bipedal Robots

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Reinforcement Learning based Control of a Quadruped Robot

Quadrupedal Locomotion in an Energy-efficient Way Based on Reinforcement Learning

Designing Synergistic Walking of a Humanoid Single Leg Driven by Pneumatic Artificial Muscles: an Empirical Study

A Motion Planning and Control Method of Quadruped Robot Based on Deep Reinforcement Learning

Robust Dynamic Locomotion via Reinforcement Learning and Novel Whole Body Controller

Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots

Passivity-Based Dynamic Bipedal Walking With Terrain Adaptability Dynamics, Control And Robotic Applications

Motion Sequence Learning for Robot Walking Based on Pose optimization

Adaptive Walking of a Quadrupedal Robot Based on Layered Biological Reflexes

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Dynamic Bipedal Maneuvers through Sim-to-Real Reinforcement Learning

Optimal Gait Control for a Tendon-driven Soft Quadruped Robot by Model-based Reinforcement Learning

Reinforcement learning for versatile, dynamic, and robust bipedal locomotion control

Dynamic Walking on Uneven Terrains with Passivity-Based Bipedal Robots

Robust Optimal Control of Point-Feet Biped Robots Using a Reinforcement Learning Approach

Robust biped locomotion using deep reinforcement learning on top of an analytical control approach