Abstract:We propose a robust dynamic walking controller consisting of a dynamic locomotion planner, a reinforcement learning process for robustness, and a novel whole-body locomotion controller (WBLC). Previous approaches specify either the position or the timing of steps, however, the proposed locomotion planner simultaneously computes both of these parameters as locomotion outputs. Our locomotion strategy relies on devising a reinforcement learning (RL) approach for robust walking. The learned policy generates multi step walking patterns, and the process is quick enough to be suitable for real-time controls. For learning, we devise an RL strategy that uses a phase space planner (PSP) and a linear inverted pendulum model to make the problem tractable and very fast. Then, the learned policy is used to provide goal-based commands to the WBLC, which calculates the torque commands to be executed in full-humanoid robots. The WBLC combines multiple prioritized tasks and calculates the associated reaction forces based on practical inequality constraints. The novel formulation includes efficient calculation of the time derivatives of various Jacobians. This provides high-fidelity dynamic control of fast motions. More specifically, we compute the time derivative of the Jacobian for various tasks and the Jacobian of the centroidal momentum task by utilizing Lie group operators and operational space dynamics respectively. The integration of RL-PSP and the WBLC provides highly robust, versatile, and practical locomotion including steering while walking and handling push disturbances of up to 520 N during an interval of 0.1 sec. Theoretical and numerical results are tested through a 3D physics-based simulation of the humanoid robot Valkyrie.

Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Control

Dynamical Obstacle Avoidance of Task- Constrained Mobile Manipulation Using Model Predictive Control

Expressive Whole-Body Control for Humanoid Robots

Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion

Design and Analysis of a Whole-Body Controller for a Velocity Controlled Robot Mobile Manipulator

Human–Robot Shared Control Based on Locally Weighted Intent Prediction for a Teleoperated Hydraulic Manipulator System

ExBody2: Advanced Expressive Humanoid Whole-Body Control

Visual Whole-Body Control for Legged Loco-Manipulation

Robust Real-Time Upper Limb Motion Transfer from Human to Humanoid for Manipulation Skill Teaching

A Hierarchical Reinforcement Learning Approach to Control Legged Mobile Manipulators

Opt2Skill: Imitating Dynamically-feasible Whole-Body Trajectories for Versatile Humanoid Loco-Manipulation

An Efficient Representation of Whole-body Model Predictive Control for Online Compliant Dual-arm Mobile Manipulation

Nonlinear MPC Based Whole-body Control for Mobile Manipulation

A Combined Learning and Optimization Framework to Transfer Human Whole-body Loco-manipulation Skills to Mobile Manipulators

Contact-Implicit Model Predictive Control for Dexterous In-hand Manipulation: A Long-Horizon and Robust Approach

Robust Dynamic Locomotion via Reinforcement Learning and Novel Whole Body Controller

Plan-Guided Reinforcement Learning for Whole-Body Manipulation

Learning Multi-Modal Whole-Body Control for Real-World Humanoid Robots

Predictive Reachability for Embodiment Selection in Mobile Manipulation Behaviors

Real-time Motion Planning for Robotic Teleoperation Using Dynamic-goal Deep Reinforcement Learning

I-CTRL: Imitation to Control Humanoid Robots Through Constrained Reinforcement Learning