Advancing Humanoid Locomotion: Mastering Challenging Terrains with Denoising World Model Learning

Xinyang Gu,Yen-Jen Wang,Xiang Zhu,Chengming Shi,Yanjiang Guo,Yichen Liu,Jianyu Chen
2024-08-27
Abstract:Humanoid robots, with their human-like skeletal structure, are especially suited for tasks in human-centric environments. However, this structure is accompanied by additional challenges in locomotion controller design, especially in complex real-world environments. As a result, existing humanoid robots are limited to relatively simple terrains, either with model-based control or model-free reinforcement learning. In this work, we introduce Denoising World Model Learning (DWL), an end-to-end reinforcement learning framework for humanoid locomotion control, which demonstrates the world's first humanoid robot to master real-world challenging terrains such as snowy and inclined land in the wild, up and down stairs, and extremely uneven terrains. All scenarios run the same learned neural network with zero-shot sim-to-real transfer, indicating the superior robustness and generalization capability of the proposed method.
Robotics,Artificial Intelligence,Systems and Control
What problem does this paper attempt to address?
### Problems the Paper Aims to Solve This paper aims to address the walking control problem of humanoid robots in complex real-world environments. Specifically, existing humanoid robots have limited walking capabilities on complex terrains and can usually only handle relatively simple terrains. These limitations are mainly due to: 1. **Model-based control methods**: Methods such as Zero Moment Point (ZMP) and Model Predictive Control (MPC) combined with Whole-Body Control (WBC), although performing well in certain tasks, rely on precise environmental dynamics modeling, making it difficult to handle complex environmental interactions, such as walking on rugged and uneven terrains. 2. **Model-free reinforcement learning methods**: Despite the enormous potential shown by model-free reinforcement learning in developing adaptive legged motion controllers in recent years, its application to humanoid robots is still limited to simple terrains. This is because humanoid robots face more challenges compared to quadruped or biped robots, such as a higher center of gravity, instability during leg swings, greater leg inertia, and additional torso and arm weight. To overcome these challenges, the paper introduces a new end-to-end reinforcement learning framework—Denoising World Model Learning (DWL). This framework reduces the gap between simulation and reality through representation learning, achieving robustness and generalization capabilities for humanoid robots in complex real-world environments. Specifically, the paper demonstrates the world's first humanoid robot capable of mastering complex real-world terrains (such as snowy grounds, slopes, stairs, and extremely uneven terrains) through zero-shot sim-to-real transfer.