Abstract:Humanoid robots can, in principle, use their legs to go almost anywhere. Developing controllers capable of traversing diverse terrains, however, remains a considerable challenge. Classical controllers are hard to generalize broadly while the learning-based methods have primarily focused on gentle terrains. Here, we present a learning-based approach for blind humanoid locomotion capable of traversing challenging natural and man-made terrain. Our method uses a transformer model to predict the next action based on the history of proprioceptive observations and actions. The model is first pre-trained on a dataset of flat-ground trajectories with sequence modeling, and then fine-tuned on uneven terrain using reinforcement learning. We evaluate our model on a real humanoid robot across a variety of terrains, including rough, deformable, and sloped surfaces. The model demonstrates robust performance, in-context adaptation, and emergent terrain representations. In real-world case studies, our humanoid robot successfully traversed over 4 miles of hiking trails in Berkeley and climbed some of the steepest streets in San Francisco.

What problem does this paper attempt to address?

The problem this paper attempts to address is how to enable humanoid robots to walk robustly on complex terrains. Specifically, the authors propose a learning-based approach that allows humanoid robots to walk on various challenging natural and artificial terrains, including steep, rugged, muddy, and sandy surfaces. Traditional controllers struggle to generalize broadly, while existing learning-based methods mainly focus on flat terrains. Therefore, the goal of this paper is to develop a humanoid robot controller capable of handling complex terrains. ### Main Issues: 1. **Limitations of Traditional Controllers**: Traditional controllers struggle to generalize across various environments, especially on complex terrains. 2. **Limitations of Existing Learning Methods**: Existing learning-based methods mainly focus on flat terrains and are ineffective on complex terrains. ### Solution: The authors propose a two-stage training method: 1. **Pre-training**: Using sequence modeling to pre-train the model on a flat terrain dataset, enabling the model to predict the next action from historical proprioceptive observations and actions. 2. **Fine-tuning**: Using reinforcement learning to fine-tune the model on uneven terrains to adapt to complex terrains. ### Experimental Results: - **Real-world Testing**: The model was tested on a real-world humanoid robot, successfully walking on various complex terrains, including steep mountain paths and the steepest streets of San Francisco. - **Performance Evaluation**: Compared to existing state-of-the-art learning methods, the proposed model performed better on various terrains, especially when dealing with obstacles and inclines. ### Conclusion: The paper demonstrates that through a general learning approach, humanoid robots can walk robustly on various complex terrains. This method not only improves the adaptability of robots but also provides new possibilities for the practical application of humanoid robots in the future.

Learning Humanoid Locomotion over Challenging Terrain

Learning Gait-conditioned Bipedal Locomotion with Motor Adaptation

Real-World Humanoid Locomotion with Reinforcement Learning

Learning Generic and Dynamic Locomotion of Humanoids Across Discrete Terrains

Learning Vision-Based Bipedal Locomotion for Challenging Terrain

Online Learning of Uneven Terrain for Humanoid Bipedal Walking

Learning Bipedal Walking On Planned Footsteps For Humanoid Robots

Perceptive Locomotion of Legged Robot Coupling Model Predictive Control and Terrain Mapping

Berkeley Humanoid: A Research Platform for Learning-based Control

Learning and Adapting Agile Locomotion Skills by Transferring Experience

Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers

Learning Bipedal Robot Locomotion from Human Movement

RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control

Learning Terrain-Adaptive Locomotion with Agile Behaviors by Imitating Animals

Learning Robust, Agile, Natural Legged Locomotion Skills in the Wild

Learning Visuo-Motor Behaviours for Robot Locomotion Over Difficult Terrain

MTAC: Hierarchical Reinforcement Learning-based Multi-gait Terrain-adaptive Quadruped Controller

Learning Humanoid Locomotion with Perceptive Internal Model

Learning Fast and Agile Quadrupedal Locomotion over Complex Terrain

Learning to walk in confined spaces using 3D representation