Manthan Patel,Jonas Frey,Deegan Atha,Patrick Spieler,Marco Hutter,Shehryar Khattak
Abstract:Autonomous robot navigation in off-road environments requires a comprehensive understanding of the terrain geometry and traversability. The degraded perceptual conditions and sparse geometric information at longer ranges make the problem challenging especially when driving at high speeds. Furthermore, the sensing-to-mapping latency and the look-ahead map range can limit the maximum speed of the vehicle. Building on top of the recent work RoadRunner, in this work, we address the challenge of long-range (100 m) traversability estimation. Our RoadRunner (M&M) is an end-to-end learning-based framework that directly predicts the traversability and elevation maps at multiple ranges (50 m, 100 m) and resolutions (0.2 m, 0.8 m) taking as input multiple images and a LiDAR voxel map. Our method is trained in a self-supervised manner by leveraging the dense supervision signal generated by fusing predictions from an existing traversability estimation stack (X-Racer) in hindsight and satellite Digital Elevation Maps. RoadRunner M&M achieves a significant improvement of up to 50% for elevation mapping and 30% for traversability estimation over RoadRunner, and is able to predict in 30% more regions compared to X-Racer while achieving real-time performance. Experiments on various out-of-distribution datasets also demonstrate that our data-driven approach starts to generalize to novel unstructured environments. We integrate our proposed framework in closed-loop with the path planner to demonstrate autonomous high-speed off-road robotic navigation in challenging real-world environments. Project Page: <a class="link-external link-https" href="https://leggedrobotics.github.io/roadrunner_mm/" rel="external noopener nofollow">this https URL</a>
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the multi - range and multi - resolution prediction of terrain traversability and elevation required for high - speed autonomous robot navigation in unstructured off - road environments. Specifically, the paper focuses on how to accurately predict the traversability and elevation maps of the terrain at low latency within a long - distance (±100 meters) range to support the high - speed autonomous navigation of robots in complex environments.
### Background and Challenges
- **Specialties of Off - road Environments**: Different from urban environments, there is no clear road definition in off - road environments, so it is difficult to distinguish between drivable and undrivable areas.
- **Limitations of Perception Conditions**: The perception conditions at long distances are poor and the geometric information is sparse, which makes heuristic - based methods difficult to be effectively applied.
- **Requirements for Real - time Performance**: In order to support high - speed navigation, the system needs to provide accurate traversability predictions at low latency, covering potential dangerous areas.
- **Limitations of Existing Methods**: Existing methods such as RoadRunner have achieved certain results, but still have deficiencies in prediction range, time consistency, and generalization ability to new environments.
### Solutions
The paper proposes an end - to - end learning framework named RoadRunner M&M, which can directly predict traversability and elevation maps with multiple ranges (±50 meters and ±100 meters) and multiple resolutions (0.2 meters and 0.8 meters) from multiple images and LiDAR voxel maps. The main contributions include:
1. **Multi - range and Multi - resolution Prediction**: By introducing a novel multi - range and multi - resolution hierarchical decoder, it can simultaneously predict elevation maps and traversability maps at multiple ranges and resolutions.
2. **Self - supervised Training**: Utilize the dense supervision signals generated by the existing traversability estimation stack (X - Racer) and satellite digital elevation maps for self - supervised training, which significantly improves performance and reduces latency.
3. **Extensive Experimental Verification**: Verified on a variety of real - world datasets, the results show that the elevation mapping and traversability estimation are respectively improved by 50% and 30%, and more areas can be predicted.
4. **Zero - shot Deployment**: Demonstrates good generalization ability on ecologically different unseen datasets, including environments such as deserts, beaches, canyons, and dense forests.
5. **Practical Applications**: Integrate the proposed framework into the path planner, demonstrating the high - speed autonomous navigation ability in complex real - world environments.
### Method Overview
- **Input Data**: Four RGB images and one LiDAR voxel map.
- **Network Architecture**: Use the Lift Splat method to process image data, the PointPillars method to process LiDAR voxel map data, and then fuse the multi - modal features and generate prediction maps through the hierarchical decoder.
- **Loss Function**: Combine the mean squared error (MSE) loss and the Smooth - L1 loss, and weight different regions (observed, future - observed, and unobserved) to improve the training effect.
### Experimental Results
- **Elevation Mapping**: In the micro - range (±50 meters) and short - range (±100 meters), compared with other methods, RoadRunner M&M has a significant improvement in the accuracy of all regions, especially the accuracy in the micro - range is improved by about 50%.
- **Traversability Estimation**: In terms of risk mean squared error (MSE), precision, recall, and F1 - score, RoadRunner M&M performs excellently, especially in the prediction coverage and accuracy of unobserved regions, which are significantly better than other methods.
### Conclusion
The method RoadRunner M&M proposed in the paper has made significant improvements in multi - range and multi - resolution terrain traversability and elevation prediction, can support high - speed off - road autonomous navigation at low latency, and has demonstrated good generalization ability in a variety of complex environments.