Abstract:Understanding terrain topology at long-range is crucial for the success of off-road robotic missions, especially when navigating at high-speeds. LiDAR sensors, which are currently heavily relied upon for geometric mapping, provide sparse measurements when mapping at greater distances. To address this challenge, we present a novel learning-based approach capable of predicting terrain elevation maps at long-range using only onboard egocentric images in real-time. Our proposed method is comprised of three main elements. First, a transformer-based encoder is introduced that learns cross-view associations between the egocentric views and prior bird-eye-view elevation map predictions. Second, an orientation-aware positional encoding is proposed to incorporate the 3D vehicle pose information over complex unstructured terrain with multi-view visual image features. Lastly, a history-augmented learn-able map embedding is proposed to achieve better temporal consistency between elevation map predictions to facilitate the downstream navigational tasks. We experimentally validate the applicability of our proposed approach for autonomous offroad robotic navigation in complex and unstructured terrain using real-world offroad driving data. Furthermore, the method is qualitatively and quantitatively compared against the current state-of-the-art methods. Extensive field experiments demonstrate that our method surpasses baseline models in accurately predicting terrain elevation while effectively capturing the overall terrain topology at long-ranges. Finally, ablation studies are conducted to highlight and understand the effect of key components of the proposed approach and validate their suitability to improve offroad robotic navigation capabilities.

Terrain traversability analysis using multi-sensor data correlation by a mobile robot

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Visual Traversability Analysis for Micro Planetary Rover

Adaptive Model Prediction Control-Based Multi-Terrain Trajectory Tracking Framework for Mobile Spherical Robots

Audio-Visual Self-Supervised Terrain Type Discovery for Mobile Platforms

Vibration-Based Terrain Identification for Planetary Exploration Robots Using Support Vector Machine

Scene understanding for a high-mobility walking robot

Pixel to Elevation: Learning to Predict Elevation Maps at Long Range using Images for Autonomous Offroad Navigation

Vision-Assisted Localization and Terrain Reconstruction with Quadruped Robots

UAV-Assisted Self-Supervised Terrain Awareness for Off-Road Navigation

Hyperspectral Imaging for Mobile Robot Navigation

Follow the Footprints: Self-supervised Traversability Estimation for Off-road Vehicle Navigation based on Geometric and Visual Cues

Adaptive ground segmentation method for real-time mobile robot control

Outdoor scene understanding of mobile robot via multi-sensor information fusion

Predictive Mapping of Spectral Signatures from RGB Imagery for Off-Road Terrain Analysis

Deep Learning Traversability Estimator for Mobile Robots in Unstructured Environments

Towards Safer Planetary Exploration: A Hybrid Architecture for Terrain Traversability Analysis in Mars Rovers

Uncertainty-Based Vibration/Gyro Composite Planetary Terrain Mapping

Terrain Analytics for Precision Agriculture with Automated Vehicle Sensors and Data Fusion

Terrain Vision Aided Online Estimation of Instantaneous Centers of Rotation for Skid-Steering Mobile Robots

RoadRunner M&M -- Learning Multi-range Multi-resolution Traversability Maps for Autonomous Off-road Navigation