Abstract:Four-dimensional (4D) radar--visual odometry (4DRVO) integrates complementary information from 4D radar and cameras, making it an attractive solution for achieving accurate and robust pose estimation. However, 4DRVO may exhibit significant tracking errors owing to three main factors: 1) sparsity of 4D radar point clouds; 2) inaccurate data association and insufficient feature interaction between the 4D radar and camera; and 3) disturbances caused by dynamic objects in the environment, affecting odometry estimation. In this paper, we present 4DRVO-Net, which is a method for 4D radar--visual odometry. This method leverages the feature pyramid, pose warping, and cost volume (PWC) network architecture to progressively estimate and refine poses. Specifically, we propose a multi-scale feature extraction network called Radar-PointNet++ that fully considers rich 4D radar point information, enabling fine-grained learning for sparse 4D radar point clouds. To effectively integrate the two modalities, we design an adaptive 4D radar--camera fusion module (A-RCFM) that automatically selects image features based on 4D radar point features, facilitating multi-scale cross-modal feature interaction and adaptive multi-modal feature fusion. In addition, we introduce a velocity-guided point-confidence estimation module to measure local motion patterns, reduce the influence of dynamic objects and outliers, and provide continuous updates during pose refinement. We demonstrate the excellent performance of our method and the effectiveness of each module design on both the VoD and in-house datasets. Our method outperforms all learning-based and geometry-based methods for most sequences in the VoD dataset. Furthermore, it has exhibited promising performance that closely approaches that of the 64-line LiDAR odometry results of A-LOAM without mapping optimization.

Robustness Improvement of Using Pre-Trained Network in Visual Odometry for On-Road Driving

Deterministic Optimality for Robust Vehicle Localization Using Visual Measurements

Learning-based Image Enhancement for Visual Odometry in Challenging HDR Environments

DeepAVO: Efficient Pose Refining with Feature Distilling for Deep Visual Odometry

RNIN-VIO: Robust Neural Inertial Navigation Aided Visual-Inertial Odometry in Challenging Scenes

End-to-end, sequence-to-sequence probabilistic visual odometry through deep neural networks

Spatio-temporal and geometry constrained network for automobile visual odometry

Robust self-supervised monocular visual odometry based on prediction-update pose estimation network.

Deep Direct Visual Odometry

4DRVO-Net: Deep 4D Radar-Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusion

Salient Sparse Visual Odometry With Pose-Only Supervision

Road-Network-Map-Assisted Vehicle Positioning Based on Pose Graph Optimization

Vision-based Navigation of Autonomous Vehicle in Roadway Environments with Unexpected Hazards

Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustment

Robustness Benchmark of Road User Trajectory Prediction Models for Automated Driving

DF-VO: What Should Be Learnt for Visual Odometry?

Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World

VisionNet: A Drivable-space-based Interactive Motion Prediction Network for Autonomous Driving

Unsupervised Monocular Visual-Inertial Odometry Network

Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning

Pose Refinement: Bridging the Gap Between Unsupervised Learning and Geometric Methods for Visual Odometry.