Abstract:Four-dimensional (4D) radar--visual odometry (4DRVO) integrates complementary information from 4D radar and cameras, making it an attractive solution for achieving accurate and robust pose estimation. However, 4DRVO may exhibit significant tracking errors owing to three main factors: 1) sparsity of 4D radar point clouds; 2) inaccurate data association and insufficient feature interaction between the 4D radar and camera; and 3) disturbances caused by dynamic objects in the environment, affecting odometry estimation. In this paper, we present 4DRVO-Net, which is a method for 4D radar--visual odometry. This method leverages the feature pyramid, pose warping, and cost volume (PWC) network architecture to progressively estimate and refine poses. Specifically, we propose a multi-scale feature extraction network called Radar-PointNet++ that fully considers rich 4D radar point information, enabling fine-grained learning for sparse 4D radar point clouds. To effectively integrate the two modalities, we design an adaptive 4D radar--camera fusion module (A-RCFM) that automatically selects image features based on 4D radar point features, facilitating multi-scale cross-modal feature interaction and adaptive multi-modal feature fusion. In addition, we introduce a velocity-guided point-confidence estimation module to measure local motion patterns, reduce the influence of dynamic objects and outliers, and provide continuous updates during pose refinement. We demonstrate the excellent performance of our method and the effectiveness of each module design on both the VoD and in-house datasets. Our method outperforms all learning-based and geometry-based methods for most sequences in the VoD dataset. Furthermore, it has exhibited promising performance that closely approaches that of the 64-line LiDAR odometry results of A-LOAM without mapping optimization.

Efficient Deep-Learning 4D Automotive Radar Odometry Method

Deep 4D Automotive Radar-Camera Fusion Odometry with Cross-Modal Transformer Fusion

4DRVO-Net: Deep 4D Radar-Visual Odometry Using Multi-Modal and Multi-Scale Adaptive Fusion

RaLL: End-to-end Radar Localization on Lidar Map Using Differentiable Measurement Model

Robust Radar Inertial Odometry in Dynamic 3D Environments

Radar Style Transfer for Metric Robot Localisation on Lidar Maps

Multi-Radar Inertial Odometry for 3D State Estimation using mmWave Imaging Radar

Efficient LiDAR Odometry for Autonomous Driving.

DGRO: Doppler Velocity and Gyroscope-Aided Radar Odometry

RadarOcc: Robust 3D Occupancy Prediction with 4D Imaging Radar

LiDAR Odometry by Deep Learning-Based Feature Points with Two-Step Pose Estimation

Loosely coupled 4D-Radar-Inertial Odometry for Ground Robots

EFEAR-4D: Ego-Velocity Filtering for Efficient and Accurate 4D radar Odometry

Radar Odometry Combining Probabilistic Estimation and Unsupervised Feature Learning

4DRadarSLAM: A 4D Imaging Radar SLAM System for Large-scale Environments based on Pose Graph Optimization

Efficient 3D Deep LiDAR Odometry

Deep Radar Inverse Sensor Models for Dynamic Occupancy Grid Maps

4D Radar-Inertial SLAM Based on Factor Graph Optimization

Self-Supervised Scene Flow Estimation with 4-D Automotive Radar