Abstract:Fusing different sensor modalities can be a difficult task, particularly if they are asynchronous. Asynchronisation may arise due to long processing times or improper synchronisation during calibration, and there must exist a way to still utilise this previous information for the purpose of safe driving, and object detection in ego vehicle/ multi-agent trajectory prediction. Difficulties arise in the fact that the sensor modalities have captured information at different times and also at different positions in space. Therefore, they are not spatially nor temporally aligned. This paper will investigate the challenge of radar and LiDAR sensors being asynchronous relative to the camera sensors, for various time latencies. The spatial alignment will be resolved before lifting into BEV space via the transformation of the radar/LiDAR point clouds into the new ego frame coordinate system. Only after this can we concatenate the radar/LiDAR point cloud and lifted camera features. Temporal alignment will be remedied for radar data only, we will implement a novel method of inferring the future radar point positions using the velocity information. Our approach to resolving the issue of sensor asynchrony yields promising results. We demonstrate velocity information can drastically improve IoU for asynchronous datasets, as for a time latency of 360 milliseconds (ms), IoU improves from 49.54 to 53.63. Additionally, for a time latency of 550ms, the camera+radar (C+R) model outperforms the camera+LiDAR (C+L) model by 0.18 IoU. This is an advancement in utilising the often-neglected radar sensor modality, which is less favoured than LiDAR for autonomous driving purposes.

Vision-Based Lateral State Estimation for Integrated Control of Automated Vehicles Considering Multirate and Unevenly Delayed Measurements

Robust and Efficient Vehicles Motion Estimation with Low-Cost Multi-Camera and Odometer-Gyroscope

Joint Estimation of Position and Attitude for Intelligent Vehicles by Fusing Delayed Visual Measurements

Hierarchical Lateral Control Scheme For Autonomous Vehicle With Uneven Time Delays Induced By Vision Sensors

LVIO-SAM: A Multi-sensor Fusion Odometry via Smoothing and Mapping

Cascaded Motion Estimation for Intelligent Vehicles in GNSS-Denied Scenes

Sensing and Control of a Quadrotor Using a Visual Inertial Fusion Method

Vision‐aided intelligent vehicle sideslip angle estimation based on a dynamic model

Real-Time Optimal States Estimation with Inertial and Delayed Visual Measurements for Unmanned Aerial Vehicles

Real-Time Longitudinal and Lateral State Estimation of Preceding Vehicle Based on Moving Horizon Estimation

MSFusion: Multilayer Sensor Fusion-Based Robust Motion Estimation

Adaptive Estimations of Tyre–road Friction Coefficient and Body’s Sideslip Angle Based on Strong Tracking and Interactive Multiple Model Theories

A New Visual Sensing System for Motion State Estimation of Lateral Localization of Intelligent Vehicles

An adaptive fuzzy-sliding lateral control strategy of automated vehicles based on vision navigation

Design of visual inertial state estimator for autonomous systems via multi-sensor fusion approach

Research on Lane a Compensation Method Based on Multi-Sensor Fusion

Host-Target Vehicle Model-Based Lateral State Estimation for Preceding Target Vehicles Considering Measurement Delay.

Improving Vision-based Self-positioning in Intelligent Transportation Systems via Integrated Lane and Vehicle Detection

Velocity Driven Vision: Asynchronous Sensor Fusion Birds Eye View Models for Autonomous Vehicles

Dual Extended Kalman Filter Based State and Parameter Estimator for Model-Based Control in Autonomous Vehicles

3D object detection and state estimation method based on stereo vision and LIDAR fusion