Abstract:Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.

Adaptable sensor fusion based on multiple Kalman filters

Multi-sensor Fusion Detection Method for Vehicle Target Based on Kalman Filter and Data Association Filter

Multi-sensor data fusion method based on adaptive Kalman filtering

Multi-sensor Data Fusion Using Adaptive Kalman Filter

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

Research on Multi-source Information Fusion based on Extended Kalman Filter

A Multi-Mems Sensors Information Fusion Algorithm

Research on multi-target tracking method based on multi-sensor fusion

Multi-sensor Hybrid Fusion Algorithm Based on Adaptive Square-Root Cubature Kalman Filter

Sensor Fusion-Based Vehicle Detection and Tracking Using a Single Camera and Radar at a Traffic Intersection

Anti-delay Kalman filter fusion algorithm for inter-vehicle sensor network with finite-step convergence

Study on Multi-Heterogeneous Sensor Data Fusion Method Based on Millimeter-Wave Radar and Camera

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

Anti-Delay Kalman Filter Fusion Algorithm for Vehicle-borne Sensor Network with Finite-Time Convergence

Multi-Sensor Optimal Data Fusion Based on the Adaptive Fading Unscented Kalman Filter

Fuzzy Adaptive Multi-Sensor Fusion Algorithm for the Vessel Heading Redundant Measurement System

Optimal vehicle position estimation using adaptive unscented Kalman filter based on sensor fusion

MSFusion: Multilayer Sensor Fusion-Based Robust Motion Estimation

Collision Risk Prediction for Vehicles with Sensor Data Fusion Through a Machine Learning Pipeline

Adaptive Unscented Kalman Filter for Target Tacking with Time-Varying Noise Covariance Based on Multi-Sensor Information Fusion