Abstract:Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.

Multi-Sensors Data Tracking Fusion Based on a Neural Network Filter

Target Tracking Based On A Distributed Neural Network Filtering Fusion

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

Research on multi-target tracking method based on multi-sensor fusion

Detection, tracking and recognition of multi-targets using fusion data from different sensors

Adaptive Multi-Pedestrian Tracking by Multi-Sensor: Track-to-Track Fusion Using Monocular 3D Detection and MMW Radar

Multi-Sensor Fusion Tracking Algorithm Based on Augmented Reality System

Multi-sensor Fusion Detection Method for Vehicle Target Based on Kalman Filter and Data Association Filter

Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

An Unscented Kalm an Filter-Based MultisensorTrack Fusion Algorithm

An Improved Unscented Particle Filter Approach for Multi-Sensor Fusion Target Tracking

Multi-Target Tracking AA Fusion Method for Asynchronous Multi-Sensor Networks

Joint State and Parameter Estimation for Multisensor Nonlinear Dynamic Systems on the Basis of Strong Tracking Filter

Target Tracking System for Multi-Sensor Data Fusion

Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing

A Multi-sensor Fusion Algorithm with Feedback Based on Fuzzy C-Means and Maximum Entropy Principle

A Multi-Level Eigenvalue Fusion Algorithm for 3D Multi-Object Tracking

Multi-Sensor Optimal Data Fusion Based on the Adaptive Fading Unscented Kalman Filter

MSFusion: Multilayer Sensor Fusion-Based Robust Motion Estimation

A Multi-Active and Multi-Passive Sensor Fusion Algorithm for Multi-Target Tracking in Dense Group Clutter Environments

Bias fusion estimation for multi-target tracking systems with multiple asynchronous sensors