Abstract:Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.

Fusion of Object Tracking and Dynamic Occupancy Grid Map

A Fusion Method Aiming at Environmental Perception of Autonomous Vehicle Based on Visual Scheme

Environment Perception Framework Fusing Multi-Object Tracking, Dynamic Occupancy Grid Maps and Digital Maps

Online Multi-Object Tracking from A Bird's-Eye View by Fusion of Millimeter-Wave Radar and Vision

A Fusion of Dynamic Occupancy Grid Mapping and Multi-object Tracking Based on Lidar and Camera Sensors

A Multi-Task Recurrent Neural Network for End-to-End Dynamic Occupancy Grid Mapping

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

Multi-Camera Object Fusion Tracking Model for Autonomous Driving.

Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

Collaborative Grid Mapping for Moving Object Tracking Evaluation

Combined Image- and World-Space Tracking in Traffic Scenes

Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing

Combined Registration and Fusion of Evidential Occupancy Grid Maps for Live Digital Twins of Traffic

Data-Driven Occupancy Grid Mapping using Synthetic and Real-World Data

Real-time Multi-object Tracking Research Based on Image and Point Cloud Fusion

Inverse Perspective Mapping-Based Neural Occupancy Grid Map for Visual Parking.

CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion

Fusion of building information and range imaging for autonomous location estimation in indoor environments

A Dynamic 3D Multi-Object Tracking Method Based on Spatiotemporal Features

A New Multimodal Map Building Method Using Multiple Object Tracking and Gaussian Process Regression