Abstract:Sensor data fusion is essential for environmental perception within smart traffic applications. By using multiple sensors cooperatively, the accuracy and probability of the perception are increased, which is crucial for critical traffic scenarios or under bad weather conditions. In this paper, a modular real-time capable multi-sensor fusion framework is presented and tested to fuse data on the object list level from distributed automotive sensors (cameras, radar, and LiDAR). The modular multi-sensor fusion architecture receives an object list (untracked objects) from each sensor. The fusion framework combines classical data fusion algorithms, as it contains a coordinate transformation module, an object association module (Hungarian algorithm), an object tracking module (unscented Kalman filter), and a movement compensation module. Due to the modular design, the fusion framework is adaptable and does not rely on the number of sensors or their types. Moreover, the method continues to operate because of this adaptable design in case of an individual sensor failure. This is an essential feature for safety-critical applications. The architecture targets environmental perception in challenging time-critical applications. The developed fusion framework is tested using simulation and public domain experimental data. Using the developed framework, sensor fusion is obtained well below 10 milliseconds of computing time using an AMD Ryzen 7 5800H mobile processor and the Python programming language. Furthermore, the object-level multi-sensor approach enables the detection of changes in the extrinsic calibration of the sensors and potential sensor failures. A concept was developed to use the multi-sensor framework to identify sensor malfunctions. This feature will become extremely important in ensuring the functional safety of the sensors for autonomous driving.

FusionTalk: An IoT-Based Reconfigurable Object Identification System

Multisensor Data Fusion for Robot Parts Recognition

A Multi-sensor Data Fusion Method Based on the Set-Pair Identity Degree and Connection Number

Iris-Face Fusion and Security Analysis Based on Fisher Discriminant

Multi-sensor data fusion for an efficient object tracking in Internet of Things (IoT)

A semantics-based approach to multi-source heterogeneous information fusion in the internet of things

IoT Solutions with Multi-Sensor Fusion and Signal-Image Encoding for Secure Data Transfer and Decision Making

Deepfusion: A Deep Learning Framework For The Fusion Of Heterogeneous Sensory Data

City Data Fusion: Sensor Data Fusion in the Internet of Things

Intelligent Transportation Application and Analysis for Multi-Sensor Information Fusion of Internet of Things

Target Recognition Algorithm Based on Optical Sensor Data Fusion

Multi-Classifier Fusion for Open-Set Specific Emitter Identification

Research on Multi-source Heterogeneous Sensor Information Fusion Method Under Internet of Things Technology

Image-fusion-based object detection using a time-of-flight camera

Thinger.io: An Open Source Platform for Deploying Data Fusion Applications in IoT Environments

Multi-Sensor Data Fusion for Real-Time Multi-Object Tracking

IFTR: An Instance-Level Fusion Transformer for Visual Collaborative Perception

A survey of identity recognition via data fusion and feature learning

An image fusion algorithm based on image clustering theory

A Multimodal Fusion System for Object Identification in Point Clouds with Density and Coverage Differences

Big Data Fusion in Internet of Things.