Abstract:The reliability of autonomous driving sensing systems impacts the overall safety of the driving system. However, perception system fault diagnosis is currently a weak area of research, with limited attention and solutions. In this paper, we present an information-fusion-based fault-diagnosis method for autonomous driving perception systems. To begin, we built an autonomous driving simulation scenario using PreScan software, which collects information from a single millimeter wave (MMW) radar and a single camera sensor. The photos are then identified and labeled via the convolutional neural network (CNN). Then, we fused the sensory inputs from a single MMW radar sensor and a single camera sensor in space and time and mapped the MMW radar points onto the camera image to obtain the region of interest (ROI). Lastly, we developed a method to use information from a single MMW radar to aid in diagnosing defects in a single camera sensor. As the simulation results show, for missing row/column pixel failure, the deviation typically falls between 34.11% and 99.84%, with a response time of 0.02 s to 1.6 s; for pixel shift faults, the deviation range is between 0.32% and 9.92%, with a response time of 0 s to 0.16 s; for target color loss, faults have a deviation range of 0.26% to 2.88% and a response time of 0 s to 0.05 s. These results prove the technology is effective in detecting sensor faults and issuing real-time fault alerts, providing a basis for designing and developing simpler and more user-friendly autonomous driving systems. Furthermore, this method illustrates the principles and methods of information fusion between camera and MMW radar sensors, establishing the foundation for creating more complicated autonomous driving systems.

Asynchronous information fusion in intelligent driving systems for target tracking using cameras and radars

Asynchronous Multi-Radar Tracking Fusion with Converted Measurements

Radar and Camera Fusion for Multi-Task Sensing in Autonomous Driving

Asynchronous Data Fusion of Two Different Sensors

An Asynchronous Data Fusion Algorithm for Target Detection Based on Multi-Sensor Networks

Two Kinds of Sensor Data Fusion in Target Tracking

Target Detection Algorithm Based on MMW Radar and Camera Fusion

Adaptive Track Fusion for Active/Passive Radar Sensor

Tracking with Sequentially Fused Radar and Acoustic Sensor Data with Propagation Delay

Target Tracking Algorithm of Radar and Infrared Sensor Based on Multi-Source Information Fusion

A multi-sensor fusion tracking system based on fuzzy tracker

Robust Target Recognition and Tracking of Self-Driving Cars With Radar and Camera Information Fusion Under Severe Weather Conditions

A New Active and Passive Asynchronous Fusion Tracking of Maneuvering Target

Distributed Fusion with Phd Filter for Multi-Target Tracking in Asynchronous Radar System

Multi-sensor Fusion Detection Method for Vehicle Target Based on Kalman Filter and Data Association Filter

Low-Cost Hybrid Multisensor Fusion Method and Implementation for Production Intelligent Vehicles.

Study of Assisted Driving System Based on Multi-Sensor Information Fusion

Target Tracking and Decision-Making for Collision Avoidance Based on Multisensor Fusion

Target Detection Based on Improved Hausdorff Distance Matching Algorithm for Millimeter-Wave Radar and Video Fusion

Fault Diagnosis of the Autonomous Driving Perception System Based on Information Fusion

A Novel Spatio-Temporal Synchronization Method of Roadside Asynchronous MMW Radar-Camera for Sensor Fusion