Abstract:In the context of autonomous driving, sensing systems play a crucial role, and their accuracy and reliability can significantly impact the overall safety of autonomous vehicles. Despite this, fault diagnosis for sensing systems has not received widespread attention, and existing research has limitations. This paper focuses on the unique characteristics of autonomous driving sensing systems and proposes a fault diagnosis method that combines hardware redundancy and analytical redundancy. Firstly, to ensure the authenticity of the study, we define 12 common real-world faults and inject them into the nuScenes dataset, creating an extended dataset. Then, employing heterogeneous hardware redundancy, we fuse MMW radar, LiDAR, and camera data, projecting them into pixel space. We utilize the "ground truth" obtained from the MMW radar to detect faults on the LiDAR and camera data. Finally, we use multidimensional temporal entropy to assess the information complexity fluctuations of LiDAR and the camera during faults. Simultaneously, we construct a CNN-based time-series data multi-classification model to identify fault types. Through experiments, our proposed method achieves 95.33% accuracy in detecting faults and 82.89% accuracy in fault diagnosis on real vehicles. The average response times for fault detection and diagnosis are 0.87 s and 1.36 s, respectively. The results demonstrate that the proposed method can effectively detect and diagnose faults in sensing systems and respond rapidly, providing enhanced reliability for autonomous driving systems.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper primarily focuses on the issue of fault diagnosis in autonomous driving perception systems. Specifically, the paper addresses the following aspects: 1. **Fault Detection and Diagnosis**: - Proposes a hybrid fault diagnosis method that combines hardware redundancy and analytical redundancy, aiming to improve the reliability and safety of autonomous driving perception systems. - Uses MMW radar as a benchmark, integrating LiDAR and camera data, and employs information complexity assessment to detect and diagnose faults. 2. **Delay Issues in Actual Deployment**: - Existing hardware redundancy methods exhibit different delay characteristics in actual vehicle deployment, which may lead to misjudgments in practical applications. - The proposed method can effectively operate in actual vehicle deployment and considers the delay characteristics during the fault detection phase. 3. **Data Analysis Issues in Analytical Redundancy Methods**: - Mainstream analytical redundancy methods typically analyze data collected post-event, which cannot guarantee real-time response speed. - The proposed method achieves real-time fault detection, ensuring high accuracy while enhancing response speed. ### Specific Contributions - **Real-time Fault Detection**: Designed a real-time fault detection method, particularly suitable for autonomous driving perception systems, ensuring high detection accuracy. - **Fault Type Diagnosis Based on Information Complexity**: Utilizes information complexity metrics to diagnose fault types without requiring precise modeling and complex data processing, thereby improving response speed. ### Fault Classification The paper defines 12 common fault types, including interference-level (single-modal) and alignment-level (multi-modal) faults. These fault types include but are not limited to: - Density Decrease - Cutout - LiDAR Crosstalk - FOV Lost - Gaussian Corruption - Uniform Corruption - Impulse Corruption - Spatial Misalignment - Temporal Misalignment ### Dataset Expansion To validate the effectiveness of the proposed fault diagnosis method, the paper extends the nuScenes dataset by adding 12 fault injection scenarios and conducts detailed experimental verification. The results indicate that the method has high accuracy and rapid response capability in fault detection and diagnosis.

A Hybrid Fault Diagnosis Method for Autonomous Driving Sensing Systems Based on Information Complexity

Fault Diagnosis of the Autonomous Driving Perception System Based on Information Fusion

A data-driven hybrid sensor fault detection/diagnosis method with flight test data

Deep Learning Based Data Fusion for Sensor Fault Diagnosis and Tolerance in Autonomous Vehicles

Electronic Systems Diagnosis Fault in Gasoline Engines Based on Multi-Information Fusion

Modular Fault Diagnosis Framework for Complex Autonomous Driving Systems

Multi-Sensor Fault Diagnosis Based on Time Series in an Intelligent Mechanical System

Hydraulic system fault diagnosis decoupling method based on 2D time-series modeling and self-attention fusion

Scene Semantic Anomaly Detection of Multi-Sensor in Autonomous Driving

An Intelligent Diagnostic Method for Multisource Coupling Faults of Complex Mechanical Systems

Cross-modal Matching CNN for Autonomous Driving Sensor Data Monitoring

Hybrid multi-mode machine learning-based fault diagnosis strategies with application to aircraft gas turbine engines

A Hybrid Model-Data Vehicle Sensor and Actuator Fault Detection and Diagnosis System

A Multisensor Information Fusion Method for High-Reliability Fault Diagnosis of Rotating Machinery

Motor fault diagnosis based on multisensor-driven visual information fusion

Monitoring and Fault Diagnosis of Hybrid Systems

A Random Forest and Model-Based Hybrid Method of Fault Diagnosis for Satellite Attitude Control Systems

Sensitivity-Based Fault Detection and Isolation Algorithm for Road Vehicle Chassis Sensors

A Hybrid Approach for Automatic Incident Detection.

Self-Detection and Self-Diagnosis Methods for Sensors in Intelligent Integrated Sensing System