Abstract:This paper explores the role and challenges of Artificial Intelligence (AI) algorithms, specifically AI-based software elements, in autonomous driving systems. These AI systems are fundamental in executing real-time critical functions in complex and high-dimensional environments. They handle vital tasks like multi-modal perception, cognition, and decision-making tasks such as motion planning, lane keeping, and emergency braking. A primary concern relates to the ability (and necessity) of AI models to generalize beyond their initial training data. This generalization issue becomes evident in real-time scenarios, where models frequently encounter inputs not represented in their training or validation data. In such cases, AI systems must still function effectively despite facing distributional or domain shifts. This paper investigates the risk associated with overconfident AI models in safety-critical applications like autonomous driving. To mitigate these risks, methods for training AI models that help maintain performance without overconfidence are proposed. This involves implementing certainty reporting architectures and ensuring diverse training data. While various distribution-based methods exist to provide safety mechanisms for AI models, there is a noted lack of systematic assessment of these methods, especially in the context of safety-critical automotive applications. Many methods in the literature do not adapt well to the quick response times required in safety-critical edge applications. This paper reviews these methods, discusses their suitability for safety-critical applications, and highlights their strengths and limitations. The paper also proposes potential improvements to enhance the safety and reliability of AI algorithms in autonomous vehicles in the context of rapid and accurate decision-making processes.

BRAUM: Analyzing and Protecting Autonomous Machine Software Stack

Poltergeist: Acoustic Adversarial Machine Learning against Cameras and Computer Vision

Adversarial Computer Vision Via Acoustic Manipulation of Camera Sensors

VAP: The Vulnerability-Adaptive Protection Paradigm Toward Reliable Autonomous Machines

Guardauto: A Decentralized Runtime Protection System for Autonomous Driving

Runtime unknown unsafe scenarios identification for SOTIF of autonomous vehicles

Realistic Extreme Behavior Generation for Improved AV Testing

ML-driven Malware that Targets AV Safety

Strategic Safety-Critical Attacks Against an Advanced Driver Assistance System

Kayotee: A Fault Injection-based System to Assess the Safety and Reliability of Autonomous Vehicles to Faults and Errors

Work-in-Progress: Crash Course: Can (Under Attack) Autonomous Driving Beat Human Drivers?

AVstack: An Open-Source, Reconfigurable Platform for Autonomous Vehicle Development

Road to safe autonomy with data and formal reasoning

Cyberattacks on Adaptive Cruise Control Vehicles: An Analytical Characterization

MAVFI: An End-to-End Fault Analysis Framework with Anomaly Detection and Recovery for Micro Aerial Vehicles

Too Afraid to Drive: Systematic Discovery of Semantic DoS Vulnerability in Autonomous Driving Planning under Physical-World Attacks

Autonomous Driving Security: State of the Art and Challenges

COMPUTER-AIDED DESIGN OF FAULT-TOLERANT HARDWARE ARCHITECTURES FOR AUTONOMOUS DRIVING SYSTEMS

VCAT: Vulnerability-aware and Curiosity-driven Adversarial Training for Enhancing Autonomous Vehicle Robustness

The Security of Autonomous Driving: Threats, Defenses, and Future Directions

Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications