Abstract:This paper explores the role and challenges of Artificial Intelligence (AI) algorithms, specifically AI-based software elements, in autonomous driving systems. These AI systems are fundamental in executing real-time critical functions in complex and high-dimensional environments. They handle vital tasks like multi-modal perception, cognition, and decision-making tasks such as motion planning, lane keeping, and emergency braking. A primary concern relates to the ability (and necessity) of AI models to generalize beyond their initial training data. This generalization issue becomes evident in real-time scenarios, where models frequently encounter inputs not represented in their training or validation data. In such cases, AI systems must still function effectively despite facing distributional or domain shifts. This paper investigates the risk associated with overconfident AI models in safety-critical applications like autonomous driving. To mitigate these risks, methods for training AI models that help maintain performance without overconfidence are proposed. This involves implementing certainty reporting architectures and ensuring diverse training data. While various distribution-based methods exist to provide safety mechanisms for AI models, there is a noted lack of systematic assessment of these methods, especially in the context of safety-critical automotive applications. Many methods in the literature do not adapt well to the quick response times required in safety-critical edge applications. This paper reviews these methods, discusses their suitability for safety-critical applications, and highlights their strengths and limitations. The paper also proposes potential improvements to enhance the safety and reliability of AI algorithms in autonomous vehicles in the context of rapid and accurate decision-making processes.

Predicting future AI failures from historic examples

Artificial Intelligence Safety and Cybersecurity: a Timeline of AI Failures

Understanding and Avoiding AI Failures: A Practical Guide

Predictable Artificial Intelligence

AI Failures: A Review of Underlying Issues

Classification Schemas for Artificial Intelligence Failures

AI Safety: A Climb To Armageddon?

On Safety Assessment of Artificial Intelligence

Artificial intelligence (AI) and process safety: Some cautionary observations

Statistical perspectives on reliability of artificial intelligence systems

Ten AI Stepping Stones for Cybersecurity

Reliability: understanding cognitive human bias in artificial intelligence for national security and intelligence analysis

Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications

Can We Rely on AI?

Advancing the cybersecurity of the healthcare system with self-optimising and self-adaptative artificial intelligence (part 2)

Better Future through AI: Avoiding Pitfalls and Guiding AI Towards its Full Potential

Why AI is Harder Than We Think

Strategies to architect AI Safety: Defense to guard AI from Adversaries

Techniques for Adversarial Examples Threatening the Safety of Artificial Intelligence Based Systems

Artificial Intelligence Applications for Confronting Cybersecurity Issues

What AI evaluations for preventing catastrophic risks can and cannot do