Inherent Diverse Redundant Safety Mechanisms for AI-based Software Elements in Automotive Applications

Mandar Pitale, Alireza Abbaspour, Devesh Upadhyay

2024-02-13

Abstract:This paper explores the role and challenges of Artificial Intelligence (AI) algorithms, specifically AI-based software elements, in autonomous driving systems. These AI systems are fundamental in executing real-time critical functions in complex and high-dimensional environments. They handle vital tasks like multi-modal perception, cognition, and decision-making tasks such as motion planning, lane keeping, and emergency braking. A primary concern relates to the ability (and necessity) of AI models to generalize beyond their initial training data. This generalization issue becomes evident in real-time scenarios, where models frequently encounter inputs not represented in their training or validation data. In such cases, AI systems must still function effectively despite facing distributional or domain shifts. This paper investigates the risk associated with overconfident AI models in safety-critical applications like autonomous driving. To mitigate these risks, methods for training AI models that help maintain performance without overconfidence are proposed. This involves implementing certainty reporting architectures and ensuring diverse training data. While various distribution-based methods exist to provide safety mechanisms for AI models, there is a noted lack of systematic assessment of these methods, especially in the context of safety-critical automotive applications. Many methods in the literature do not adapt well to the quick response times required in safety-critical edge applications. This paper reviews these methods, discusses their suitability for safety-critical applications, and highlights their strengths and limitations. The paper also proposes potential improvements to enhance the safety and reliability of AI algorithms in autonomous vehicles in the context of rapid and accurate decision-making processes.

Artificial Intelligence

What problem does this paper attempt to address?

This paper discusses the challenges of applying artificial intelligence (AI) in autonomous driving systems, particularly the safety risks caused by AI models being too overconfident. AI systems are responsible for executing real-time critical functions such as multimodal perception, cognition, and decision-making tasks like motion planning, lane keeping, and emergency braking. However, they must still be able to operate effectively when encountering inputs that are not seen in the training data, even in the presence of distribution or domain shifts. The paper highlights the potential dangers of AI models being too overconfident in safety-critical applications, such as misclassifying objects or predicting paths incorrectly in autonomous driving.

The paper analyzes the risks of overconfident AI models and proposes training methods to maintain performance without generating excessive confidence, including implementing deterministic reporting architectures and ensuring diverse training data. While there are various distribution-based approaches available to provide safety mechanisms for AI models, systematic evaluations of these methods, especially in safety-critical automotive applications, are lacking. Many existing methods are not suitable for the rapid response time required by safety-critical edge applications.

The paper reviews these methods, discusses their applicability in safety-critical applications, and highlights their advantages and limitations. Potential improvement measures are proposed to enhance the safety and reliability of AI algorithms in autonomous vehicles, particularly in fast and accurate decision-making processes.

The main contributions of the paper include:
1. Reviewing the latest literature in the field of AI confidence.
2. Summarizing the advantages and disadvantages of these methods from the perspective of automotive safety.
3. Proposing new solutions to enhance the safety of AI models by using intrinsic diverse redundant security architectures and analyzing the wide-ranging impact of these safety measures on the overall reliability of AI components.

The subsequent sections of the paper extensively discuss the issues of overconfident AI models, define relevant terms, and review related work, including recent solutions for managing overconfident AI models such as anomaly detection, distribution monitoring, and uncertainty estimation methods. Finally, a method is proposed to improve the safety of AI models by using intrinsic diverse redundant architectures, and the impact of these safety measures on the overall system reliability is discussed.