Abstract:Autonomous driving has advanced significantly due to sensors, machine learning, and artificial intelligence improvements. However, prevailing methods struggle with intricate scenarios and causal relationships, hindering adaptability and interpretability in varied environments. To address the above problems, we introduce LeapAD, a novel paradigm for autonomous driving inspired by the human cognitive process. Specifically, LeapAD emulates human attention by selecting critical objects relevant to driving decisions, simplifying environmental interpretation, and mitigating decision-making complexities. Additionally, LeapAD incorporates an innovative dual-process decision-making module, which consists of an Analytic Process (System-II) for thorough analysis and reasoning, along with a Heuristic Process (System-I) for swift and empirical processing. The Analytic Process leverages its logical reasoning to accumulate linguistic driving experience, which is then transferred to the Heuristic Process by supervised fine-tuning. Through reflection mechanisms and a growing memory bank, LeapAD continuously improves itself from past mistakes in a closed-loop environment. Closed-loop testing in CARLA shows that LeapAD outperforms all methods relying solely on camera input, requiring 1-2 orders of magnitude less labeled data. Experiments also demonstrate that as the memory bank expands, the Heuristic Process with only 1.8B parameters can inherit the knowledge from a GPT-4 powered Analytic Process and achieve continuous performance improvement. Project page: <a class="link-external link-https" href="https://pjlab-adg.github.io/LeapAD" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the lack of adaptability and interpretability in current autonomous driving technology when dealing with complex scenarios and causal relationships. Specifically: 1. **Adaptability problem**: Existing methods have difficulty making accurate decisions when faced with complex and changeable driving environments. This is because these methods mainly rely on pattern induction from a large amount of training data and lack the ability to understand and reason about unseen scenarios. 2. **Interpretability problem**: Existing methods have low transparency in the decision - making process and it is difficult to explain the logic behind their decisions, which limits the credibility of the system and user acceptance. To solve these problems, the author proposes a new paradigm named LeapAD, which is inspired by the human cognitive process and aims to improve the adaptability and interpretability of autonomous driving systems by simulating the human attention mechanism and dual - process decision - making module. Specific contributions include: - **Innovative closed - loop autonomous driving method**: LeapAD simplifies the environmental description and decision - making process by focusing on key objects, thereby imitating the behavior of human drivers. - **Dual - process decision - making module inspired by human cognitive theory**: This module consists of an analytic process and a heuristic process. The former is responsible for detailed analysis and reasoning, while the latter conducts rapid empirical processing. - **Continuous learning and generalization ability**: Through the analytic process and the reflection mechanism, the system can continuously accumulate high - quality driving experience and transfer it to the heuristic process to achieve continuous learning and improvement. - **Experimental verification**: Experiments conducted in the CARLA simulator show that LeapAD not only outperforms other methods when using only camera input but also requires far less labeled data than other methods. Through these designs, LeapAD aims to improve the adaptability and interpretability of autonomous driving systems in complex environments, thereby promoting the development of autonomous driving technology.

Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving

Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving

Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

Humanlike Driving: Empirical Decision-Making System for Autonomous Vehicles

End-to-End Autonomous Driving without Costly Modularization and 3D Manual Annotation

Planning-oriented Autonomous Driving

Think Twice Before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving

Parameterized Decision-Making with Multi-Modality Perception for Autonomous Driving

Parameterized Decision-making with Multi-modal Perception for Autonomous Driving

Design of a Deep Learning-Based Autonomous Vehicle Environmental Perception and Decision-Making System

Towards Knowledge-driven Autonomous Driving

An Integrated Framework of Lateral and Longitudinal Behavior Decision-Making for Autonomous Driving Using Reinforcement Learning

Continual Learning for Adaptable Car-Following in Dynamic Traffic Environments

Planning Automated Driving with Accident Experience Referencing and Common-sense Inferencing

GAD-Generative Learning for HD Map-Free Autonomous Driving

A Safe and Efficient Self-evolving Algorithm for Decision-making and Control of Autonomous Driving Systems

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

LLM4Drive: A Survey of Large Language Models for Autonomous Driving

SafeDrive: Knowledge- and Data-Driven Risk-Sensitive Decision-Making for Autonomous Vehicles with Large Language Models

Road Traffic Law Adaptive Decision-making for Self-Driving Vehicles