Abstract:Large language models (LLMs) as autonomous agents offer a novel avenue for tackling real-world challenges through a knowledge-driven manner. These LLM-enhanced methodologies excel in generalization and interpretability. However, the complexity of driving tasks often necessitates the collaboration of multiple, heterogeneous agents, underscoring the need for such LLM-driven agents to engage in cooperative knowledge sharing and cognitive synergy. Despite the promise of LLMs, current applications predominantly center around single agent scenarios. To broaden the horizons of knowledge-driven strategies and bolster the generalization capabilities of autonomous agents, we propose the KoMA framework consisting of multi-agent interaction, multi-step planning, shared-memory, and ranking-based reflection modules to enhance multi-agents' decision-making in complex driving scenarios. Based on the framework's generated text descriptions of driving scenarios, the multi-agent interaction module enables LLM agents to analyze and infer the intentions of surrounding vehicles, akin to human cognition. The multi-step planning module enables LLM agents to analyze and obtain final action decisions layer by layer to ensure consistent goals for short-term action decisions. The shared memory module can accumulate collective experience to make superior decisions, and the ranking-based reflection module can evaluate and improve agent behavior with the aim of enhancing driving safety and efficiency. The KoMA framework not only enhances the robustness and adaptability of autonomous driving agents but also significantly elevates their generalization capabilities across diverse scenarios. Empirical results demonstrate the superiority of our approach over traditional methods, particularly in its ability to handle complex, unpredictable driving environments without extensive retraining.

KoMA: Knowledge-driven Multi-agent Framework for Autonomous Driving with Large Language Models

DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving

AgentsCoDriver: Large Language Model Empowered Collaborative Driving with Lifelong Learning

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

A Language Agent for Autonomous Driving

Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework

Facilitating Autonomous Driving Tasks with Large Language Models

DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models

KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graph

Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving

CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic

Large Language Models for Autonomous Driving (LLM4AD): Concept, Benchmark, Simulation, and Real-Vehicle Experiment

CALMM-Drive: Confidence-Aware Autonomous Driving with Large Multimodal Model

OmniDrive: A Holistic LLM-Agent Framework for Autonomous Driving with 3D Perception, Reasoning and Planning

DriveLLM: Charting the Path Toward Full Autonomous Driving with Large Language Models

Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles

Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles

A Superalignment Framework in Autonomous Driving with Large Language Models

World knowledge-enhanced Reasoning Using Instruction-guided Interactor in Autonomous Driving

A Survey on Multimodal Large Language Models for Autonomous Driving