Abstract:Artificial intelligence (AI) plays a crucial role in autonomous driving (AD) research, propelling its development towards intelligence and efficiency. Currently, the development of AD technology follows two main technical paths: modularization and end-to-end. Modularization decompose the driving task into modules such as perception, prediction, planning, and control, and train them separately. Due to the inconsistency of training objectives between modules, the integrated effect suffers from bias. End-to-end attempts to address this issue by utilizing a single model that directly maps from sensor data to control signals. This path has limited learning capabilities in a comprehensive set of features and struggles to handle unpredictable long-tail events and complex urban traffic scenarios. In the face of challenges encountered in both paths, many researchers believe that large language models (LLMs) with powerful reasoning capabilities and extensive knowledge understanding may be the solution, expecting LLMs to provide AD systems with deeper levels of understanding and decision-making capabilities. In light of the challenges faced by both paths, many researchers believe that LLMs, with their powerful reasoning abilities and extensive knowledge, could offer a solution. To understand if LLMs could enhance AD, this paper conducts a thorough analysis of the potential applications of LLMs in AD systems, including exploring their optimization strategies in both modular and end-to-end approaches, with a particular focus on how LLMs can tackle the problems and challenges present in current solutions. Furthermore, we discuss an important question: Can LLM-based artificial general intelligence (AGI) be a key to achieve high-level AD? We further analyze the potential limitations and challenges that LLMs may encounter in promoting the development of AD technology.

Applications of Large Scale Foundation Models for Autonomous Driving

Prospective Role of Foundation Models in Advancing Autonomous Vehicles

LLM4Drive: A Survey of Large Language Models for Autonomous Driving

A Survey on Large Language Model-empowered Autonomous Driving

A Survey for Foundation Models in Autonomous Driving

Large Language Models for Autonomous Driving (LLM4AD): Concept, Benchmark, Simulation, and Real-Vehicle Experiment

Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

Facilitating Autonomous Driving Tasks with Large Language Models

Empowering Autonomous Driving with Large Language Models: A Safety Perspective

DriveLLM: Charting the Path Toward Full Autonomous Driving with Large Language Models

Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles

Receive, Reason, and React: Drive as You Say, With Large Language Models in Autonomous Vehicles

Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles

Testing Large Language Models on Driving Theory Knowledge and Skills for Connected Autonomous Vehicles

Large Language Models for Human-like Autonomous Driving: A Survey

Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunities

DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models

Large Language Models for Intelligent Transportation: A Review of the State of the Art and Challenges

Large-scale Foundation Models and Generative AI for BigData Neuroscience

A survey on large language model based autonomous agents