Abstract:Vehicle motion planning is an essential component of autonomous driving technology. Current rule-based vehicle motion planning methods perform satisfactorily in common scenarios but struggle to generalize to long-tailed situations. Meanwhile, learning-based methods have yet to achieve superior performance over rule-based approaches in large-scale closed-loop scenarios. To address these issues, we propose PlanAgent, the first mid-to-mid planning system based on a Multi-modal Large Language Model (MLLM). MLLM is used as a cognitive agent to introduce human-like knowledge, interpretability, and common-sense reasoning into the closed-loop planning. Specifically, PlanAgent leverages the power of MLLM through three core modules. First, an Environment Transformation module constructs a Bird's Eye View (BEV) map and a lane-graph-based textual description from the environment as inputs. Second, a Reasoning Engine module introduces a hierarchical chain-of-thought from scene understanding to lateral and longitudinal motion instructions, culminating in planner code generation. Last, a Reflection module is integrated to simulate and evaluate the generated planner for reducing MLLM's uncertainty. PlanAgent is endowed with the common-sense reasoning and generalization capability of MLLM, which empowers it to effectively tackle both common and complex long-tailed scenarios. Our proposed PlanAgent is evaluated on the large-scale and challenging nuPlan benchmarks. A comprehensive set of experiments convincingly demonstrates that PlanAgent outperforms the existing state-of-the-art in the closed-loop motion planning task. Codes will be soon released.

Knowledge Organization And Logical Description For Multi-Agent Tactical Plan Recognition

Plan Library Organization and Solution for Multi-Agent Intention Recognition

A Plan Recognition Approach for Agent in Adversarial Multi-Agent System

Analysis,Description and Design for A Tactical Plan Recognition Model

Study on Plan Recognition Based Multi-Agent Collaboration

Action-Model Based Multi-agent Plan Recognition

Multi-Agent Plan Recognition with Partial Team Traces and Plan Libraries.

An AI Planning-Based Approach to the Multi-Agent Plan Recognition Problem

Probabilistic Plan Recognition Under Temporal Logic Tasks

Logical Agents that Plan, Execute, and Monitor Communication

Embodied Multi-Agent Task Planning from Ambiguous Instruction

Multiagent Motion Planning Based on Deep Reinforcement Learning in Complex Environments

Multi-agent Cooperation: A Description Logic View

Multi-agent Layered Cooperation Planning Method with Domain Knowledge.

A Multi-Stage Attack Planning Problem Description Model Based on Knowledge Representation

A multi-agent planning approach integrated with learning mechanism

Team Plan Recognition: A Review of the State of the Art

Research on Tactical Intention Recognition Method Based on Probabilistic Network Ontology Language

PlanAgent: A Multi-modal Large Language Agent for Closed-loop Vehicle Motion Planning

Multi-agent Knowing How via Multi-step Plans - A Dynamic Epistemic Planning Based Approach.

Study Of Ontology-Based Multi-Agent System For Automatic Process