Abstract:Large language models (LLMs) have demonstrated remarkable zero-shot generalization abilities: state-of-the-art chatbots can provide plausible answers to many common questions that arise in daily life. However, so far, LLMs cannot reliably solve long-horizon planning problems. By contrast, classical planners, once a problem is given in a formatted way, can use efficient search algorithms to quickly identify correct, or even optimal, plans. In an effort to get the best of both worlds, this paper introduces LLM+P, the first framework that incorporates the strengths of classical planners into LLMs. LLM+P takes in a natural language description of a planning problem, then returns a correct (or optimal) plan for solving that problem in natural language. LLM+P does so by first converting the language description into a file written in the planning domain definition language (PDDL), then leveraging classical planners to quickly find a solution, and then translating the found solution back into natural language. Along with LLM+P, we define a diverse set of different benchmark problems taken from common planning scenarios. Via a comprehensive set of experiments on these benchmark problems, we find that LLM+P is able to provide optimal solutions for most problems, while LLMs fail to provide even feasible plans for most problems.\footnote{The code and results are publicly available at <a class="link-external link-https" href="https://github.com/Cranial-XIX/llm-pddl.git" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to address the deficiencies of large - language models (LLMs) in solving long - term planning problems. Specifically, although LLMs perform well on many daily problems, they cannot reliably solve long - term tasks in robot planning. In contrast, given a formatted problem, classical planners can use efficient search algorithms to quickly find the correct or optimal solution. Therefore, this paper proposes the LLM + P framework, integrating the advantages of classical planners into LLMs to achieve the optimal solution of planning problems described in natural language. ### Main contributions 1. **LLM+P framework**: This framework can convert planning problems described in natural language into PDDL format, use classical planners to find the optimal solution, and then convert the solution back into natural language. 2. **Benchmark tests**: A set of diverse benchmark problems covering multiple robot - planning scenarios is defined, and the effectiveness of LLM + P is verified through experiments. 3. **Experimental results**: Experiments show that LLM + P can provide optimal solutions for most problems, while using LLMs alone is difficult to provide feasible plans. ### Method overview 1. **Problem - description conversion**: The LLM converts the planning problem described in natural language into PDDL format. 2. **Planning solution**: Use classical planners to quickly find the optimal solution. 3. **Solution translation**: Convert the solution generated by the planner back into natural language. ### Experimental results - **BARMAN**: The success rate of LLM + P is 20% (100% optimal), while that of other methods is 0. - **BLOCKSWORLD**: The success rate of LLM + P is 90%, while the success rate of other methods is lower. - **GRIPPERS**: The success rate of LLM + P is 95% (100% optimal), while the success rate of other methods is lower. - **STORAGE**: The success rate of LLM + P is 85%, while that of other methods is 0. - **TERMES**: The success rate of LLM + P is 20%, while that of other methods is 0. - **TYREWORLD**: The success rate of LLM + P is 10% (90% optimal), while the success rate of other methods is lower. ### Conclusion The LLM + P framework proposed in this paper effectively combines the advantages of LLMs and classical planners and can generate optimal solutions in a variety of robot - planning tasks. Future research directions include enabling the LLM to automatically detect when to use LLM + P and reducing the dependence on information provided by humans.

LLM+P: Empowering Large Language Models with Optimal Planning Proficiency

Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models

Query-Efficient Planning with Language Models

Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planning

Planning Anything with Rigor: General-Purpose Zero-Shot Planning with LLM-based Formalized Programming

NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions

Translating Natural Language to Planning Goals with Large-Language Models

Exploring and Benchmarking the Planning Capabilities of Large Language Models

LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Planner

Large language model empowered participatory urban planning

Understanding the Capabilities of Large Language Models for Automated Planning

Interactive and Expressive Code-Augmented Planning with Large Language Models

LASP: Surveying the State-of-the-Art in Large Language Model-Assisted AI Planning

LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models

On the Limit of Language Models as Planning Formalizers

Learning to Plan with Natural Language

On the Planning Abilities of Large Language Models : A Critical Investigation

Large Language Models as Planning Domain Generators

Planning-Driven Programming: A Large Language Model Programming Workflow