Abstract:Large Language Models (LLMs) have shown promise in solving natural language-described planning tasks, but their direct use often leads to inconsistent reasoning and hallucination. While hybrid LLM-symbolic planning pipelines have emerged as a more robust alternative, they typically require extensive expert intervention to refine and validate generated action schemas. It not only limits scalability but also introduces a potential for biased interpretation, as a single expert's interpretation of ambiguous natural language descriptions might not align with the user's actual intent. To address this, we propose a novel approach that constructs an action schema library to generate multiple candidates, accounting for the diverse possible interpretations of natural language descriptions. We further introduce a semantic validation and ranking module that automatically filter and rank the generated schemas and plans without expert-in-the-loop. The experiments showed our pipeline maintains superiority in planning over the direct LLM planning approach. These findings demonstrate the feasibility of a fully automated end-to-end LLM-symbolic planner that requires no expert intervention, opening up the possibility for a broader audience to engage with AI planning with less prerequisite of domain expertise.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the inconsistent reasoning and hallucination issues existing in current large - language models (LLMs) when directly generating planning tasks described in natural language. Although LLMs show potential in solving planning tasks described in natural language, their direct application often leads to inconsistent reasoning and generation of incorrect plans. To address these problems, researchers have proposed a hybrid method combining LLMs and symbolic planning, namely the LLM - symbolic planning pipeline. However, this method usually requires a great deal of expert intervention to refine and verify the generated action patterns, which not only limits the scalability of the method but may also introduce the interpretation bias of individual experts. For this reason, the paper proposes a new method, aiming to construct an action - pattern library to generate multiple candidate solutions, taking into account the diverse interpretations of natural - language descriptions. In addition, a semantic verification and ranking module is introduced, which can automatically filter and rank the generated action patterns and plans without the need for expert involvement. Experimental results show that this pipeline outperforms the method of directly using LLM in terms of planning, demonstrating the feasibility of a fully - automatic end - to - end LLM - symbolic planner. This planner does not require expert intervention, providing the possibility for a wider range of users to participate in AI planning and reducing the need for domain - specific expertise. In short, the paper mainly solves the following problems: 1. **Reducing dependence on expert intervention**: By constructing a diverse action - pattern library and an automated verification and ranking module, the dependence on experts is reduced, and the scalability and efficiency of the method are improved. 2. **Handling the ambiguity of natural - language descriptions**: By generating multiple candidate action patterns, multiple possible interpretations of natural - language descriptions are considered, thereby better capturing the actual intentions of users. 3. **Improving the reliability and quality of planning tasks**: By combining the flexibility of LLMs and the accuracy of symbolic planning, the reliability and quality of generated plans are improved, especially in short - term planning tasks. These innovations together form the core contributions of the paper, providing a more efficient and reliable solution for the AI planning field.

Planning in the Dark: LLM-Symbolic Planning Pipeline without Experts

Human-Centered Planning

Language Models can Infer Action Semantics for Symbolic Planners from Environment Feedback

Query-Efficient Planning with Language Models

LLM+P: Empowering Large Language Models with Optimal Planning Proficiency

LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks

What's the Plan? Evaluating and Developing Planning-Aware Techniques for Language Models

Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving

Large Language Models as Planning Domain Generators

Language-Augmented Symbolic Planner for Open-World Task Planning

LLM-SAP: Large Language Models Situational Awareness Based Planning

On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)

Large Language Models as Commonsense Knowledge for Large-Scale Task Planning

Leveraging Environment Interaction for Automated PDDL Translation and Planning with Large Language Models

NL2Plan: Robust LLM-Driven Planning from Minimal Text Descriptions

Improving Planning with Large Language Models: A Modular Agentic Architecture

On the Planning Abilities of Large Language Models : A Critical Investigation

LLM-Assist: Enhancing Closed-Loop Planning with Language-Based Reasoning

LASP: Surveying the State-of-the-Art in Large Language Model-Assisted AI Planning

On the Planning Abilities of Large Language Models (A Critical Investigation with a Proposed Benchmark)