Interactive and Expressive Code-Augmented Planning with Large Language Models

Anthony Z. Liu,Xinhe Wang,Jacob Sansom,Yao Fu,Jongwook Choi,Sungryull Sohn,Jaekyeom Kim,Honglak Lee

2024-11-21

Abstract:Large Language Models (LLMs) demonstrate strong abilities in common-sense reasoning and interactive decision-making, but often struggle with complex, long-horizon planning tasks. Recent techniques have sought to structure LLM outputs using control flow and other code-adjacent techniques to improve planning performance. These techniques include using variables (to track important information) and functions (to divide complex tasks into smaller re-usable sub-tasks). However, purely code-based approaches can be error-prone and insufficient for handling ambiguous or unstructured data. To address these challenges, we propose REPL-Plan, an LLM planning approach that is fully code-expressive (it can utilize all the benefits of code) while also being dynamic (it can flexibly adapt from errors and use the LLM for fuzzy situations). In REPL-Plan, an LLM solves tasks by interacting with a Read-Eval-Print Loop (REPL), which iteratively executes and evaluates code, similar to language shells or interactive code notebooks, allowing the model to flexibly correct errors and handle tasks dynamically. We demonstrate that REPL-Plan achieves strong results across various planning domains compared to previous methods.

Computation and Language,Machine Learning

What problem does this paper attempt to address?

This paper attempts to solve the problems encountered by large - language models (LLMs) when dealing with complex, long - term planning tasks. Although LLMs perform well in common - sense reasoning and interactive decision - making, they are often prone to errors or make inaccurate short - term decisions when handling tasks that require long - time spans and multi - step decomposition. These problems include:

1. **Ambiguous sub - problems**: Many tasks require planners to be able to solve some "ambiguous" tasks that are difficult to solve directly by code, such as interpreting unstructured observation data or making subjective choices (such as "purchasing the product that best meets the user's request").
2. **Bottom - up programming nature**: Solving problems using code usually requires gradually solving tasks from the bottom up, which includes writing sub - procedure functions and considering case statements for each possible function output or sub - task result. This approach requires precise pre - thinking.
3. **Coding errors**: Even for experienced human programmers, it is difficult to write accurate code at once.

To solve the above problems, the author proposes **REPL - Plan**, which is a method that utilizes the interaction between LLMs and an extended REPL (Read - Eval - Print Loop) environment to achieve dynamic and expressively rich code - enhanced planning. REPL - Plan allows LLMs to interact with the environment by writing code line by line, and can recursively generate sub - REPLs to solve complex sub - tasks, thereby achieving top - down task decomposition. This method not only improves the flexibility of task solving, but also enhances the ability to handle ambiguous sub - problems and reduces the impact of coding errors.