Abstract:To enable non-experts to specify long-horizon, multi-robot collaborative tasks, language models are increasingly used to translate natural language commands into formal specifications. However, because translation can occur in multiple ways, such translations may lack accuracy or lead to inefficient multi-robot planning. Our key insight is that concise hierarchical specifications can simplify planning while remaining straightforward to derive from human instructions. We propose~\acronym{}, a framework that translates natural language commands into hierarchical Linear Temporal Logic (LTL) and solves the corresponding planning problem. The translation involves two steps leveraging Large Language Models (LLMs). First, an LLM transforms instructions into a Hierarchical Task Tree, capturing logical and temporal relations. Next, a fine-tuned LLM converts sub-tasks into flat LTL formulas, which are aggregated into hierarchical specifications, with the lowest level corresponding to ordered robot actions. These specifications are then used with off-the-shelf planners. Our~\acronym{} demonstrates the potential of LLMs in hierarchical reasoning for multi-robot task planning. Evaluations in simulation and real-world experiments with human participants show that~\acronym{} outperforms existing methods, handling more complex instructions while achieving higher success rates and lower costs in task allocation and planning. Additional details are available at <a class="link-external link-https" href="https://nl2hltl2plan.github.io" rel="external noopener nofollow">this https URL</a> .

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use natural - language instructions to efficiently plan multi - robot collaborative tasks. Specifically, the paper focuses on how to transform natural - language commands from non - expert users into formal task specifications for robots to understand and execute. Traditional methods from natural language to task planning have problems with accuracy and efficiency when dealing with complex instructions, and the NL2HLTL2PLAN framework proposed in this paper aims to simplify this process through Hierarchical Linear Temporal Logic (HLTL) and improve the success rate and efficiency of multi - robot task planning. ### Main Problems and Solutions 1. **Problem**: Existing natural - language processing methods may lead to inaccurate or inefficient multi - robot planning when transforming natural - language instructions into formal specifications that robots can understand due to diverse translation methods. **Solution**: The paper proposes a framework named NL2HLTL2PLAN. This framework uses large - language models (LLMs) to transform natural - language commands into Hierarchical Linear Temporal Logic (HLTL) and solves the corresponding planning problems through off - the - shelf planners. This process is divided into two steps: - **First step**: Use an LLM to transform natural - language instructions into a Hierarchical Task Tree (HTT) to capture logical and temporal relationships. - **Second step**: Use a fine - tuned LLM to transform subtasks into flat LTL formulas, which are aggregated into hierarchical specifications, with the lowest level corresponding to ordered robot actions. 2. **Problem**: Existing methods perform poorly when dealing with complex instructions, especially in multi - robot systems. **Solution**: NL2HLTL2PLAN simplifies task representation through a hierarchical structure, making planning more efficient and easier to derive from human instructions. Experimental results show that NL2HLTL2PLAN exhibits a higher success rate and lower cost when dealing with more complex instructions. ### Experimental Verification The paper verifies the effectiveness of NL2HLTL2PLAN through simulation environments and real - world experiments: - **Simulation experiment**: Experiments were carried out using the AI2 - THOR simulator and the ALFRED dataset to evaluate the performance of NL2HLTL2PLAN when dealing with complex mobile manipulation tasks. The results show that NL2HLTL2PLAN outperforms the existing SMART - LLM method in terms of success rate, travel cost, and completion time. - **Real - world experiment**: Through a real - world tabletop experiment, robot arms were made to place fruits and vegetables on colored plates according to natural - language instructions. The experimental results show that NL2HLTL2PLAN has good adaptability when dealing with instructions in different user styles, and the generated plans are superior to the method of directly using an LLM in terms of travel distance and running time. ### Conclusion NL2HLTL2PLAN effectively solves the transformation problem from natural - language instructions to multi - robot task planning by introducing Hierarchical Linear Temporal Logic, improving the accuracy and efficiency of planning. This provides new possibilities for non - expert users to interact with multi - robot systems through natural language.

Nl2Hltl2Plan: Scaling Up Natural Language Understanding for Multi-Robots Through Hierarchical Temporal Logic Task Representation

FLTRNN: Faithful Long-Horizon Task Planning for Robotics with Large Language Models

Simultaneous Task Allocation and Planning for Multi-Robots under Hierarchical Temporal Logic Specifications

Decomposition-based Hierarchical Task Allocation and Planning for Multi-Robots under Hierarchical Temporal Logic Specifications

MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model

Conformal Temporal Logic Planning using Large Language Models

Leveraging Large Language Model for Heterogeneous Ad Hoc Teamwork Collaboration

MHRC: Closed-loop Decentralized Multi-Heterogeneous Robot Collaboration with Large Language Models

SELP: Generating Safe and Efficient Task Plans for Robot Agents with Large Language Models

Robot Task Planning Based on Large Language Model Representing Knowledge with Directed Graph Structures

LaMMA-P: Generalizable Multi-Agent Long-Horizon Task Allocation and Planning with LM-Driven PDDL Planner

COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models

Large Language Models for Orchestrating Bimanual Robots

Enhancing Robot Task Planning and Execution through Multi-Layer Large Language Models

Hierarchical Planning with State Abstractions for Temporal Task Specifications.

Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments

DELTA: Decomposed Efficient Long-Term Robot Task Planning using Large Language Models

LLM as BT-Planner: Leveraging LLMs for Behavior Tree Generation in Robot Task Planning

SMART-LLM: Smart Multi-Agent Robot Task Planning using Large Language Models

DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models

Towards Human Awareness in Robot Task Planning with Large Language Models