DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models

Yongdong Wang,Runze Xiao,Jun Younes Louhi Kasahara,Ryosuke Yajima,Keiji Nagatani,Atsushi Yamashita,Hajime Asama

2024-11-14

Abstract:Large Language Models (LLMs) have demonstrated significant reasoning capabilities in robotic systems. However, their deployment in multi-robot systems remains fragmented and struggles to handle complex task dependencies and parallel execution. This study introduces the DART-LLM (Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models) system, designed to address these challenges. DART-LLM utilizes LLMs to parse natural language instructions, decomposing them into multiple subtasks with dependencies to establish complex task sequences, thereby enhancing efficient coordination and parallel execution in multi-robot systems. The system includes the QA LLM module, Breakdown Function modules, Actuation module, and a Vision-Language Model (VLM)-based object detection module, enabling task decomposition and execution from natural language instructions to robotic actions. Experimental results demonstrate that DART-LLM excels in handling long-horizon tasks and collaborative tasks with complex dependencies. Even when using smaller models like Llama 3.1 8B, the system achieves good performance, highlighting DART-LLM's robustness in terms of model size. Please refer to the project website \url{<a class="link-external link-https" href="https://wyd0817.github.io/project-dart-llm/" rel="external noopener nofollow">this https URL</a>} for videos and code.

Robotics

What problem does this paper attempt to address?

This paper attempts to solve the problems of task - dependent management and parallel execution encountered by multi - robot systems when performing complex tasks. Specifically, although existing large - language models (LLMs) have demonstrated remarkable reasoning abilities in robotic systems, there are still fragmentation problems in the deployment of multi - robot systems, especially in handling complex task dependencies and parallel execution. These problems limit the effectiveness of multi - robot systems in collaborating to complete complex tasks.

To this end, the paper proposes the DART - LLM (Dependency - Aware Multi - Robot Task Decomposition and Execution using Large Language Models) system, aiming to improve the coordination ability and parallel execution efficiency of multi - robot systems in complex task sequences by using large - language models to parse natural - language instructions and decompose them into multiple subtasks with dependencies. The DART - LLM system includes a question - answering large - language model module (QA LLM module), a task - breakdown function module (Breakdown Function modules), an execution module (Actuation module), and a vision - language model (VLM) - based object - detection module. These modules work together to achieve efficient task decomposition and execution from natural - language instructions to robot actions.

Through experimental verification, the DART - LLM system performs well in handling long - term tasks and collaborative tasks with complex dependencies. Even when using a smaller model (such as Llama 3.1 8B), it can achieve good performance, indicating the adaptability and robustness of DART - LLM under different model sizes.