LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behavior Trees

Haotian Zhou,Yunhan Lin,Longwu Yan,Jihong Zhu,Huasong Min

2024-04-08

Abstract:Large Language Models (LLMs) have been widely utilized to perform complex robotic tasks. However, handling external disturbances during tasks is still an open challenge. This paper proposes a novel method to achieve robotic adaptive tasks based on LLMs and Behavior Trees (BTs). It utilizes ChatGPT to reason the descriptive steps of tasks. In order to enable ChatGPT to understand the environment, semantic maps are constructed by an object recognition algorithm. Then, we design a Parser module based on Bidirectional Encoder Representations from Transformers (BERT) to parse these steps into initial BTs. Subsequently, a BTs Update algorithm is proposed to expand the initial BTs dynamically to control robots to perform adaptive tasks. Different from other LLM-based methods for complex robotic tasks, our method outputs variable BTs that can add and execute new actions according to environmental changes, which is robust to external disturbances. Our method is validated with simulation in different practical scenarios.

Robotics

What problem does this paper attempt to address?

The paper aims to address the issue of robots' insufficient ability to cope with external disturbances when performing complex tasks. Although large language models (LLMs) excel in handling complex robotic tasks, their performance in the face of external disturbances still needs improvement. The paper proposes a new method based on LLMs and behavior trees (BTs) to achieve adaptive tasks for robots. Specifically, the method uses ChatGPT to infer the descriptive steps of tasks and enables ChatGPT to understand the environment by constructing a semantic map. Subsequently, a parsing module based on Bidirectional Encoder Representations from Transformers (BERT) is designed to parse these steps into initial behavior trees (BTs). Then, a behavior tree updating algorithm is proposed to dynamically expand the initial behavior trees to control the robot in performing adaptive tasks. Compared to existing LLM-based methods, this method can add and execute new actions according to environmental changes, thereby exhibiting stronger robustness to external disturbances. The main contributions of the paper include: 1. Proposing a new method for performing adaptive tasks for robots by automatically constructing and dynamically expanding behavior trees. To the best of the authors' knowledge, this is the first time behavior trees have been generated through LLMs. 2. Comparing with advanced LLM-based methods and discussing the advantages and limitations of this method.

LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behavior Trees

LLM as BT-Planner: Leveraging LLMs for Behavior Tree Generation in Robot Task Planning

A Study on Training and Developing Large Language Models for Behavior Tree Generation

BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs

LLM-BRAIn: AI-driven Fast Generation of Robot Behaviour Tree based on Large Language Model

Behavior Tree Generation using Large Language Models for Sequential Manipulation Planning with Human Instructions and Feedback

HBTP: Heuristic Behavior Tree Planning with Large Language Model Reasoning

Automatic Behavior Tree Expansion with LLMs for Robotic Manipulation

Integrating Intent Understanding and Optimal Behavior Planning for Behavior Tree Generation from Human Instructions

Large Language Models for Robotics: Opportunities, Challenges, and Perspectives

FLTRNN: Faithful Long-Horizon Task Planning for Robotics with Large Language Models

Decision-Making in Robotic Grasping with Large Language Models.

Generalizable Long-Horizon Manipulations with Large Language Models

DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models

Applying Large Language Model to a Control System for Multi-Robot Task Assignment

Consolidating Trees of Robotic Plans Generated Using Large Language Models to Improve Reliability

Enhancing Robot Task Planning and Execution through Multi-Layer Large Language Models

MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model

MHRC: Closed-loop Decentralized Multi-Heterogeneous Robot Collaboration with Large Language Models

RoboCoder: Robotic Learning from Basic Skills to General Tasks with Large Language Models