LLM-BRAIn: AI-driven Fast Generation of Robot Behaviour Tree based on Large Language Model

Artem Lykov,Dzmitry Tsetserukou

2023-05-31

Abstract:This paper presents a novel approach in autonomous robot control, named LLM-BRAIn, that makes possible robot behavior generation, based on operator's commands. LLM-BRAIn is a transformer-based Large Language Model (LLM) fine-tuned from Stanford Alpaca 7B model to generate robot behavior tree (BT) from the text description. We train the LLM-BRAIn on 8,5k instruction-following demonstrations, generated in the style of self-instruct using text-davinchi-003. The developed model accurately builds complex robot behavior while remaining small enough to be run on the robot's onboard microcomputer. The model gives structural and logical correct BTs and can successfully manage instructions that were not presented in training set. The experiment did not reveal any significant subjective differences between BTs generated by LLM-BRAIn and those created by humans (on average, participants were able to correctly distinguish between LLM-BRAIn generated BTs and human-created BTs in only 4.53 out of 10 cases, indicating that their performance was close to random chance). The proposed approach potentially can be applied to mobile robotics, drone operation, robot manipulator systems and Industry 4.0.

Robotics

What problem does this paper attempt to address?

The paper aims to address a problem in autonomous robot control, specifically how to quickly generate a robot's Behavior Tree (BT) based on operator instructions. Specifically, the paper proposes a new method called LLM-BRAIn, which utilizes a Large Language Model (LLM) to generate a robot's Behavior Tree from textual descriptions. This method allows non-technical experts to conveniently control robots through natural language instructions, thereby enhancing the flexibility and usability of robots in practical applications. Additionally, the model is capable of generating structurally and logically correct BTs, and experimental results show that the BTs generated by LLM-BRAIn are almost indistinguishable from those created by humans in terms of subjective perception. This indicates that the model has the potential for widespread application in fields such as mobile robots, drone operations, robot operating systems, and Industry 4.0.

LLM-BRAIn: AI-driven Fast Generation of Robot Behaviour Tree based on Large Language Model

BTGenBot: Behavior Tree Generation for Robotic Tasks with Lightweight LLMs

LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behavior Trees

Behavior Tree Generation using Large Language Models for Sequential Manipulation Planning with Human Instructions and Feedback

LLM as BT-Planner: Leveraging LLMs for Behavior Tree Generation in Robot Task Planning

A Study on Training and Developing Large Language Models for Behavior Tree Generation

Integrating Intent Understanding and Optimal Behavior Planning for Behavior Tree Generation from Human Instructions

LLM-MARS: Large Language Model for Behavior Tree Generation and NLP-enhanced Dialogue in Multi-Agent Robot Systems

Automatic Behavior Tree Expansion with LLMs for Robotic Manipulation

Generative Expressive Robot Behaviors using Large Language Models

A Paragraph is All It Takes: Rich Robot Behaviors from Interacting, Trusted LLMs

Learning and Executing Re-usable Behaviour Trees from Natural Language Instruction

Grounding Language Models in Autonomous Loco-manipulation Tasks

Grounding LLMs For Robot Task Planning Using Closed-loop State Feedback

Large Language Model-Based Code Generation for the Control of Construction Assembly Robots: A Hierarchical Generation Approach

Consolidating Trees of Robotic Plans Generated Using Large Language Models to Improve Reliability

HBTP: Heuristic Behavior Tree Planning with Large Language Model Reasoning

Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models

Drive Like a Human: Rethinking Autonomous Driving with Large Language Models

Language Models as Zero-Shot Trajectory Generators

REAL: Resilience and Adaptation using Large Language Models on Autonomous Aerial Robots