Abstract:Large language models (LLMs) have demonstrated exciting progress in acquiring diverse new capabilities through in-context learning, ranging from logical reasoning to code-writing. Robotics researchers have also explored using LLMs to advance the capabilities of robotic control. However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot. On the other hand, reward functions are shown to be flexible representations that can be optimized for control policies to achieve diverse tasks, while their semantic richness makes them suitable to be specified by LLMs. In this work, we introduce a new paradigm that harnesses this realization by utilizing LLMs to define reward parameters that can be optimized and accomplish variety of robotic tasks. Using reward as the intermediate interface generated by LLMs, we can effectively bridge the gap between high-level language instructions or corrections to low-level robot actions. Meanwhile, combining this with a real-time optimizer, MuJoCo MPC, empowers an interactive behavior creation experience where users can immediately observe the results and provide feedback to the system. To systematically evaluate the performance of our proposed method, we designed a total of 17 tasks for a simulated quadruped robot and a dexterous manipulator robot. We demonstrate that our proposed method reliably tackles 90% of the designed tasks, while a baseline using primitive skills as the interface with Code-as-policies achieves 50% of the tasks. We further validated our method on a real robot arm where complex manipulation skills such as non-prehensile pushing emerge through our interactive system.

MaestroMotif: Skill Design from Artificial Intelligence Feedback

Motif: Intrinsic Motivation from Artificial Intelligence Feedback

Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skills

Agentic Skill Discovery

Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models

Skill-Mix: a Flexible and Expandable Family of Evaluations for AI models

Skill matters: Dynamic skill learning for multi-agent cooperative reinforcement learning

Language to Rewards for Robotic Skill Synthesis

Choreographer: Learning and Adapting Skills in Imagination

Skill Machines: Temporal Logic Skill Composition in Reinforcement Learning

Maestro: A Gamified Platform for Teaching AI Robustness

Augmenting Autotelic Agents with Large Language Models

MultiSkill: Evaluating Large Multimodal Models for Fine-grained Alignment Skills

Winning Is Not Everything: Enhancing Game Development With Intelligent Agents

Hierarchical Cooperative Multi-Agent Reinforcement Learning with Skill Discovery

Skill-Based Reinforcement Learning with Intrinsic Reward Matching

Heterogeneous Skill Learning for Multi-agent Tasks

ChatPCG: Large Language Model-Driven Reward Design for Procedural Content Generation

You Have Thirteen Hours in Which to Solve the Labyrinth: Enhancing AI Game Masters with Function Calling

Abstract then Play: A Skill-centric Reinforcement Learning Framework for Text-based Games.

Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models