Abstract:Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSAC-LLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at <a class="link-external link-https" href="https://hsacllm.github.io/" rel="external noopener nofollow">this https URL</a>.

Linguistic decision making for robot route learning

Decision-Making in Robotic Grasping with Large Language Models.

Learning Models for Following Natural Language Directions in Unknown Environments

Adaptive and transparent decision-making in autonomous robots through graph-structured world models

Building Decision Making Models Through Language Model Regime

Language-Conditioned Offline RL for Multi-Robot Navigation

Hey Robot! Personalizing Robot Navigation through Model Predictive Control with a Large Language Model

SRLM: Human-in-Loop Interactive Social Robot Navigation with Large Language Model and Deep Reinforcement Learning

Hierarchical Decision Making by Generating and Following Natural Language Instructions

LanguageMPC: Large Language Models as Decision Makers for Autonomous Driving

Breaking Down the Task: A Unit-Grained Hybrid Training Framework for Vision and Language Decision Making

Can Large Language Models Solve Robot Routing?

From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control

CARTIER: Cartographic lAnguage Reasoning Targeted at Instruction Execution for Robots

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation

Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models

Interactive Learning of State Representation through Natural Language Instruction and Explanation

Long-horizon Locomotion and Manipulation on a Quadrupedal Robot with Large Language Models

Towards Interactive and Learnable Cooperative Driving Automation: a Large Language Model-Driven Decision-Making Framework

Nl2Hltl2Plan: Scaling Up Natural Language Understanding for Multi-Robots Through Hierarchical Temporal Logic Task Representation

Large Language Model As Autonomous Decision Maker