Abstract:Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSAC-LLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at <a class="link-external link-https" href="https://hsacllm.github.io/" rel="external noopener nofollow">this https URL</a>.

Socially Aware Object Goal Navigation with Heterogeneous Scene Representation Learning

ChatNav: Leveraging LLM to Zero-shot Semantic Reasoning in Object Navigation

SemNav-HRO: A Target-Driven Semantic Navigation Strategy with Human–robot–object Ternary Fusion

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation

Think Holistically, Act Down-to-Earth: A Semantic Navigation Strategy with Continuous Environmental Representation and Multi-step Forward Planning

An Object-driven Navigation Strategy Based on Active Perception and Semantic Association

A Study on Learning Social Robot Navigation with Multimodal Perception

3D-Aware Object Goal Navigation Via Simultaneous Exploration and Identification

Rethinking Social Robot Navigation: Leveraging the Best of Two Worlds

Enabling Socially Competent navigation through incorporating HRI

HSPNav: Hierarchical Scene Prior Learning for Visual Semantic Navigation Towards Real Settings

Embodied Contrastive Learning with Geometric Consistency and Behavioral Awareness for Object Navigation

Learning Cross Dimension Scene Representation for Interactive Navigation Agents in Obstacle-Cluttered Environments

Task-Driven Graph Attention for Hierarchical Relational Object Navigation

Multi-Object Navigation with dynamically learned neural implicit representations

Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks

OpenObject-NAV: Open-Vocabulary Object-Oriented Navigation Based on Dynamic Carrier-Relationship Scene Graph

Object Goal Navigation using Goal-Oriented Semantic Exploration

Socially-Aware Navigation: A Non-linear Multi-Objective Optimization Approach

Learning Heterogeneous Relation Graph and Value Regularization Policy for Visual Navigation

CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMs