Abstract:Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSAC-LLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at <a class="link-external link-https" href="https://hsacllm.github.io/" rel="external noopener nofollow">this https URL</a>.

The Emotionally Intelligent Robot: Improving Social Navigation in Crowded Environments

SocioSense: Robot navigation amongst pedestrians with social and psychological constraints

EmoiPlanner: Human emotion and intention aware socially acceptable robot navigation in human‐centric environments

Social Robot Navigation with Adaptive Proxemics Based on Emotions

ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation

EWareNet: Emotion Aware Human Intent Prediction and Adaptive Spatial Profile Fusion for Social Robot Navigation

The Socially Invisible Robot: Navigation in the Social World using Robot Entitativity

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

From Faces to Traffic Lights: A Multi-Scale Approach for Emotional State Representation

An Emotion-Driven Attention Model For Service Robot

Socially aware robot navigation in crowds via deep reinforcement learning with resilient reward functions

Enabling Socially Competent navigation through incorporating HRI

SACSoN: Scalable Autonomous Control for Social Navigation

Rethinking Social Robot Navigation: Leveraging the Best of Two Worlds

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation

Socially Compliant Robot Navigation in Crowded Environment by Human Behavior Resemblance Using Deep Reinforcement Learning

Learning Social Navigation from Demonstrations with Conditional Neural Processes

Socially-Aware Navigation: A Non-linear Multi-Objective Optimization Approach

Robotic Etiquette: Socially Acceptable Navigation of Service Robots with Human Motion Pattern Learning and Prediction

Towards Safe and Efficient Last-Mile Delivery: A Multi-Modal Socially Aware Navigation Framework for Autonomous Robots on Pedestrian-Crowded Sidewalks