Abstract:Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSAC-LLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at <a class="link-external link-https" href="https://hsacllm.github.io/" rel="external noopener nofollow">this https URL</a>.

Door and Doorway Etiquette for Virtual Humans

A Behavior Architecture for Fast Humanoid Robot Door Traversals

Simulation of bi-direction pedestrian movement in corridor

Inserting Virtual Pedestrians into Pedestrian Groups Video with Behavior Consistency

UniDoorManip: Learning Universal Door Manipulation Policy Over Large-scale and Diverse Door Manipulation Environments

Imagined Potential Games: A Framework for Simulating, Learning and Evaluating Interactive Behaviors

Deconstructing Pedestrian Crossing Decision-making in Interactions with Continuous Traffic: an Anthropomorphic Model

Would You Mind Me if I Pass by You? Socially-Appropriate Behaviour for an Omni-based Social Robot in Narrow Environment

Dynamic Game-Theoretical Decision-Making Framework for Vehicle-Pedestrian Interaction with Human Bounded Rationality

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation

Unfreezing autonomous vehicles with game theory, proxemics, and trust

Behavior-based Cellular Automaton Model for Pedestrian Dynamics

Towards Courteous Behavior and Trajectory Planning for Automated Driving

EmoiPlanner: Human emotion and intention aware socially acceptable robot navigation in human‐centric environments

Game and Simulation Design for Studying Pedestrian-Automated Vehicle Interactions

Microscopic modeling of attention-based movement behaviors

Deconstructing Pedestrian Crossing Decisions in Interactions with Continuous Traffic: an Anthropomorphic Model

Learning to Open and Traverse Doors with a Legged Manipulator

Avoidance Behaviors of Pedestrians in a Virtual-Reality-based Experiment

Editing Driver Character: Socially-Controllable Behavior Generation for Interactive Traffic Simulation