Abstract:Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSAC-LLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at <a class="link-external link-https" href="https://hsacllm.github.io/" rel="external noopener nofollow">this https URL</a>.

Socially Compliant Navigation Through Raw Depth Inputs with Generative Adversarial Imitation Learning

Navigation by Imitation in a Pedestrian-Rich Environment

CrowdGAIL: A spatiotemporal aware method for agent navigation

Learning World Transition Model for Socially Aware Robot Navigation

Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning

Socially compliant mobile robot navigation via inverse reinforcement learning

Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environments

Learning to Socially Navigate in Pedestrian-rich Environments with Interaction Capacity

Navigating the Human Maze: Real-Time Robot Pathfinding with Generative Imitation Learning

Learning Social Navigation from Demonstrations with Conditional Neural Processes

Learning a Group-Aware Policy for Robot Navigation

Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

Subgoal-Driven Navigation in Dynamic Environments Using Attention-Based Deep Reinforcement Learning

Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation

Learning Early Social Maneuvers for Enhanced Social Navigation

Enhancing Navigational Safety in Crowded Environments using Semantic-Deep-Reinforcement-Learning-based Navigation

Socially Aware Motion Planning with Deep Reinforcement Learning

Learning Safety-Aware Policy with Imitation Learning for Context-Adaptive Navigation

Mapless Navigation With Safety-Enhanced Imitation Learning