Abstract:Robot navigation is an important research field with applications in various domains. However, traditional approaches often prioritize efficiency and obstacle avoidance, neglecting a nuanced understanding of human behavior or intent in shared spaces. With the rise of service robots, there's an increasing emphasis on endowing robots with the capability to navigate and interact in complex real-world environments. Socially aware navigation has recently become a key research area. However, existing work either predicts pedestrian movements or simply emits alert signals to pedestrians, falling short of facilitating genuine interactions between humans and robots. In this paper, we introduce the Hybrid Soft Actor-Critic with Large Language Model (HSAC-LLM), an innovative model designed for socially-aware navigation in robots. This model seamlessly integrates deep reinforcement learning with large language models, enabling it to predict both continuous and discrete actions for navigation. Notably, HSAC-LLM facilitates bidirectional interaction based on natural language with pedestrian models. When a potential collision with pedestrians is detected, the robot can initiate or respond to communications with pedestrians, obtaining and executing subsequent avoidance strategies. Experimental results in 2D simulation, the Gazebo environment, and the real-world environment demonstrate that HSAC-LLM not only efficiently enables interaction with humans but also exhibits superior performance in navigation and obstacle avoidance compared to state-of-the-art DRL algorithms. We believe this innovative paradigm opens up new avenues for effective and socially aware human-robot interactions in dynamic environments. Videos are available at <a class="link-external link-https" href="https://hsacllm.github.io/" rel="external noopener nofollow">this https URL</a>.

Advancing Behavior Generation in Mobile Robotics through High-Fidelity Procedural Simulations

HumRoboSim: an Autonomous Humanoid Robot Simulation System

FastSim: A Modular and Plug-and-Play Simulator for Aerial Robots

A Mobile Robot Experiment System with Lightweight Simulator Generator for Deep Reinforcement Learning Algorithm.

EXPLORING THE FUSION OF INTELLIGENT AGENTS AND UNITY ML: A COMPREHENSIVE ANALYSIS OF ROBOT MOTION SIMULATION

Imagined Potential Games: A Framework for Simulating, Learning and Evaluating Interactive Behaviors

SOCIALGYM 2.0: Simulator for Multi-Agent Social Robot Navigation in Shared Human Spaces

The Radiance of Neural Fields: Democratizing Photorealistic and Dynamic Robotic Simulation

NeuronsGym: A Hybrid Framework and Benchmark for Robot Tasks with Sim2Real Policy Learning

RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator

Rapid Development of a Mobile Robot Simulation Environment

Learning Interactive Real-World Simulators

Development of a Simulation Environment for Robot Soccer Game with Deep Reinforcement Learning and Role Assignment

IRASim: Learning Interactive Real-Robot Action Simulators

UNav-Sim: A Visually Realistic Underwater Robotics Simulator and Synthetic Data-generation Framework

Gen2Sim: Scaling up Robot Learning in Simulation with Generative Models

BeSimulator: A Large Language Model Powered Text-based Behavior Simulator

Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning

Sim-to-Real Transfer with Neural-Augmented Robot Simulation

Flightmare: A Flexible Quadrotor Simulator

Enhancing Socially-Aware Robot Navigation through Bidirectional Natural Language Conversation