Abstract:3D LiDAR sensors can provide 3D point clouds of the environment, and are widely used in automobile navigation; while 2D LiDAR sensors can only provide point cloud in a 2D sweeping plane, and then are only used for navigating robots of small height, e.g., floor mopping robots. In this letter, we propose a simple yet effective deep reinforcement learning (DRL) method with our self-state-attention unit and give a solution that can use low-cost devices (i.e., a 2D LiDAR sensor and a monocular camera) to navigate a tall mobile robot of one meter height. The overrall pipeline is that we (1) infer the dense depth information of RGB images with the aid of the 2D LiDAR sensor data (i.e., point clouds in a plane with fixed height), (2) further filter the dense depth map into a 2D minimal depth data and fuse with 2D LiDAR data, and (3) make use of DRL module with our self-state-attention unit to a partially observable sequential decision making problem that can deal with partially accurate data. We present a novel DRL training scheme for robot navigation, proposing a concise and effective self-state-attention unit and proving that applying this unit can replace multi-stage training, achieve better results and generalization capability. Experiments on both simulated data and a real robot show that our method can perform efficient collision avoidance only using low-cost 2D LiDAR sensor and monocular camera.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to use low - cost 2D LiDAR sensors and monocular cameras to achieve high - efficiency obstacle avoidance in robot navigation. Specifically, the paper proposes a method based on deep reinforcement learning (DRL), combined with self - state - attention unit and sensor fusion technology, to improve the obstacle - avoidance ability of robots. This method is especially suitable for mobile robots with a relatively high height, such as the multi - functional robot shown in Figure 1. ### Main Problems 1. **Application of Low - Cost Sensors**: Existing obstacle - avoidance methods usually rely on expensive 3D LiDAR sensors, while this paper aims to use low - cost 2D LiDAR sensors and monocular cameras to achieve efficient obstacle avoidance. 2. **Decision - Making in Partially Observable Environments**: In actual environments, robots can often only obtain partially accurate data. How to make effective obstacle - avoidance decisions in such cases is a challenge. 3. **Reward Weight Adjustment**: In scenarios with different levels of complexity, it is very difficult to adjust the reward weights, which affects the generalization ability of the model. ### Solutions 1. **Depth Estimation and Sensor Fusion**: - Use 2D LiDAR sensor data to assist in the dense depth estimation of RGB images. - Compress the estimated dense depth map into 2D minimum - depth data and fuse it with 2D LiDAR data to generate enhanced depth - perception information. 2. **Self - State - Attention Unit**: - Introduce a self - state - attention unit to dynamically adjust the importance of input data according to the type, target position, and speed of the robot. - In this way, complex multi - stage training can be avoided, and the generalization ability and obstacle - avoidance effect of the model can be improved. 3. **Deep Reinforcement Learning Framework**: - Design a simple DRL framework based on the partially observable Markov decision process (POMDP). - Use the Proximal Policy Optimization (PPO) algorithm for training and optimize the loss function. ### Experimental Verification 1. **Accuracy of Depth Estimation**: - Conduct experiments on the NYU - Depth - v2 dataset to verify the accuracy of the proposed depth - estimation method. - The results show that using 2D LiDAR sensor data significantly improves the accuracy of depth estimation and is superior to existing methods. 2. **Performance of the DRL Framework**: - Evaluate the performance of the DRL framework in multiple test scenarios, including static scenarios, dynamic scenarios, and randomly complex obstacle scenarios. - The experimental results show that the proposed method performs excellently in terms of success rate and average number of time steps and has good generalization ability. ### Conclusion The paper proposes a low - cost obstacle - avoidance method based on deep reinforcement learning and self - state - attention unit, which successfully solves the problems of high - cost - sensor dependence and obstacle - avoidance in partially observable environments. The experimental results verify the effectiveness and generalization ability of this method.

Deep Reinforcement Learning for Robot Collision Avoidance With Self-State-Attention and Sensor Fusion

Deep-Reinforcement-Learning-Based Collision Avoidance of Autonomous Driving System for Vulnerable Road User Safety

Multimodal Deep Reinforcement Learning with Auxiliary Task for Obstacle Avoidance of Indoor Mobile Robot

Robot obstacle avoidance system using deep reinforcement learning

Collision-Free Robot Navigation in Crowded Environments using Learning based Convex Model Predictive Control

Distributed Non-Communicating Multi-Robot Collision Avoidance Via Map-Based Deep Reinforcement Learning

Realtime Collision Avoidance for Mobile Robots in Dense Crowds using Implicit Multi-sensor Fusion and Deep Reinforcement Learning

Autonomous Navigation by Mobile Robot with Sensor Fusion Based on Deep Reinforcement Learning

Deep Reinforcement Learning of Map-Based Obstacle Avoidance for Mobile Robot Navigation

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

Efficient Multi-agent Navigation with Lightweight DRL Policy

Crowd-Steer: Realtime Smooth and Collision-Free Robot Navigation in Densely Crowded Scenarios Trained using High-Fidelity Simulation

DRQN-based 3D Obstacle Avoidance with a Limited Field of View

Monocular Camera-based Complex Obstacle Avoidance via Efficient Deep Reinforcement Learning

SSRL: A Safe and Smooth Reinforcement Learning Approach for Collision Avoidance in Navigation

Deep Reinforcement Learning Based Mobile Robot Navigation Using Sensor Fusion

Ad Hoc-Obstacle Avoidance-Based Navigation System Using Deep Reinforcement Learning for Self-Driving Vehicles

The Impact of LiDAR Configuration on Goal-Based Navigation within a Deep Reinforcement Learning Framework

Research on collision avoidance algorithm of unmanned surface vehicle based on deep reinforcement learning

A Sim-to-Real Pipeline for Deep Reinforcement Learning for Autonomous Robot Navigation in Cluttered Rough Terrain