Abstract:In narrow spaces, motion planning based on the traditional hierarchical autonomous system could cause collisions due to mapping, localization, and control noises, especially for car-like Ackermann-steering robots which suffer from non-convex and non-holonomic kinematics. To tackle these problems, we leverage deep reinforcement learning which is verified to be effective in self-decision-making, to self-explore in narrow spaces without a given map and destination while avoiding collisions. Specifically, based on our Ackermann-steering rectangular-shaped ZebraT robot and its Gazebo simulator, we propose the rectangular safety region to represent states and detect collisions for rectangular-shaped robots, and a carefully crafted reward function for reinforcement learning that does not require the waypoint guidance. For validation, the robot was first trained in a simulated narrow track. Then, the well-trained model was transferred to other simulation tracks and could outperform other traditional methods including classical and learning methods. Finally, the trained model is demonstrated in the real world with our ZebraT robot.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in narrow spaces, motion planning based on traditional hierarchical autonomous systems may lead to collision problems, especially for Ackermann - steering robots (car - like robots) under non - convex and non - holonomic dynamic constraints. Specifically, the paper focuses on how to make rectangular - shaped Ackermann - steering robots autonomously explore narrow spaces and avoid collisions without a pre - given map and destination. ### Main Challenges 1. **Non - convex Dynamic Constraints**: Ackermann - steering robots cannot move sideways or turn in place, which limits their maneuverability. 2. **Collision Detection Difficulties Caused by Geometric Shape**: Since the robot is rectangular, traditional laser - scanning - based methods may lead to over - coverage problems, thus affecting the accuracy of collision detection. 3. **Lack of Guidance Information**: When exploring in narrow spaces, the robot has no pre - set map or destination. Therefore, it is necessary to design a reward function that does not depend on waypoints or destinations to guide the exploration behavior. ### Solutions To solve the above problems, the paper proposes the following methods: 1. **Rectangular Safe - Area Representation**: By introducing a collision - detection area adapted to rectangular robots, the problem of over - coverage in traditional methods is solved. This safe area can represent the state of the robot more accurately and improve the accuracy of collision detection. 2. **Well - Designed Reward Function**: In order to encourage the robot to explore without waypoints or destinations, the author designs a reward function (FOMT) consisting of four components, namely: - \(R_f\): Forward - movement reward, which encourages the robot to move in an open direction. - \(R_o\): Obstacle - distance reward, which penalizes the behavior of approaching obstacles. - \(R_m\): Center - keeping reward, which encourages the robot to stay in the center position between obstacles on both sides. - \(R_t\): Time - saving reward, which penalizes staying in the same position for a long time. ### Experimental Verification - **Simulation Training**: First, train the robot in the Gazebo simulator to verify the effectiveness of the proposed rectangular safe area and reward function. - **Cross - Environment Transfer**: Transfer the trained model to other simulation environments to evaluate its generalization ability. - **Real - World Demonstration**: Finally, test it using the ZebraT robot in the real world to show the successful application of the learned strategy. ### Summary The main contribution of the paper is to propose a new collision - detection method and reward - function design suitable for rectangular Ackermann - steering robots, enabling the robots to efficiently explore narrow spaces and avoid collisions without a map and destination.

Unguided Self-exploration in Narrow Spaces with Safety Region Enhanced Reinforcement Learning for Ackermann-steering Robots

RL and ANN Based Modular Path Planning Controller for Resource-Constrained Robots in the Indoor Complex Dynamic Environment

A safe reinforcement learning approach for autonomous navigation of mobile robots in dynamic environments

Dynamic warning zone and a short-distance goal for autonomous robot navigation using deep reinforcement learning

Safe and socially compliant robot navigation in crowds with fast-moving pedestrians via deep reinforcement learning

Safety-guided Deep Reinforcement Learning for Path Planning of Autonomous Mobile Robots

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

R-SARL: Crowd-aware Navigation Based Deep Reinforcement Learning for Nonholonomic Robot in Complex Environments

Subgoal-Driven Navigation in Dynamic Environments Using Attention-Based Deep Reinforcement Learning

Robot obstacle avoidance system using deep reinforcement learning

A Mapless Local Path Planning Approach Using Deep Reinforcement Learning Framework

Collision-Free Robot Navigation in Crowded Environments using Learning based Convex Model Predictive Control

Socially Compliant Robot Navigation in Crowded Environment by Human Behavior Resemblance Using Deep Reinforcement Learning

The Multi-Dimensional Actions Control Approach for Obstacle Avoidance Based on Reinforcement Learning

A Soft Actor-Critic Deep Reinforcement-Learning-Based Robot Navigation Method Using LiDAR

A Deep Safe Reinforcement Learning Approach for Mapless Navigation.

Learning to Navigate in Indoor Environments: from Memorizing to Reasoning

Accelerated Sim-to-Real Deep Reinforcement Learning: Learning Collision Avoidance from Human Player

Tactile Aware Dynamic Obstacle Avoidance in Crowded Environment with Deep Reinforcement Learning

Completing Explorer Games with a Deep Reinforcement Learning Framework Based on Behavior Angle Navigation

Deep Reinforcement Learning for Robot Collision Avoidance With Self-State-Attention and Sensor Fusion