Abstract:Path planning algorithm has always been the core of intelligent robot research; a good path planning algorithm can significantly enhance the efficiency of robots in executing tasks. As the application scenarios for intelligent robots continue to diversify, their adaptability to the environment has become a key focus in current path planning algorithm research. As one of the classic reinforcement learning algorithms, Q-learning (QL) algorithm has its inherent advantages in adapting to the environment, but it also faces various challenges and shortcomings. These issues are primarily centered around suboptimal path planning, slow convergence speed, weak generalization capability and poor obstacle avoidance performance. In order to solve these issues in the QL algorithm, we have carried out the following work. (1) We redesign the reward mechanism of QL algorithm. The traditional Q-learning algorithm's reward mechanism is simple to implement but lacks directionality. We propose a combined reward mechanism of "static assignment + dynamic adjustment." This mechanism can address the issue of random path selection and ultimately lead to optimal path planning. (2) We redesign the greedy strategy of QL algorithm. In the traditional Q-learning algorithm, the greedy factor in the strategy is either randomly generated or set manually, which limits its applicability to some extent. It is difficult to effectively applied to different physical environments and scenarios, which is the fundamental reason for the poor generalization capability of the algorithm. We propose a dynamic adjustment of the greedy factor, known as the greedy strategy, which significantly improves the efficiency of Q-learning algorithm and enhances its generalization capability so that the algorithm has a wider range of application scenarios. (3) We introduce a concept to enhance the algorithm's obstacle avoidance performance. We design the expansion distance, which pre-sets a "collision buffer" between the obstacle and agent to enhance the algorithm's obstacle avoidance performance.

An Optimized Q-Learning Algorithm Based on the Thinking of Tabu Search

Gradient Q : A Unified Algorithm with Function Approximation for Reinforcement Learning

An Improved Q-learning Algorithm Based on Exploration Region Expansion Strategy

An Improved Tentative Q Learning Algorithm for Robot Learning.

Tabu Search Exploiting Local Optimality in Binary Optimization

A Novel Experience-Based Exploration Method for Q-Learning.

A new Q-learning algorithm based on the metropolis criterion.

An optimized Q-Learning algorithm for mobile robot local path planning

An Adaptive Exploratory Q-Learning Algorithm for Multiple Target Path Planning.

Mobile robot navigation method based on improved Q-learning algorithm

Greedy exploration policy of Q-learning based on state balance

A Quasi-Oppositional Learning of Updating Quantum State and Q-learning Based on the Dung Beetle Algorithm for Global Optimization

Research on Improvement of Q-Learning and Its Simulation Experiments

Q_Learning Based on Active Backup and Memory Mechanism

Discrete-time optimal control scheme based on Q-learning algorithm

The Optimization of Path Planning for Multi-Robot System Using Boltzmann Policy Based Q-learning Algorithm

Surrogate-Assisted Evolutionary Q-Learning for Black-Box Dynamic Time-Linkage Optimization Problems

A Turbo Q-Learning (TQL) for Energy Efficiency Optimization in Heterogeneous Networks

ETQ-learning: an improved Q-learning algorithm for path planning

The Improvement of Q-learning Applied to Imperfect Information Game

Optimal Path Planning Based on Goal Heuristics IQLA for Mobile Robots