Abstract:Path planning algorithm has always been the core of intelligent robot research; a good path planning algorithm can significantly enhance the efficiency of robots in executing tasks. As the application scenarios for intelligent robots continue to diversify, their adaptability to the environment has become a key focus in current path planning algorithm research. As one of the classic reinforcement learning algorithms, Q-learning (QL) algorithm has its inherent advantages in adapting to the environment, but it also faces various challenges and shortcomings. These issues are primarily centered around suboptimal path planning, slow convergence speed, weak generalization capability and poor obstacle avoidance performance. In order to solve these issues in the QL algorithm, we have carried out the following work. (1) We redesign the reward mechanism of QL algorithm. The traditional Q-learning algorithm's reward mechanism is simple to implement but lacks directionality. We propose a combined reward mechanism of "static assignment + dynamic adjustment." This mechanism can address the issue of random path selection and ultimately lead to optimal path planning. (2) We redesign the greedy strategy of QL algorithm. In the traditional Q-learning algorithm, the greedy factor in the strategy is either randomly generated or set manually, which limits its applicability to some extent. It is difficult to effectively applied to different physical environments and scenarios, which is the fundamental reason for the poor generalization capability of the algorithm. We propose a dynamic adjustment of the greedy factor, known as the greedy strategy, which significantly improves the efficiency of Q-learning algorithm and enhances its generalization capability so that the algorithm has a wider range of application scenarios. (3) We introduce a concept to enhance the algorithm's obstacle avoidance performance. We design the expansion distance, which pre-sets a "collision buffer" between the obstacle and agent to enhance the algorithm's obstacle avoidance performance.

A modified Q-learning algorithm for robot path planning in a digital twin assembly system

Research on robot path planning based on improved A* algorithm and DWA

Path Planning Optimization for Teaching and Playback Welding Robot

A Digital Twin-Driven Guidance Method for Human-Machine Collaborative Assembly Operations Based on Machine Learning and Computer Vision

Multi‐robot path planning based on a deep reinforcement learning DQN algorithm

A Motion Planning Method for Visual Servoing Using Deep Reinforcement Learning in Autonomous Robotic Assembly

Digital Twin Implementation of Autonomous Planning Arc Welding Robot System

Improved Robot Path Planning Method Based on Deep Reinforcement Learning

A digital twin-driven human–robot collaborative assembly-commissioning method for complex products

Path Planning for Autonomous Vehicles in Unknown Dynamic Environment Based on Deep Reinforcement Learning

A digital twin-driven dynamic path planning approach for multiple automatic guided vehicles based on deep reinforcement learning

A Path-Planning Approach Based on Potential and Dynamic Q-Learning for Mobile Robots in Unknown Environment

A digital twin-based frame work for task planning and robot programming in HRC

ETQ-learning: an improved Q-learning algorithm for path planning

Digital Twin-Based Task Rescheduling for Robotic Assembly Line

Study on intelligent assembly process planning and execution system based on digital twin

Cyber-Physical System Enabled Path Planning Simulation for Collaborative Industrial Robots

Robot Path Planning Research Incorporating Improved A* Algorithm and DWA Algorithm

Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment

Improved Algorithm for Robot Online Path Planning

Genetic Algorithm-Based Trajectory Optimization for Digital Twin Robots