Abstract:Q-Learning is a mainstream reinforcement learning algorithm atpresent,but its convergence speed is poor in random environment.Previous studies have improved the overestimation problem of Spee-dy Q-Learning,and have proposed Double Speedy Q-Learning algorithm.However,the Double Speedy Q-Learning algorithm does not consider the self-loop structure exis-ting in the random environment,that is,the probability of entering the current state when the agent performs an action,which will not be conducive to the agent's learning in the random environment,thereby affecting the convergence speed of the algorithm.Aiming at the self-loop structure existing in Double Speedy Q-Learning,the Bellman operator of Double Speedy Q-Learning algorithm is improved by using successive over-relaxation technology,and the Double Speedy Q-Learning algorithm based on successive over relaxation (DSQL-SOR) is proposed to further improve the convergence speed of the Double Speedy Q-Learning algorithm.By using numerical experiments to compare the error between the actual rewards and expected rewards of DSQL-SOR and other algorithms,the experimental results show that the proposed algorithm has a lower error of 0.6 than the existing mainstream algorithm SQL,which is lower than the successive over-relaxation algorithm GSQL 0.5,indicating that the performance of the DSQL-SOR algorithm is better than other algorithms.The experiment also tests the scalability of the DSQL-SOR algorithm.When the state space is increased from 10 to 1 000,the average time of each iteration increases slowly,always maintaining at the magnitude of 10－4,indicating that DSQL-SOR has strong scalability.

Soft Q Network

Gradient Q : A Unified Algorithm with Function Approximation for Reinforcement Learning

Gradient Q(σ, Λ): A Unified Algorithm with Function Approximation for Reinforcement Learning

BrainQN: Enhancing the Robustness of Deep Reinforcement Learning with Spiking Neural Networks

Sparse Q-Learning: Offline Reinforcement Learning with Implicit Value Regularization

Double Speedy Q-Learning Based on Successive over Relaxation

Does DQN Learn?

DQN with model-based exploration: efficient learning on environments with sparse rewards

Soft Policy Gradient Method for Maximum Entropy Deep Reinforcement Learning

On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$-Greedy Exploration

Qualitative Measurements of Policy Discrepancy for Return-Based Deep Q-Network

SVQN: Sequential Variational Soft Q-Learning Networks

Multiagent Soft Q-Learning

Regularized Softmax Deep Multi-Agent Q-Learning.

Using Deep Q-Learning to Control Optimization Hyperparameters

Finite-Time Error Analysis of Soft Q-Learning: Switching System Approach

Deep Q-Learning: Theoretical Insights from an Asymptotic Analysis

XQSV: A Structurally Variable Network to Imitate Human Play in Xiangqi

Look where you look! Saliency-guided Q-networks for generalization in visual Reinforcement Learning

Graying the black box: Understanding DQNs

Local Planning Strategy Based on Deep Reinforcement Learning Over Estimation Suppression