Abstract:Deep reinforcement learning based on the asynchronous method is a new kind of reinforcement learning. It takes a multithreading way to enable multiple agents to update the parameters asynchronously in different exploration spaces. In this way, agents no longer need experience to reply and can update parameters online. At the same time, the asynchronous method can greatly improve the convergence speed of the algorithms and significantly improve the convergence performance of the algorithms. Asynchronous deep reinforcement learning algorithms, especially asynchronous advantage actor-critic algorithm, are very effective in solving practical problems and have been widely used. However, in existing asynchronous deep reinforcement learning algorithms, when each thread pushes updates to the global thread, it adopts a uniform learning rate, and fails to take account of the different information transmitted by different threads at each update. When the update of the agent to global thread is more biased towards failure information, it has no obvious help to update the parameters of the learning system. Therefore, we introduce the dynamic weights to asynchronous deep reinforcement learning algorithms and propose a new reinforcement learning algorithm named asynchronous advantage actor-critic with dynamic updating weights (DWA3C). When the information pushed by an agent is obviously helpful for the improvement of the system performance, we will enhance the update range, otherwise, we will weaken that. In this way, we can significantly improve the convergence efficiencies and convergence performances of the asynchronous deep reinforcement learning algorithms. And we also test the effectiveness of the algorithm through experiments. The experimental results show that, in the same running time, the proposed algorithm can significantly improve the convergence efficiency and convergence performance compared with the existing algorithms.

Averaged-A3C for Asynchronous Deep Reinforcement Learning.

Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup

Recursive Least Squares Advantage Actor-Critic Algorithms

Implementation of value based curiosity mechanism in Reinforcement Learning algorithm based on A3C

Jointly Pre-training with Supervised, Autoencoder, and Value Losses for Deep Reinforcement Learning

Applications of Asynchronous Deep Reinforcement Learning Based on Dynamic Updating Weights

Double A3C: Deep Reinforcement Learning on OpenAI Gym Games

Deep Reinforcement Learning with Importance Weighted A3C for QoE enhancement in Video Delivery Services

ReLU to the Rescue: Improve Your On-Policy Actor-Critic with Positive Advantages

DSAC-T: Distributional Soft Actor-Critic with Three Refinements

An Advanced Actor-Critic Algorithm for Training Video Game AI

Applying Online Expert Supervision in Deep Actor-Critic Reinforcement Learning.

A Comparative Study of Deep Reinforcement Learning Models: DQN vs PPO vs A2C

Asynchronous Episodic Deep Deterministic Policy Gradient: Toward Continuous Control in Computationally Complex Environments

Regularized Anderson Acceleration for Off-Policy Deep Reinforcement Learning

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

Playing First-Person-Shooter Games with A3C-Anticipator Network Based Agents Using Reinforcement Learning.

Average-Reward Reinforcement Learning with Trust Region Methods

Reinforcement Learning through Asynchronous Advantage Actor-Critic on a GPU

Regularization-Adapted Anderson Acceleration for Multi-Agent Reinforcement Learning

Generalizing soft actor-critic algorithms to discrete action spaces