Abstract:Deep reinforcement learning based on the asynchronous method is a new kind of reinforcement learning. It takes a multithreading way to enable multiple agents to update the parameters asynchronously in different exploration spaces. In this way, agents no longer need experience to reply and can update parameters online. At the same time, the asynchronous method can greatly improve the convergence speed of the algorithms and significantly improve the convergence performance of the algorithms. Asynchronous deep reinforcement learning algorithms, especially asynchronous advantage actor-critic algorithm, are very effective in solving practical problems and have been widely used. However, in existing asynchronous deep reinforcement learning algorithms, when each thread pushes updates to the global thread, it adopts a uniform learning rate, and fails to take account of the different information transmitted by different threads at each update. When the update of the agent to global thread is more biased towards failure information, it has no obvious help to update the parameters of the learning system. Therefore, we introduce the dynamic weights to asynchronous deep reinforcement learning algorithms and propose a new reinforcement learning algorithm named asynchronous advantage actor-critic with dynamic updating weights (DWA3C). When the information pushed by an agent is obviously helpful for the improvement of the system performance, we will enhance the update range, otherwise, we will weaken that. In this way, we can significantly improve the convergence efficiencies and convergence performances of the asynchronous deep reinforcement learning algorithms. And we also test the effectiveness of the algorithm through experiments. The experimental results show that, in the same running time, the proposed algorithm can significantly improve the convergence efficiency and convergence performance compared with the existing algorithms.

An Effective Asynchronous Framework for Small Scale Reinforcement Learning Problems

Asynchronous reinforcement learning algorithms for solving discrete space path planning problems

A New Asynchronous Architecture for Tabular Reinforcement Learning Algorithms

Learning to Cooperate: Application of Deep Reinforcement Learning for Online AGV Path Finding.

Asynchronous Methods for Deep Reinforcement Learning

Applications of Asynchronous Deep Reinforcement Learning Based on Dynamic Updating Weights

A Parallel Scheduling Algorithm for Reinforcement Learning in Large State Space

A Distributed Path Planning Algorithm via Reinforcement Learning

Asynchronous Methods for Model-Based Reinforcement Learning

Robot Path Planning in Dynamic Environments Based on Deep Reinforcement Learning

Asynchronous Reinforcement Learning Framework for Net Order Exploration in Detailed Routing

A Data-efficiency Training Framework for Deep Reinforcement Learning

Efficient and scalable reinforcement learning for large-scale network control

A Hierarchical SLAM Framework Based on Deep Reinforcement Learning for Active Exploration

Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations

Enhancing Offline Coverage Path Planning with Deep Reinforcement Learning

An Offline-Transfer-Online Framework for Cloud-Edge Collaborative Distributed Reinforcement Learning

Asynchronous Episodic Deep Deterministic Policy Gradient: Toward Continuous Control in Computationally Complex Environments

Planning, Fast and Slow: Online Reinforcement Learning with Action-Free Offline Data Via Multiscale Planners

Policy evaluation for reinforcement learning over asynchronous multi-agent networks

Research on Game-Playing Agents Based on Deep Reinforcement Learning.