Abstract:Deep reinforcement learning based on the asynchronous method is a new kind of reinforcement learning. It takes a multithreading way to enable multiple agents to update the parameters asynchronously in different exploration spaces. In this way, agents no longer need experience to reply and can update parameters online. At the same time, the asynchronous method can greatly improve the convergence speed of the algorithms and significantly improve the convergence performance of the algorithms. Asynchronous deep reinforcement learning algorithms, especially asynchronous advantage actor-critic algorithm, are very effective in solving practical problems and have been widely used. However, in existing asynchronous deep reinforcement learning algorithms, when each thread pushes updates to the global thread, it adopts a uniform learning rate, and fails to take account of the different information transmitted by different threads at each update. When the update of the agent to global thread is more biased towards failure information, it has no obvious help to update the parameters of the learning system. Therefore, we introduce the dynamic weights to asynchronous deep reinforcement learning algorithms and propose a new reinforcement learning algorithm named asynchronous advantage actor-critic with dynamic updating weights (DWA3C). When the information pushed by an agent is obviously helpful for the improvement of the system performance, we will enhance the update range, otherwise, we will weaken that. In this way, we can significantly improve the convergence efficiencies and convergence performances of the asynchronous deep reinforcement learning algorithms. And we also test the effectiveness of the algorithm through experiments. The experimental results show that, in the same running time, the proposed algorithm can significantly improve the convergence efficiency and convergence performance compared with the existing algorithms.

An Accelerated Asynchronous Advantage Actor-Critic Algorithm Applied in Papermaking

Neuron Controller With Adaptive Regulated Gain For Paper Machine

Cross-directional control strategy and its parallel implementation of paper-making processes

The Application Of Gpc And Its Parallel Implementation For Cross-Directional Basis Weight Profile Control In Paper Machines

Cross-directional Modeling and Control of Paper-Making Processes in the Discrete Wavelet Transform Domain

Applications of Asynchronous Deep Reinforcement Learning Based on Dynamic Updating Weights

Modeling and Control of Basis Weight and Moisture in Papermaking Process Using Artificial Neural Networks

An Improved Time Delay DMC and Its Application

Application of GPC and its parallel implementation for cross-directional basis weight profile control in paper machines

Mixed-Framework-Based Energy Optimization of Chemi-Mechanical Pulping

Achieving Optimal Paper Properties: A Layered Multiscale kMC and LSTM-ANN-Based Control Approach for Kraft Pulping

Towards Understanding Asynchronous Advantage Actor-critic: Convergence and Linear Speedup

MD‐CD Interaction Modeling and Control of Paper Machines

Virtual process control system and its employment on the pulp & paper industry

A High Performance Control System for Pressurized Headbox of Paper Machines

Block Discouple Control of Cross-Directional Basis Weight on Paper Machines*

Asynchronous Credit Assignment Framework for Multi-Agent Reinforcement Learning

A multi-level action coupling reinforcement learning approach for online two-stage flexible assembly flow shop scheduling

Modeling and Optimization of Paper-making Wastewater Treatment Based on Reinforcement Learning

Agent Modeling as Auxiliary Task for Deep Reinforcement Learning

Adaptive recursive kernel learning with application to online quality prediction of industrial rubber mixing process