Abstract:As the large-scale integration of renewable energy into the power grid, coal-fired power units have to undertake the task of load-frequency regulation, the optimization control of the coal-fired power units is becoming increasingly important. Reinforcement learning is being used for optimization control of the process increasingly but may encounter high learning costs. In this paper, a process model-assisted optimization control framework called PMA-OC is proposed to achieve the optimization control of coal-fired power unit. In this framework, a low-precision process model developed by process operation knowledge is employed to achieve the initial control based on model predictive control algorithm (MPC) and improve the dynamic characteristics of the coordinated control system. Then the deep reinforcement learning algorithm (DRL) is utilized to improve the initial control continuously, through constantly interacting with the improved process. With the full mining of process knowledge from the power unit, the improved process offers the better training samples for DRL, reduces the learning complexity of DRL, and strengthens the learning efficiency of DRL. Finally, the proposed control framework is employed to achieve the optimization control of the coordinated control system, and simulation results indicate that the proposed control framework outperforms the existing standalone MPC and DRL methods in terms of anti-interference ability, dynamic response time, and performs well control capability. Note to Practitioners—The flexible and stable operation of coal-fired power units is of great significance to the peak load regulation and supply guarantee of the power grid. For practitioners of the coal-fired power unit, this paper proposes a process model-assisted optimization control framework to promote the process knowledge utilization. In this framework, process knowledge is used to implement preliminary control on coal-fired power units to reduce the complex dynamic characteristics of coal-fired power units. Then, the reinforcement learning algorithm is used to continuously improve the control performance of coal-fired power units by interacting with the improved process. Finally, the effectiveness of the proposed method is verified by a supercritical unit. Although the proposed method is applied to the optimal control of coal-fired power units, its ideas can be used for the reference in other complex industrial processes.

Interval Weight State Update Approach with Deep Reinforcement Learning for Dynamic Load Change Tasks in Air Separation Processes

Performance Design of a Cryogenic Air Separation Unit for Variable Working Conditions Using the Lumped Parameter Model

Get the design right, the rest is automatic: Towards an adaptive MPC controller

Application of Multi-variable Model Predictive Control to Automatic Variable-loading of Air Separation Unit

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

Learning MPC for Process Dynamic Working Condition Change Tasks under Model Mismatch

Dynamic Load Change Operation Education in Air Separation Processes Using a Multivariable and Nonlinear Model

Multi-loop Constrained Iterative Model Predictive Control Using ARX -PLS Decoupling Structure

Automatic Load Change System of Cryogenic Air Separation Process

Development of AI-based process controller of sour water treatment unit using deep reinforcement learning

Self-Evolving Offset-Free Model Predictive Control with Model-Plant Mismatch for Dynamic Working-Point Change Tasks in Industrial Processes

Safe reinforcement learning method integrating process knowledge for real-time scheduling of gas supply network

Shadow operator: Effective dynamic load change operation training in air separation processes based on industrial nonlinear MPC and Bloom's taxonomy

Human operator decision support for highly transient industrial processes: a reinforcement learning approach

Real-Time Implementation of Improved State-Space MPC for Air Supply in a Coke Furnace

State Space Model Predictive Control for Advanced Process Operation: A Review of Recent Development, New Results, and Insight

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems

Flexible Cryogenic Air Separation Unit—An Application for Low-Carbon Fossil-Fuel Plants

Process Model-Assisted Optimization Control Method Based on Model Predictive Control and Deep Reinforcement Learning for Coordinated Control System in Coal-Fired Power Unit

MPC-based Reinforcement Learning for a Simplified Freight Mission of Autonomous Surface Vehicles