Abstract:In the context of intelligent manufacturing in the process industry, traditional model-based optimization control methods cannot adapt to the situation of drastic changes in working conditions or operating modes. Reinforcement learning (RL) directly achieves the control objective by interacting with the environment, and has significant advantages in the presence of uncertainty since it does not require an explicit model of the operating plant. However, most RL algorithms fail to retain transfer learning capabilities in the presence of mode variation, which becomes a practical obstacle to industrial process control applications. To address these issues, we design a framework that uses local data augmentation to improve the training efficiency and transfer learning (adaptability) performance. Therefore, this paper proposes a novel RL control algorithm, CBR-MA-DDPG, organically integrating case-based reasoning (CBR), model-assisted (MA) experience augmentation, and deep deterministic policy gradient (DDPG). When the operating mode changes, CBR-MA-DDPG can quickly adapt to the varying environment and achieve the desired control performance within several training episodes. Experimental analyses on a continuous stirred tank reactor (CSTR) and an organic Rankine cycle (ORC) demonstrate the superiority of the proposed method in terms of both adaptability and control performance/robustness. The results show that the control performance of the CBR-MA-DDPG agent outperforms the conventional PI and MPC control schemes, and that it has higher training efficiency than the state-of-the-art DDPG, TD3, and PPO algorithms in transfer learning scenarios with mode shift situations.

Enhanced Reinforcement Learning in Two-Layer Economic Model Predictive Control for Operation Optimization in Dynamic Environment

A Reinforcement Learning-based Economic Model Predictive Control Framework for Autonomous Operation of Chemical Reactors

Dynamic Modeling And Nonlinear Predictive Control Based On Partitioned Model And Nonlinear Optimization

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

ILC-based Two-Layer Strategy for Economic Performance Improvement in Industrial MPC Systems

Economic Model Predictive Control of Enhanced Operation Performance for Industrial Hierarchical Systems

Enhancing Dynamic Operation Optimization Feasibility for Constrained Economic Model Predictive Control Systems

Model Predictive Control Guided Reinforcement Learning Control Scheme

Direct learning of improved control policies from historical plant data

Two‐layer dynamic economic nonlinear model predictive control for a Lithium‐ion battery charge process with random disturbances

A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control

Fast Two-Layer Nonlinear Economic Predictive Control Using Machine Learning for ORC Systems with Non-Gaussian Disturbances

Distributed Economic Model Predictive Control for an Industrial Fluid Catalytic Cracking Unit Ensuring Safe Operation

A Novel Sim2real Reinforcement Learning Algorithm for Process Control

Economic Model Predictive Control for Achieving Offset-Free Operation Performance of Industrial Constrained Systems

Reinforcement Learning-Based Model Predictive Control for Discrete-Time Systems.

Mixed Reinforcement Learning for Efficient Policy Optimization in Stochastic Environments

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

Iterative Learning Economic Model Predictive Control.

Two-Layer Model Predictive Control For Chemical Process Model With Integrating Controlled Variables