Abstract:In the context of intelligent manufacturing in the process industry, traditional model-based optimization control methods cannot adapt to the situation of drastic changes in working conditions or operating modes. Reinforcement learning (RL) directly achieves the control objective by interacting with the environment, and has significant advantages in the presence of uncertainty since it does not require an explicit model of the operating plant. However, most RL algorithms fail to retain transfer learning capabilities in the presence of mode variation, which becomes a practical obstacle to industrial process control applications. To address these issues, we design a framework that uses local data augmentation to improve the training efficiency and transfer learning (adaptability) performance. Therefore, this paper proposes a novel RL control algorithm, CBR-MA-DDPG, organically integrating case-based reasoning (CBR), model-assisted (MA) experience augmentation, and deep deterministic policy gradient (DDPG). When the operating mode changes, CBR-MA-DDPG can quickly adapt to the varying environment and achieve the desired control performance within several training episodes. Experimental analyses on a continuous stirred tank reactor (CSTR) and an organic Rankine cycle (ORC) demonstrate the superiority of the proposed method in terms of both adaptability and control performance/robustness. The results show that the control performance of the CBR-MA-DDPG agent outperforms the conventional PI and MPC control schemes, and that it has higher training efficiency than the state-of-the-art DDPG, TD3, and PPO algorithms in transfer learning scenarios with mode shift situations.

Batch process control based on reinforcement learning with segmented prioritized experience replay

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

Two Dimensional (2D) Feedback Control Scheme Based on Deep Reinforcement Learning Algorithm for Nonlinear Non-repetitive Batch Processes

Efficient Reinforcement-Learning Control Algorithm Using Experience Reuse

Iterative Learning Control (ILC) Guided Reinforcement Learning Control (RLC) Scheme for Batch Processes

Reinforcement learning for batch bioprocess optimization

A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control

Optimized SAC Deep Reinforcement Learning Control for Electro-hydraulic Servo Systems

110Th Anniversary: an Overview on Learning-Based Model Predictive Control for Batch Processes

Deployable Reinforcement Learning with Variable Control Rate

A Novel Sim2real Reinforcement Learning Algorithm for Process Control

Balanced Prioritized Experience Replay in Off-Policy Reinforcement Learning

Model Predictive Control Guided Reinforcement Learning Control Scheme

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

A Novel Heterogeneous Actor-critic Algorithm with Recent Emphasizing Replay Memory

An efficient and adaptive design of reinforcement learning environment to solve job shop scheduling problem with soft actor-critic algorithm

Human operator decision support for highly transient industrial processes: a reinforcement learning approach

Iterative Batch Reinforcement Learning via Safe Diversified Model-based Policy Search

Ai-based optimal control of fed-batch biopharmaceutical process leveraging deep reinforcement learning

Learning End-to-End Visual Servoing Using an Improved Soft Actor-Critic Approach with Centralized Novelty Measurement

Sampled-data Control Through Model-Free Reinforcement Learning with Effective Experience Replay