Abstract:In the context of intelligent manufacturing in the process industry, traditional model-based optimization control methods cannot adapt to the situation of drastic changes in working conditions or operating modes. Reinforcement learning (RL) directly achieves the control objective by interacting with the environment, and has significant advantages in the presence of uncertainty since it does not require an explicit model of the operating plant. However, most RL algorithms fail to retain transfer learning capabilities in the presence of mode variation, which becomes a practical obstacle to industrial process control applications. To address these issues, we design a framework that uses local data augmentation to improve the training efficiency and transfer learning (adaptability) performance. Therefore, this paper proposes a novel RL control algorithm, CBR-MA-DDPG, organically integrating case-based reasoning (CBR), model-assisted (MA) experience augmentation, and deep deterministic policy gradient (DDPG). When the operating mode changes, CBR-MA-DDPG can quickly adapt to the varying environment and achieve the desired control performance within several training episodes. Experimental analyses on a continuous stirred tank reactor (CSTR) and an organic Rankine cycle (ORC) demonstrate the superiority of the proposed method in terms of both adaptability and control performance/robustness. The results show that the control performance of the CBR-MA-DDPG agent outperforms the conventional PI and MPC control schemes, and that it has higher training efficiency than the state-of-the-art DDPG, TD3, and PPO algorithms in transfer learning scenarios with mode shift situations.

Where Reinforcement Learning Meets Process Control: Review and Guidelines

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

A review On reinforcement learning: Introduction and applications in industrial process control

Reinforcement Learning in Process Industries: Review and Perspective

Facilitating Reinforcement Learning for Process Control Using Transfer Learning: Perspectives

Human operator decision support for highly transient industrial processes: a reinforcement learning approach

Integrating process design and control using reinforcement learning

Deep reinforcement learning for process design: Review and perspective

Optimal data-driven control of manufacturing processes using reinforcement learning: an application to wire arc additive manufacturing

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Ai-based optimal control of fed-batch biopharmaceutical process leveraging deep reinforcement learning

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

MFRL-BI: Design of a Model-free Reinforcement Learning Process Control Scheme by Using Bayesian Inference

Transfer learning for process design with reinforcement learning

Machine Learning for Process Control of (Bio)Chemical Processes

Automated Reinforcement Learning: An Overview

Deep Reinforcement Learning

FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control

Reinforcement learning for batch bioprocess optimization

Overcoming Challenges of Applying Reinforcement Learning for Intelligent Vehicle Control

Deep Reinforcement Learning for Process Synthesis