Abstract:Integrating distributed solar fields (DSFs) into conventional heat and power plants (CHPs) of industries is mostly constrained by the availability of a real-time capable control scheme. Safe and efficient operation of industrial DSF requires the supply of a fluctuating and periodically available energy at the required temperature while reducing losses and ensuring operational constraint. Reinforcement learning (RL), specifically Q-learning methods, are being recently applied for such control tasks. However, existing RL approaches cannot be directly applied to continuous domain control tasks or tasks with operation constraints. While applying deep learning in model-free RL can remove these limitations, tractability and scalability boundaries on performing the deep learning are becoming significant, with scarce data and multi-objective concerns. In contrast, model-based deep RLs can be applied for sample efficiency and satisfying operational constraints, but their modeling framework is not general and accurate. To address these challenges, the work here develops a hybrid fuzzy convolution model (HFCM) to take full advantages of data, models (dynamic and steady-state), and prior knowledge on industrial DSF. The HFCM is then extended for use in deep deterministic policy gradient (DDPG) algorithm to learn the DSF control task. This is done so by solving a multi-objective optimization problem, which is formulated as a constrained Markov decision process (CMDP) with continuous state and actions. Some practical and relevant findings were made with this HFCM. Firstly, it allowed the DDPG to learn independently a temperature state and a disturbance state, using only temperature measurement and single realizations of state vector. It also permitted the testing of an operational state violation strategy on a DDPG actions, and at the same time, the correction of the effects at a safety layer, if needed. Furthermore, the proposed control strategy simultaneously reduced mean temperature tracking error by 24%–51% and energy gain by 13.9%–17.35% when compared respectively to model-free DDPG and MPC baselines.

Supervised integrated deep deterministic policy gradient model for enhanced control of chemical processes

A Reinforcement Learning-Based Temperature Control of Fluidized Bed Reactor in Gas-Phase Polyethylene Process

Accelerating Reinforcement Learning with Local Data Enhancement for Process Control

Dynamic Modeling And Nonlinear Predictive Control Based On Partitioned Model And Nonlinear Optimization

Sensors Integrated Control of PEMFC Gas Supply System Based on Large-Scale Deep Reinforcement Learning

Model Free Deep Deterministic Policy Gradient Controller for Setpoint Tracking of Non-minimum Phase Systems

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

Advancing Process Control in Fluidized Bed Biomass Gasification Using Model-Based Deep Reinforcement Learning

Control-Informed Reinforcement Learning for Chemical Processes

Design Of Model Predictive Controller Based On Neural Systems

Deep Learning-Based Model Predictive Control for Continuous Stirred-Tank Reactor System

Fuzzy-based predictive deep reinforcement learning for robust and constrained optimal control of industrial solar thermal plants

Intelligent Controller Based on Distributed Deep Reinforcement Learning for PEMFC Air Supply System

Development of AI-based process controller of sour water treatment unit using deep reinforcement learning

FiDi-RL: Incorporating Deep Reinforcement Learning with Finite-Difference Policy Search for Efficient Learning of Continuous Control

Deep reinforcement learning-assisted extended state observer for run-to-run control in the semiconductor manufacturing process

A Novel Integral Reinforcement Learning-Based Control Method Assisted by Twin Delayed Deep Deterministic Policy Gradient for Solid Oxide Fuel Cell in DC Microgrid

Ai-based optimal control of fed-batch biopharmaceutical process leveraging deep reinforcement learning

Optimizing olefin purification: An artificial intelligence-based process-conscious PI controller tuning for double dividing wall column distillation

Reinforcement learning control method for real‐time hybrid simulation based on deep deterministic policy gradient algorithm

Solving Reach-Avoid-Stay Problems Using Deep Deterministic Policy Gradients