Abstract:This paper investigates the performance of several most popular deep reinforcement learning (DRL) algorithms applied to fluid flow and convective heat transfer systems, providing credible guidance and evaluation on their characteristics and performance. The studied algorithms are selected by considering the popularity, category, and advancement for guaranteeing the significance of the current study. The effectiveness and feasibility of all DRL algorithms are first demonstrated by studying a two-dimensional multi-heat-source cooling problem. Compared with the best manually optimized control, all DRL algorithms can find better control strategies that realize a further temperature reduction of 3–7 K. For problems with complex control objectives and environments, PPO (proximal policy optimization) shows an outstanding performance that accurately and dynamically constrains the oscillation of the solid temperature within 0.5 K around the target value, which is far beyond the capability of the manually optimized control. With the presented performance and the supplemented generalization test, the characteristic and specialty of the DRL algorithms are analyzed. The value-based methods have better training efficiency on simple cooling tasks with linear reward, while the policy-based methods show remarkable convergence on demanding tasks with nonlinear reward. Among the algorithms studied, the single-step PPO and prioritized experience replay deep Q-networks should be highlighted: the former has the advantage of considering multiple control targets and the latter obtains the best result in all generalization testing tasks. In addition, randomly resetting the environment is confirmed to be indispensable for the trained agent executing long-term control, which is strongly recommended to be included in follow-up studies.

Fast-apply deep autoregressive recurrent proximal policy optimization for controlling hot water systems

Deep Reinforcement Learning based Optimal Control of Hot Water Systems

Controlling Pressure of Gas Pipeline Network Based on Mixed Proximal Policy Optimization

A Reinforcement Learning-Based Temperature Control of Fluidized Bed Reactor in Gas-Phase Polyethylene Process

An adaptive control framework based on Reinforcement learning to balance energy, comfort and hygiene in heat pump water heating systems

Deep Reinforcement Learning for Autonomous Water Heater Control

Comparative Analysis of Reinforcement Learning Approaches for Multi-Objective Optimization in Residential Hybrid Energy Systems

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Real-Time Scheduling of Pumps in Water Distribution Systems Based on Exploration-Enhanced Deep Reinforcement Learning

Energy Consumption Optimization for Heating, Ventilation and Air Conditioning Systems Based on Deep Reinforcement Learning

Deep Reinforcement Learning for Real-Time Energy Management in Smart Home

A data-driven DRL-based home energy management system optimization framework considering uncertain household parameters

Reinforcement Learning Applied to an Electric Water Heater: From Theory to Practice

Intelligent control of district heating system based on RDPG

Reinforcement Learning for proactive operation of residential energy systems by learning stochastic occupant behavior and fluctuating solar energy: Balancing comfort, hygiene and energy use

Adaptive control for circulating cooling water system using deep reinforcement learning

Demand response optimization strategy of electric water heater based on RL

Hybrid-Model-Based Deep Reinforcement Learning for Heating, Ventilation, and Air-Conditioning Control

USING DEEP LEARNING TO OPTIMIZE HVAC SYSTEMS IN RESIDENTIAL BUILDINGS

Interior-point policy optimization based multi-agent deep reinforcement learning method for secure home energy management under various uncertainties