Abstract:Deep reinforcement learning (DRL) has gradually emerged as a novel and effective method for intelligent control of conjugate heat transfer. Through proper training, DRL agent usually can find a better control strategy than the one optimized manually. For numerous numerical simulation-based investigations, the promising results have only been obtained on 2-dimensional models due to the heavy burden of data acquisition. This paper proposes a novel strategy of transferring the control policy learned in 2-dimensional environment into a new 3-dimensional environment. PDD-DQN (Prioritized Dueling Double Deep Q-Networks) algorithm is used to recognize the underlying relationship between the forced fluid flow and heat transfer performance to produce a control strategy. The DRL controller is first trained on a simple 2-dimensional cavity with one heat source, and the DRL controller is able to reduce the maximum temperature to 315 K which is 2 K lower than the manually optimized control strategy; then the control strategy is transferred to 3-dimensional models where the maximum temperature is further reduced to 308 K; and compared to training the DRL agent directly on 3-dimensional model, the policy-transfer strategy requires only 10% computational expenditure. The proposed strategy is then applied to a more complex testbed with multiple heat sources for further investigating its ability; in both 2D and 3D environment, the DRL controller can cool the maximum temperature to be 307 K which is 8 K lower than the manually optimized control policy; and the computational cost drop to be 3%. The generalization ability of the trained DRL agent in inter-dimensional geometries is confirmed. For the cases with huge grid size, the policy-transfer strategy can economize exponential computational cost, which is of great significance for applying DRL methods to practical thermal control problems.

The Application of DQN in Thermal Process Control

A Reinforcement Learning-Based Temperature Control of Fluidized Bed Reactor in Gas-Phase Polyethylene Process

Process Controller in DQN Method

A Knowledge-based Reinforcement Learning Control Approach Using Deep Q Network for Cooling Tower in HVAC Systems

Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets

Closed-loop forced heat convection control using deep reinforcement learning

Close d-loop force d heat convection control using deep reinforcement learning

A Comparative Study of DQN and D3QN for HVAC System Optimization Control

Deep reinforcement learning optimal control strategy for temperature setpoint real-time reset in multi-zone building HVAC system

Deep Reinforcement Learning: Framework, Applications, and Embedded Implementations

Cross temporal-spatial transferability investigation of deep reinforcement learning control strategy in the building HVAC system level

Comparison of Deep Reinforcement Learning Algorithms in Data Center Cooling Management: A Case Study

A Deep Reinforcement Learning Approach to Improve the Learning Performance in Process Control

Drqn-Based Global Optimal Control of Air Conditioning Water System

Towards Optimal HVAC Control in Non-Stationary Building Environments Combining Active Change Detection and Deep Reinforcement Learning

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Control policy transfer of deep reinforcement learning based intelligent forced heat convection control

Deep Reinforcement Learning-Pid Based Supervisor Control Method for Indirect-Contact Heat Transfer Processes in Energy Systems

ED-DQN: An event-driven deep reinforcement learning control method for multi-zone residential buildings

Model predictive control guided reinforcement learning control scheme

Adaptive control for circulating cooling water system using deep reinforcement learning