Abstract:Deep reinforcement learning (DRL) has gradually emerged as a novel and effective method for intelligent control of conjugate heat transfer. Through proper training, DRL agent usually can find a better control strategy than the one optimized manually. For numerous numerical simulation-based investigations, the promising results have only been obtained on 2-dimensional models due to the heavy burden of data acquisition. This paper proposes a novel strategy of transferring the control policy learned in 2-dimensional environment into a new 3-dimensional environment. PDD-DQN (Prioritized Dueling Double Deep Q-Networks) algorithm is used to recognize the underlying relationship between the forced fluid flow and heat transfer performance to produce a control strategy. The DRL controller is first trained on a simple 2-dimensional cavity with one heat source, and the DRL controller is able to reduce the maximum temperature to 315 K which is 2 K lower than the manually optimized control strategy; then the control strategy is transferred to 3-dimensional models where the maximum temperature is further reduced to 308 K; and compared to training the DRL agent directly on 3-dimensional model, the policy-transfer strategy requires only 10% computational expenditure. The proposed strategy is then applied to a more complex testbed with multiple heat sources for further investigating its ability; in both 2D and 3D environment, the DRL controller can cool the maximum temperature to be 307 K which is 8 K lower than the manually optimized control policy; and the computational cost drop to be 3%. The generalization ability of the trained DRL agent in inter-dimensional geometries is confirmed. For the cases with huge grid size, the policy-transfer strategy can economize exponential computational cost, which is of great significance for applying DRL methods to practical thermal control problems.

Control policy transfer of deep reinforcement learning based intelligent forced heat convection control

Closed-loop forced heat convection control using deep reinforcement learning

A Reinforcement Learning-Based Temperature Control of Fluidized Bed Reactor in Gas-Phase Polyethylene Process

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Accelerating and improving deep reinforcement learning-based active flow control: Transfer training of policy network

Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets

Policy transfer of reinforcement learning-based flow control: From two- to three-dimensional environment

Multi-source transfer learning method for enhancing the deployment of deep reinforcement learning in multi-zone building HVAC control

Deep reinforcement transfer learning for active flow control of a 3D square cylinder under state dimension mismatch

Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control

Robust active flow control over a range of Reynolds numbers using an artificial neural network trained through deep reinforcement learning

Surrogate Empowered Sim2Real Transfer of Deep Reinforcement Learning for ORC Superheat Control

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

One for Many: Transfer Learning for Building HVAC Control

Adaptive control for circulating cooling water system using deep reinforcement learning

Residual Physics and Post-Posed Shielding for Safe Deep Reinforcement Learning Method

Deep Reinforcement Learning for Heat Pump Control

Intelligent control of district heating system based on RDPG

Controlling Rayleigh–Bénard convection via reinforcement learning

Intelligent multi-zone residential HVAC control strategy based on deep reinforcement learning

Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-Bénard convection