Abstract:Rayleigh-Bénard convection (RBC) is a recurrent phenomenon in several industrial and geoscience flows and a well-studied system from a fundamental fluid-mechanics viewpoint. However, controlling RBC, for example by modulating the spatial distribution of the bottom-plate heating in the canonical RBC configuration, remains a challenging topic for classical control-theory methods. In the present work, we apply deep reinforcement learning (DRL) for controlling RBC. We show that effective RBC control can be obtained by leveraging invariant multi-agent reinforcement learning (MARL), which takes advantage of the locality and translational invariance inherent to RBC flows inside wide channels. The MARL framework applied to RBC allows for an increase in the number of control segments without encountering the curse of dimensionality that would result from a naive increase in the DRL action-size dimension. This is made possible by the MARL ability for re-using the knowledge generated in different parts of the RBC domain. We show in a case study that MARL DRL is able to discover an advanced control strategy that destabilizes the spontaneous RBC double-cell pattern, changes the topology of RBC by coalescing adjacent convection cells, and actively controls the resulting coalesced cell to bring it to a new stable configuration. This modified flow configuration results in reduced convective heat transfer, which is beneficial in several industrial processes. Therefore, our work both shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics. These results are useful for both gaining further understanding of the intrinsic properties of RBC, as well as for developing industrial applications.

Controlling Rayleigh–Bénard convection via reinforcement learning

Stabilizing/Destabilizing The Large-Scale Circulation In Turbulent Rayleigh-Benard Convection With Sidewall Temperature Control

Controlling flow reversal in two-dimensional Rayleigh-Benard convection

Flow Reversal and Multiple States in Turbulent Rayleigh–Bénard Convection with Partially Isothermal Plates

Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need

Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-Bénard convection

Closed-loop forced heat convection control using deep reinforcement learning

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Control policy transfer of deep reinforcement learning based intelligent forced heat convection control

Control of quasi-equilibrium state of annular flow through reinforcement learning

Adaptive temperature control of a reverse flow process by using reinforcement learning approach

Deep Reinforcement Learning for the Heat Transfer Control of Pulsating Impinging Jets

Constrained Reinforcement Learning for Safe Heat Pump Control

Model-Based Reinforcement Learning Control of Reaction-Diffusion Problems

Reinforcement learning with thermal fluctuations at the nanoscale

Marginally-Stable Thermal Equilibria of Rayleigh-Bénard Convection

Reinforcement Learning with thermal fluctuations at the nano-scale

Deep reinforcement learning for turbulent drag reduction in channel flows

Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning

Towards practical reinforcement learning for tokamak magnetic control

Steady Rayleigh–Bénard convection between no-slip boundaries