Abstract:Rayleigh-Bénard convection (RBC) is a recurrent phenomenon in several industrial and geoscience flows and a well-studied system from a fundamental fluid-mechanics viewpoint. However, controlling RBC, for example by modulating the spatial distribution of the bottom-plate heating in the canonical RBC configuration, remains a challenging topic for classical control-theory methods. In the present work, we apply deep reinforcement learning (DRL) for controlling RBC. We show that effective RBC control can be obtained by leveraging invariant multi-agent reinforcement learning (MARL), which takes advantage of the locality and translational invariance inherent to RBC flows inside wide channels. The MARL framework applied to RBC allows for an increase in the number of control segments without encountering the curse of dimensionality that would result from a naive increase in the DRL action-size dimension. This is made possible by the MARL ability for re-using the knowledge generated in different parts of the RBC domain. We show in a case study that MARL DRL is able to discover an advanced control strategy that destabilizes the spontaneous RBC double-cell pattern, changes the topology of RBC by coalescing adjacent convection cells, and actively controls the resulting coalesced cell to bring it to a new stable configuration. This modified flow configuration results in reduced convective heat transfer, which is beneficial in several industrial processes. Therefore, our work both shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics. These results are useful for both gaining further understanding of the intrinsic properties of RBC, as well as for developing industrial applications.

Fluid Directed Rigid Body Control using Deep Reinforcement Learning

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning

Position-Based Fluid Control

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

Deep reinforcement transfer learning of active control for bluff body flows at high Reynolds number

Deep reinforcement learning for active control of a three-dimensional bluff body wake

Adaptive control of transonic buffet and buffeting flow with deep reinforcement learning

Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization

Model-Based Reinforcement Learning for Control of Strongly-Disturbed Unsteady Aerodynamic Flows

Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning

Flow Shape Design for Microfluidic Devices Using Deep Reinforcement Learning

Reinforcement Learning for Bluff Body Active Flow Control in Experiments and Simulations

Deep reinforcement transfer learning for active flow control of a 3D square cylinder under state dimension mismatch

Active Control of Flow over Rotating Cylinder by Multiple Jets using Deep Reinforcement Learning

Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions

Position and Attitude Tracking Control of a Biomimetic Underwater Vehicle Via Deep Reinforcement Learning

Robust and Adaptive Deep Reinforcement Learning for Enhancing Flow Control around a Square Cylinder with Varying Reynolds Numbers

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Effective control of two-dimensional Rayleigh--Bénard convection: invariant multi-agent reinforcement learning is all you need