Abstract:As a promising machine learning method for active flow control (AFC), deep reinforcement learning (DRL) has been successfully applied in various scenarios, such as the drag reduction for stationary cylinders under both laminar and weakly turbulent conditions. However, current applications of DRL in AFC still suffer from drawbacks including excessive sensor usage, unclear search paths and insufficient robustness tests. In this study, we aim to tackle these issues by applying DRL-guided self-rotation to suppress the vortex-induced vibration (VIV) of a circular cylinder under the lock-in condition. With a state space consisting only of the acceleration, velocity and displacement of the cylinder, the DRL agent learns an effective control strategy that successfully suppresses the VIV amplitude by $99.6\,\%$ . Through systematic comparisons between different combinations of sensory-motor cues as well as sensitivity analysis, we identify three distinct stages of the search path related to the flow physics, in which the DRL agent adjusts the amplitude, frequency and phase lag of the actions. Under the deterministic control, only a little forcing is required to maintain the control performance, and the VIV frequency is only slightly affected, showing that the present control strategy is distinct from those utilizing the lock-on effect. Through dynamic mode decomposition analysis, we observe that the growth rates of the dominant modes in the controlled case all become negative, indicating that DRL remarkably enhances the system stability. Furthermore, tests involving various Reynolds numbers and upstream perturbations confirm that the learned control strategy is robust. Finally, the present study shows that DRL is capable of controlling VIV with a very small number of sensors, making it effective, efficient, interpretable and robust. We anticipate that DRL could provide a general framework for AFC and a deeper understanding of the underlying physics.

DRLinFluids---An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM

DRLinSPH: An open-source platform using deep reinforcement learning and SPHinXsys for fluid-structure-interaction problems

DRLFluent: A Distributed Co-Simulation Framework Coupling Deep Reinforcement Learning with Ansys-Fluent on High-Performance Computing Systems

Applying Deep Reinforcement Learning To Active Flow Control In Weakly Turbulent Conditions

An efficient intrusive deep reinforcement learning framework for OpenFOAM

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization

Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

Optimal parallelization strategies for active flow control in deep reinforcement learning-based computational fluid dynamics

A Review on Bio-inspired Fluid Mechanics via Deep Reinforcement Learning.

Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Closed-loop forced heat convection control using deep reinforcement learning

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Close d-loop force d heat convection control using deep reinforcement learning

Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning

Deep Reinforcement Learning Finds a New Strategy for Vortex-Induced Vibration Control

Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach

Adaptive control of transonic buffet and buffeting flow with deep reinforcement learning

Exploiting Locality and Translational Invariance to Design Effective Deep Reinforcement Learning Control of the 1-Dimensional Unstable Falling Liquid Film