Abstract:As a promising machine learning method for active flow control (AFC), deep reinforcement learning (DRL) has been successfully applied in various scenarios, such as the drag reduction for stationary cylinders under both laminar and weakly turbulent conditions. However, current applications of DRL in AFC still suffer from drawbacks including excessive sensor usage, unclear search paths and insufficient robustness tests. In this study, we aim to tackle these issues by applying DRL-guided self-rotation to suppress the vortex-induced vibration (VIV) of a circular cylinder under the lock-in condition. With a state space consisting only of the acceleration, velocity and displacement of the cylinder, the DRL agent learns an effective control strategy that successfully suppresses the VIV amplitude by $99.6\,\%$ . Through systematic comparisons between different combinations of sensory-motor cues as well as sensitivity analysis, we identify three distinct stages of the search path related to the flow physics, in which the DRL agent adjusts the amplitude, frequency and phase lag of the actions. Under the deterministic control, only a little forcing is required to maintain the control performance, and the VIV frequency is only slightly affected, showing that the present control strategy is distinct from those utilizing the lock-on effect. Through dynamic mode decomposition analysis, we observe that the growth rates of the dominant modes in the controlled case all become negative, indicating that DRL remarkably enhances the system stability. Furthermore, tests involving various Reynolds numbers and upstream perturbations confirm that the learned control strategy is robust. Finally, the present study shows that DRL is capable of controlling VIV with a very small number of sensors, making it effective, efficient, interpretable and robust. We anticipate that DRL could provide a general framework for AFC and a deeper understanding of the underlying physics.

Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions

Applying Deep Reinforcement Learning To Active Flow Control In Weakly Turbulent Conditions

Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

Active Flow Control Using Machine Learning: A Brief Review

Deep Reinforcement Learning Finds a New Strategy for Vortex-Induced Vibration Control

Flow Control in Wings and Discovery of Novel Approaches via Deep Reinforcement Learning

Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach

Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization

Towards Active Flow Control Strategies Through Deep Reinforcement Learning

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

A Review on Bio-inspired Fluid Mechanics via Deep Reinforcement Learning.

Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality

Robust and Adaptive Deep Reinforcement Learning for Enhancing Flow Control around a Square Cylinder with Varying Reynolds Numbers

Reinforcement learning for wind-farm flow control: Current state and future actions

An application of data driven reward of deep reinforcement learning by dynamic mode decomposition in active flow control

Reinforcement Learning for Active Flow Control in Experiments

Deep Reinforcement Learning for Active Flow Control around a Circular Cylinder Using Unsteady-mode Plasma Actuators

Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning

Optimal parallelization strategies for active flow control in deep reinforcement learning-based computational fluid dynamics

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning