Abstract:Reinforcement learning is applied to the development of control strategies in order to reduce skin friction drag in a fully developed turbulent channel flow at a low Reynolds number. Motivated by the so-called opposition control (Choi et al. 1993), in which a control input is applied so as to cancel the wall-normal velocity fluctuation on a detection plane at a certain distance from the wall, we consider wall blowing and suction as a control input, and its spatial distribution is determined by the instantaneous streamwise and wall-normal velocity fluctuations at the distance of 15 wall units above the wall. Deep neural network is used to express the complex relationship between the sensing information and the control input, and it is trained so as to maximize the expected long-term reward, i.e., drag reduction. When only the wall-normal velocity fluctuation is measured and a linear network is used, the present framework successfully reproduces the optimal linear weight for the opposition control reported in the previous study (Chung & Talha 2011). In contrast, when a non-linear network is used, more complex control strategies based on the instantaneous streamwise and wall-normal velocity fluctuations are obtained. Specifically, the obtained control strategies abruptly switch between strong wall blowing and suction for downwelling of a high-speed fluid toward the wall and upwelling of a low-speed fluid away from the wall, respectively. The obtained control policies lead to drag reduction rates as high as 37 %, which is higher than 23 % achieved by the conventional opposition control at the same Reynolds number. The present results indicate that reinforcement learning can be a novel framework for the development of effective control strategies through systematic learning based on a large number of trials.

Drag-reduction strategies in wall-bounded turbulent flows using deep reinforcement learning

Deep reinforcement learning for turbulent drag reduction in channel flows

Drag Reduction in Flows Past 2D and 3D Circular Cylinders Through Deep Reinforcement Learning

Learning in two dimensions and controlling in three: Generalizable drag reduction strategies for flows past circular cylinders through deep reinforcement learning

Reinforcement Learning of Control Strategies for Reducing Skin Friction Drag in a Fully Developed Channel Flow

Towards Active Flow Control Strategies Through Deep Reinforcement Learning

Reinforcement Learning for Bluff Body Active Flow Control in Experiments and Simulations

Deep reinforcement learning for the management of the wall regeneration cycle in wall-bounded turbulent flows

Accelerating and improving deep reinforcement learning-based active flow control: Transfer training of policy network

Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at $Re_D=3900$

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach

Deep Reinforcement Learning for Active Flow Control around a Circular Cylinder Using Unsteady-mode Plasma Actuators

Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing

Active flow control for bluff body drag reduction using reinforcement learning with partial measurements

Reinforcement Learning for Active Flow Control in Experiments

Reducing the Drag of a Bluff Body by Deep Reinforcement Learning

Deep reinforcement transfer learning for active flow control of a 3D square cylinder under state dimension mismatch

Active flow control for three-dimensional cylinders through deep reinforcement learning

Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning