Abstract:The application of deep reinforcement learning (DRL) to train an agent capable of learning control laws for pulsed jets to manipulate the wake of a bluff body is presented and discussed. The work has been performed experimentally at a value of the Reynolds number Re∼105 adopting a single-step approach for the training of the agent. Two main aspects are targeted: first, the dimension of the state, allowing us to draw conclusions on its effect on the training of the neural network; second, the capability of the agent to learn optimal strategies aimed at maximizing more complex tasks identified with the reward. The agent is trained to learn strategies that minimize drag only or minimize drag while maximizing the power budget of the fluidic system. The results show that independently on the definition of the reward, the DRL learns forcing conditions that yield values of drag reduction that are as large as 10% when the reward is based on the drag minimization only. On the other hand, when also the power budget is accounted for, the agent learns forcing configurations that yield lower drag reduction (5%) but characterized by large values of the efficiency. A comparison between the natural and the forced conditions is carried out in terms of the pressure distribution across the model's base. The different structure of the wake that is obtained depending on the training of the agent suggests that the possible forcing configuration yielding similar values of the reward is local minima for the problem. This represents, to the authors' knowledge, the first application of a single-step DRL in an experimental framework at large values of the Reynolds number to control the wake of a three-dimensional bluff body.

Active Flow Control for Bluff Body Drag Reduction Using Reinforcement Learning with Partial Measurements

Active flow control for bluff body drag reduction using reinforcement learning with partial measurements

Reinforcement Learning for Bluff Body Active Flow Control in Experiments and Simulations

Reducing the Drag of a Bluff Body by Deep Reinforcement Learning

Deep reinforcement learning for active control of a three-dimensional bluff body wake

Machine-learning flow control with few sensor feedback and measurement noise

Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning

Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing

Deep reinforcement learning for turbulent drag reduction in channel flows

Acoustics-based Active Control of Unsteady Flow Dynamics using Reinforcement Learning Driven Synthetic Jets

Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth

Towards Active Flow Control Strategies Through Deep Reinforcement Learning

Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate

Robust and Adaptive Deep Reinforcement Learning for Enhancing Flow Control around a Square Cylinder with Varying Reynolds Numbers

Efficient Active Flow Control Strategy for Confined Square Cylinder Wake Using Deep Learning-Based Surrogate Model and Reinforcement Learning

Deep reinforcement transfer learning of active control for bluff body flows at high Reynolds number

Reinforcement Learning for Active Flow Control in Experiments

Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at $Re_D=3900$

Model-Based Reinforcement Learning for Control of Strongly-Disturbed Unsteady Aerodynamic Flows

Deep reinforcement learning based synthetic jet control on disturbed flow over airfoil

Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning