Abstract:The application of deep reinforcement learning (DRL) to train an agent capable of learning control laws for pulsed jets to manipulate the wake of a bluff body is presented and discussed. The work has been performed experimentally at a value of the Reynolds number Re∼105 adopting a single-step approach for the training of the agent. Two main aspects are targeted: first, the dimension of the state, allowing us to draw conclusions on its effect on the training of the neural network; second, the capability of the agent to learn optimal strategies aimed at maximizing more complex tasks identified with the reward. The agent is trained to learn strategies that minimize drag only or minimize drag while maximizing the power budget of the fluidic system. The results show that independently on the definition of the reward, the DRL learns forcing conditions that yield values of drag reduction that are as large as 10% when the reward is based on the drag minimization only. On the other hand, when also the power budget is accounted for, the agent learns forcing configurations that yield lower drag reduction (5%) but characterized by large values of the efficiency. A comparison between the natural and the forced conditions is carried out in terms of the pressure distribution across the model's base. The different structure of the wake that is obtained depending on the training of the agent suggests that the possible forcing configuration yielding similar values of the reward is local minima for the problem. This represents, to the authors' knowledge, the first application of a single-step DRL in an experimental framework at large values of the Reynolds number to control the wake of a three-dimensional bluff body.

Bluff body uses deep-reinforcement-learning trained active flow control to achieve hydrodynamic stealth

Deep reinforcement learning-based active flow control of an elliptical cylinder: transitioning from an elliptical cylinder to a circular cylinder and a flat plate

Active Flow Control for Bluff Body Drag Reduction Using Reinforcement Learning with Partial Measurements

Reinforcement Learning for Bluff Body Active Flow Control in Experiments and Simulations

Efficient Active Flow Control Strategy for Confined Square Cylinder Wake Using Deep Learning-Based Surrogate Model and Reinforcement Learning

Acoustics-based Active Control of Unsteady Flow Dynamics using Reinforcement Learning Driven Synthetic Jets

Deep reinforcement learning for active control of a three-dimensional bluff body wake

Mitigating the lift of a circular cylinder in wake flow using deep reinforcement learning guided self-rotation

Active flow control with rotating cylinders by an artificial neural network trained by deep reinforcement learning

Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning

Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing

Reducing the Drag of a Bluff Body by Deep Reinforcement Learning

Deep reinforcement transfer learning of active control for bluff body flows at high Reynolds number

Robust and Adaptive Deep Reinforcement Learning for Enhancing Flow Control around a Square Cylinder with Varying Reynolds Numbers

Biomimetic bluff body drag reduction by self-adaptive porous flaps

Wake Control of a Bluff-Body Via Distributed Jets over Its Surface

Deep reinforcement learning based synthetic jet control on disturbed flow over airfoil

Dynamic kirigami structures for wake flow control behind a circular cylinder

Active control for enhancing vortex induced vibration of a circular cylinder based on deep reinforcement learning

Deep Reinforcement Learning for Active Control of Flow over a Circular Cylinder with Rotational Oscillations