Abstract:Recent advancements in artificial intelligence and deep learning offer tremendous opportunities to tackle high-dimensional and challenging problems. Particularly, deep reinforcement learning (DRL) has been shown to be able to address optimal decision-making problems and control complex dynamical systems. DRL has received increased attention in the realm of computational fluid dynamics (CFD) due to its demonstrated ability to optimize complex flow control strategies. However, DRL algorithms often suffer from low sampling efficiency and require numerous interactions between the agent and the environment, necessitating frequent data exchanges. One significant bottleneck in coupled DRL–CFD algorithms is the extensive data communication between DRL and CFD codes. Non-intrusive algorithms where the DRL agent treats the CFD environment as a black box may come with the deficiency of increased computational cost due to overhead associated with the information exchange between the two DRL and CFD modules. In this article, a TensorFlow-based intrusive DRL–CFD framework is introduced where the agent model is integrated within the open-source CFD solver OpenFOAM. The integration eliminates the need for any external information exchange during DRL episodes. The framework is parallelized using the message passing interface to manage parallel environments for computationally intensive CFD cases through distributed computing. The performance and effectiveness of the framework are verified by controlling the vortex shedding behind two and three-dimensional cylinders, achieved as a result of minimizing drag and lift forces through an active flow control mechanism. The simulation results indicate that the trained controller can stabilize the flow and effectively mitigate the vortex shedding.

Deep reinforcement learning for propulsive performance of a flapping foil

Learn to Flap: Foil Non-parametric Path Planning via Deep Reinforcement Learning

Precision Landing of Autonomous Parafoil System Via Deep Reinforcement Learning

Deep reinforcement learning based synthetic jet control on disturbed flow over airfoil

A field programmable gate array-based deep reinforcement learning framework for experimental active flow control and its application in airfoil flow separation elimination

Deep Learning Based Adaptive Deformation of Aerodynamic Shape for Ducted Propellers

Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control

Learning swimming via deep reinforcement learning

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

Surrogate model-based deep reinforcement learning for experimental study of active flow control of circular cylinder

Deep Reinforcement Learning for Active Flow Control around a Circular Cylinder Using Unsteady-mode Plasma Actuators

Learning the aerodynamic design of supercritical airfoils through deep reinforcement learning

Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing

Deep reinforcement transfer learning for active flow control of a 3D square cylinder under state dimension mismatch

Towards Active Flow Control Strategies Through Deep Reinforcement Learning

A deep reinforcement learning control approach for high-performance aircraft

An efficient intrusive deep reinforcement learning framework for OpenFOAM

Deep Learning Models for Flapping Fin Unmanned Underwater Vehicle Control System Gait Optimization

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Physics-informed reinforcement learning for motion control of a fish-like swimming robot

Mitigating the lift of a circular cylinder in wake flow using deep reinforcement learning guided self-rotation