Abstract:Abstract Efficient exploration of design spaces is highly sought after in engineering applications. A spectrum of tools has been proposed to deal with the computational difficulties associated with such problems. In the context of our case study, these tools can be broadly classified into optimization and supervised learning approaches. Optimization approaches, while successful, are inherently data inefficient, with evolutionary optimization-based methods being a good example. This inefficiency stems from data not being reused from previous design explorations. Alternately, supervised learning-based design paradigms are data efficient. However, the quality of ensuing solutions depends heavily on the quality of data available. Furthermore, it is difficult to incorporate physics models and domain knowledge aspects of design exploration into pure-learning-based methods. In this work, we formulate a reinforcement learning (RL)-based design framework that mitigates disadvantages of both approaches. Our framework simultaneously finds solutions that are more efficient compared with supervised learning approaches while using data more efficiently compared with genetic algorithm (GA)-based optimization approaches. We illustrate our framework on a problem of microfluidic device design for flow sculpting, and our results show that a single generic RL agent is capable of exploring the solution space to achieve multiple design objectives. Additionally, we demonstrate that the RL agent can be used to solve more complex problems using a targeted refinement step. Thus, we address the data efficiency limitation of optimization-based methods and the limited data problem of supervised learning-based methods. The versatility of our framework is illustrated by utilizing it to gain domain insights and to incorporate domain knowledge. We envision such RL frameworks to have an impact on design science.

A Review on Bio-inspired Fluid Mechanics via Deep Reinforcement Learning.

Deep Reinforcement Learning: A New Beacon for Intelligent Active Flow Control

Deep reinforcement learning in fluid mechanics: A promising method for both active flow control and shape optimization

Recent advances in applying deep reinforcement learning for flow control: perspectives and future directions

Applying Deep Reinforcement Learning To Active Flow Control In Weakly Turbulent Conditions

DRLinFluids---An open-source python platform of coupling Deep Reinforcement Learning and OpenFOAM

Active Flow Control Using Machine Learning: A Brief Review

How to Control Hydrodynamic Force on Fluidic Pinball via Deep Reinforcement Learning

Deep Learning for Fluid Velocity Field Estimation: A Review

Flow Shape Design for Microfluidic Devices Using Deep Reinforcement Learning

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting

Accelerating Deep Reinforcement Learning strategies of Flow Control through a multi-environment approach

Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning

A trans-disciplinary review of deep learning research for water resources scientists

Performance analysis of reinforcement learning algorithms on intelligent closed-loop control on fluid flow and convective heat transfer

Recent Advances on Machine Learning for Computational Fluid Dynamics: A Survey

Deep reinforcement learning challenges and opportunities for urban water systems

Recent Progress of Machine Learning in Flow Modeling and Active Flow Control

Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning

An application of data driven reward of deep reinforcement learning by dynamic mode decomposition in active flow control