Abstract:Abstract Efficient exploration of design spaces is highly sought after in engineering applications. A spectrum of tools has been proposed to deal with the computational difficulties associated with such problems. In the context of our case study, these tools can be broadly classified into optimization and supervised learning approaches. Optimization approaches, while successful, are inherently data inefficient, with evolutionary optimization-based methods being a good example. This inefficiency stems from data not being reused from previous design explorations. Alternately, supervised learning-based design paradigms are data efficient. However, the quality of ensuing solutions depends heavily on the quality of data available. Furthermore, it is difficult to incorporate physics models and domain knowledge aspects of design exploration into pure-learning-based methods. In this work, we formulate a reinforcement learning (RL)-based design framework that mitigates disadvantages of both approaches. Our framework simultaneously finds solutions that are more efficient compared with supervised learning approaches while using data more efficiently compared with genetic algorithm (GA)-based optimization approaches. We illustrate our framework on a problem of microfluidic device design for flow sculpting, and our results show that a single generic RL agent is capable of exploring the solution space to achieve multiple design objectives. Additionally, we demonstrate that the RL agent can be used to solve more complex problems using a targeted refinement step. Thus, we address the data efficiency limitation of optimization-based methods and the limited data problem of supervised learning-based methods. The versatility of our framework is illustrated by utilizing it to gain domain insights and to incorporate domain knowledge. We envision such RL frameworks to have an impact on design science.

A reinforcement learning approach with masked agents for chemical process flowsheet design

Successive Convex Approximation Based Off-Policy Optimization for Constrained Reinforcement Learning

Combining Reinforcement Learning with Mathematical Programming: an Approach for Optimal Design of Heat Exchanger Networks

Flowsheet synthesis through hierarchical reinforcement learning and graph neural networks

Sample-efficient multi-agent reinforcement learning with masked reconstruction

Safe reinforcement learning method integrating process knowledge for real-time scheduling of gas supply network

Transfer learning for process design with reinforcement learning

Deep reinforcement learning uncovers processes for separating azeotropic mixtures without prior knowledge

Distributional Reinforcement Learning for Scheduling of Chemical Production Processes

Heuristic Algorithm-based Action Masking Reinforcement Learning (HAAM-RL) with Ensemble Inference Method

A Deep Reinforcement Learning Approach to the Flexible Flowshop Scheduling Problem with Makespan Minimization

Accelerating reinforcement learning with case-based model-assisted experience augmentation for process control

Perimeter Control Using Deep Reinforcement Learning: A Model-free Approach towards Homogeneous Flow Rate Optimization

Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization

A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting

State-wise Constrained Policy Optimization

Searching for High-Value Molecules Using Reinforcement Learning and Transformers

Integrating process design and control using reinforcement learning

Single-step deep reinforcement learning for open-loop control of laminar and turbulent flows

Efficient Domain Coverage for Vehicles with Second-Order Dynamics via Multi-Agent Reinforcement Learning

Value-Added Chemical Discovery Using Reinforcement Learning