Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).

Iterative surrogate model optimization (ISMO): An active learning algorithm for PDE constrained optimization with deep neural networks

A Direct Coupled Solution Methodology for Efficient Robust Optimizations of Inverse Problems under Uncertainty

Active-Learning-Driven Surrogate Modeling for Efficient Simulation of Parametric Nonlinear Systems

Deep Neural Network-Based Surrogate-Assisted Inverse Optimization for High-Speed Interconnects

Enhancing Multi-Objective Optimization through Machine Learning-Supported Multiphysics Simulation

Adaptive machine learning-based surrogate modeling to accelerate PDE-constrained optimization in enhanced oil recovery

Learning a Neural Solver for Parametric PDE to Enhance Physics-Informed Methods

An Improved Regularity-Based Vector Evolutionary Algorithm for Multi-Objective Optimizations.

Active learning of deep surrogates for PDEs: application to metasurface design

A Fast Optimization Algorithm Based on Multi-RBF For High-Dimensional and Computationally Expensive Problems

Data efficient surrogate modeling for engineering design: Ensemble-free batch mode deep active learning for regression

High-Fidelity Surrogate Based Multi-Objective Optimization Algorithm

ODE-based Learning to Optimize

Modeling Design and Control Problems Involving Neural Network Surrogates

AI‐enhanced iterative solvers for accelerating the solution of large‐scale parametrized systems

A Neural Network Approach for Stochastic Optimal Control

An Operator Learning Approach to Nonsmooth Optimal Control of Nonlinear PDEs

An improved data-free surrogate model for solving partial differential equations using deep neural networks

Two-Stage Surrogate Modeling for Data-Driven Design Optimization with Application to Composite Microstructure Generation

Feasibility Study on Active Learning of Smart Surrogates for Scientific Simulations

Application of deep learning based multi-fidelity surrogate model to robust aerodynamic design optimization