Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).

A neural network approach to high-dimensional optimal switching problems with jumps in energy markets

A probabilistic numerical method for optimal multiple switching problem and application to investments in electricity generation

A Machine Learning Algorithm for Finite-Horizon Stochastic Control Problems in Economics

Deep learning algorithms for FBSDEs with jumps: Applications to option pricing and a MFG model for smart grids

A Neural Network Approach for Stochastic Optimal Control

Neural Jump Stochastic Differential Equations

Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network

Neural Network to Support Unstructured Economic Decision-Process

Alternating mixed-integer programming and neural network training for approximating stochastic two-stage problems

Deep multitask neural networks for solving some stochastic optimal control problems

A Neural Network-Based Distributional Constraint Learning Methodology for Mixed-Integer Stochastic Optimization

A forward scheme with machine learning for forward-backward SDEs with jumps by decoupling jumps

Approaching Globally Optimal Energy Efficiency in Interference Networks via Machine Learning

Learning-Assisted Optimization for Transmission Switching

A deep solver for BSDEs with jumps

Machine Learning and Hamilton-Jacobi-Bellman Equation for Optimal Decumulation: a Comparison Study

Forward-Backward Stochastic Neural Networks: Deep Learning of High-dimensional Partial Differential Equations

Neural Risk Limiting Dispatch in Power Networks: Formulation and Generalization Guarantees

Neural networks in day-ahead electricity price forecasting: Single vs. multiple outputs

Neural network as a simulation metamodel in economic analysis of risky projects

Jump Diffusion-Informed Neural Networks with Transfer Learning for Accurate American Option Pricing under Data Scarcity