Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).

Solving American option optimal control problems in financial markets using a novel neural network

Solving the optimal stopping problem with reinforcement learning: an application in financial option exercise

Optimal Control for Constrained Discrete-Time Nonlinear Systems Based on Safe Reinforcement Learning.

Neural Network Solution for General Nonlinear Optimization Problems

Deep learning of optimal exercise boundaries for American options

American Call Options Pricing With Modular Neural Networks

Financial option valuation by unsupervised learning with artificial neural networks

Optimal approximations for the free boundary problems of the space-time fractional Black-Scholes equations using a combined physics-informed neural network

Jump Diffusion-Informed Neural Networks with Transfer Learning for Accurate American Option Pricing under Data Scarcity

Pricing Options and Computing Implied Volatilities using Neural Networks

A Novel Numerical Approach of Computing American Option.

Simultaneous upper and lower bounds of American-style option prices with hedging via neural networks

Adaptive Neural Network Surrogate Model for Solving the Implied Volatility of Time-Dependent American Option Via Bayesian Inference

Neural network-based finite-horizon optimal control of uncertain affine nonlinear discrete-time systems

Reinforcement Learning-Based Control for Nonlinear Discrete-Time Systems with Unknown Control Directions and Control Constraints

Option Pricing by Neural Stochastic Differential Equations: A Simulation-Optimization Approach

On Calibration Neural Networks for extracting implied information from American options

Optimal Control Operator Perspective and a Neural Adaptive Spectral Method

A Neural Network Approach for Stochastic Optimal Control

Neural network-based finite horizon stochastic optimal control design for nonlinear networked control systems

Pontryagin Optimal Control via Neural Networks