Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).

A novel consensus PSO-assisted trajectory unified and trust-tech methodology for DNN training and its applications

A Novel Trajectory Planning Method Based on Trust Region Policy Optimization

Design and Analysis of a Novel Distributed Gradient Neural Network for Solving Consensus Problems in a Predefined Time

Joint Neural Network for Trajectory and Communication Design in Multi-DAV Systems

Learning to Plan Maneuverable and Agile Flight Trajectory with Optimization Embedded Networks

Adaptive Partitioning and Efficient Scheduling for Distributed DNN Training in Heterogeneous IoT Environment

Deep Neural Networks for Improved, Impromptu Trajectory Tracking of Quadrotors

Design of an Effective Trajectory Control Method for Quadruped Robot Via Neural Network

TSONN: Time-stepping-oriented neural network for solving partial differential equations

PTP: Parallelized Tracking and Prediction With Graph Neural Networks and Diversity Sampling

Safe and Real-Time Consistent Planning for Autonomous Vehicles in Partially Observed Environments via Parallel Consensus Optimization

PTP-STGCN: Pedestrian Trajectory Prediction Based on a Spatio-temporal Graph Convolutional Neural Network

PID Controller-Based Stochastic Optimization Acceleration for Deep Neural Networks

Multi-Agent Path Planning Method Based on Improved Deep Q-Network in Dynamic Environments

Learning Fuel-Optimal Trajectories for Space Applications via Pontryagin Neural Networks

Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methods

Trajectory Planning for Autonomous Driving in Unstructured Scenarios Based on Graph Neural Network and Numerical Optimization

MT-CNN: A Lightweight Spatial-Temporal Convolutional Neural Network for Deep Learning of Complex Trajectory Distributions based on Area Partitioning

A Predefined Fixed-Time Convergence ZNN and its Applications to Time-Varying Quadratic Programming Solving and Dual-Arm Manipulator Cooperative Trajectory Tracking

A Neural Network Approach for Stochastic Optimal Control

Learning-Initialized Trajectory Planning in Unknown Environments