Abstract:SIAM Journal on Scientific Computing, Volume 46, Issue 5, Page C535-C556, October 2024. We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semilinear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the nonviscous Hamilton–Jacobi–Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare it to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution, and, via a modification, we show the wider applicability of our scheme. Reproducibility of computational results.This paper has been awarded the "SIAM Reproducibility Badge: Code and data available" as recognition that the authors have followed reproducibility principles valued by SISC and the scientific computing community. Code and data that allow readers to reproduce the results in this paper are available at https://github.com/EmoryMLIP/NeuralSOC and in the supplementary material (NeuralSOC-main.zip [ 29.9MB]).

Optimization Of Human Control Strategy With Simultaneously Perturbed Stochastic Approximation

Performance Evaluation and Optimization of Human Control Strategy.

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Near Optimal Control for a Class of Stochastic Hybrid Systems.

Automating Shift-Scheduling Calibration by Using Bionic Optimization and Personalized Driver Models

A hybrid model-based optimal control method for nonlinear systems using simultaneous dynamic optimization strategies

A Stochastic Model Predictive Control Approach for Modelling Human Driver Steering Control

Application Of Stochastic Model Predictive Control To Modeling Driver Steering Skills

Stochastic Model Predictive Control Strategy with Short-term Forecast Optimal SOC for a Plug-in Hybrid Electric Vehicle

Human-Variability-Respecting Optimal Control for Physical Human-Machine Interaction

Modeling Driver Steering Control Based on Stochastic Model Predictive Control

Validation of Stochastic Optimal Control Models for Goal-Directed Human Movements on the Example of Human Driving Behavior

Switching-Based Stochastic Model Predictive Control Approach for Modeling Driver Steering Skill

Dynamic Event-triggered Approximate Optimal Control Strategy for Nonlinear Systems

Modeling Driver, Driver Support, and Cooperative Systems with Dynamic Optimal Control

An Optimized Control Strategy Based on Hierarchical Design, 4WS-4WD, NMPC, and Nonlinear Dynamics Models

A Neural Network Approach for Stochastic Optimal Control

Data-Enabled Human-Machine Cooperative Driving Decoupled from Various Driver Steering Characteristics and Vehicle Dynamics

Adaptive Decision and Robust Control under Parallel Precedence Constraints in Human Strategy Modeling

Real-time coordinated control strategy for the hybrid electric propulsion system of a flying car with model adaptation and game theory

Human-like Tracking Control Strategy for Autonomous Vehicle Based on Preview Feedforward and Active Disturbance Rejection Control