Abstract:We consider a class of finite time horizon nonlinear stochastic optimal control problem, where the control acts additively on the dynamics and the control cost is quadratic. This framework is flexible and has found applications in many domains. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging Monte Carlo task. The focus of this article is to propose a new Monte Carlo approach that significantly improves upon existing methodology. Our proposed methodology first tackles the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state space model associated with the control problem, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce computational cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}(\epsilon^2)$ mean squared error with a computational cost of $\mathcal{O}(\epsilon^{-2}\log(\epsilon)^2)$. In contrast, a computational cost of $\mathcal{O}(\epsilon^{-3})$ is required for existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples, which validate our theory.

Optimal control of a linear system with multiplicative noise at control parameter

Optimal Feedback Control of Strongly Non-Linear Systems Excited by Bounded Noise

Optimal Parametric Control of Nonlinear Random Vibrating Systems

Iterative design of suboptimal feedback control for bilinear parabolic PDE systems

Optimal Bilinear Control of Nonlinear Stochastic Schrödinger Equations Driven by Linear Multiplicative Noise

Multi-Objective Optimal Control with Arbitrary Additive and Multiplicative Noise

Optimal control for a nonlinear Schrödinger problem perturbed by multiplicative fractional noise

A Minimax Stochastic Optimal Control for Bounded-Uncertain Systems

Optimal control problems in transport dynamics with additive noise

Nonlinear SPDE driven by Levy noise: Well-posedness, optimal control and invariant measure

Robust Control Design for Linear Systems via Multiplicative Noise

Pontryagin's Principle for Some Probabilistic Control Problems

A Multilevel Approach for Stochastic Nonlinear Optimal Control

Stochastic Maximum Principle for Optimal Control ofPartial Differential Equations Driven by White Noise

Nonlinear Optimal Control for Stochastic Dynamical Systems

Optimal control of linear stochastic systems with applications to time lag systems

Optimal Covariance Steering for Continuous-Time Linear Stochastic Systems With Additive Noise

Optimal Control of a Nonlinear Stochastic Schrödinger Equation

Model-free Optimal Control of Discrete-Time Systems with Additive and Multiplicative Noises

Optimal Control for a Class of Infinite Dimensional Systems Involving an $L^\infty$-term in the Cost Functional

Ensemble Control of Stochastic Linear Systems