Abstract:We consider a class of finite time horizon nonlinear stochastic optimal control problem, where the control acts additively on the dynamics and the control cost is quadratic. This framework is flexible and has found applications in many domains. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging Monte Carlo task. The focus of this article is to propose a new Monte Carlo approach that significantly improves upon existing methodology. Our proposed methodology first tackles the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state space model associated with the control problem, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce computational cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}(\epsilon^2)$ mean squared error with a computational cost of $\mathcal{O}(\epsilon^{-2}\log(\epsilon)^2)$. In contrast, a computational cost of $\mathcal{O}(\epsilon^{-3})$ is required for existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples, which validate our theory.

Optimal control problems driven by nonlinear degenerate Fokker-Planck equations

Optimal Control of the Nonlinear Stochastic Fokker--Planck Equation

Optimal Control of the Fokker–Planck Equation with Space-Dependent Controls

Optimal Control of Stochastic Differential Equations via Fokker–Planck Equations

Controlling a Nonlinear Fokker–Planck Equation via Inputs with Nonlocal Action

Controlling a nonlinear Fokker-Planck equation via inputs with nonlocal action

Optimal control of diffusion processes: $\infty$-order variational analysis and numerical solution

Optimal control of nonlocal continuity equations: numerical solution

Mean-field sparse optimal control

Optimal movement control models of Langevin and Hamiltonian types

Optimal control of conditioned processes with feedback controls

Optimal Control of McKean-Vlasov equations with controlled stochasticity

Existence of an optimal control for stochastic control systems with nonlinear cost functional

Optimal Control of Semilinear Elliptic Partial Differential Equations with Non-Lipschitzian Nonlinearities

Degenerate Mean Field Type Control with Linear and Unbounded Diffusion, and their Associated Equations

A Multilevel Approach for Stochastic Nonlinear Optimal Control

Optimal Control of a Nonlinear Stochastic Schrödinger Equation

Optimal control of distributed ensembles with application to Bloch equations

Optimal Control for a Class of Infinite Dimensional Systems Involving an $L^\infty$-term in the Cost Functional

Data-Driven Optimal Control Using Perron-Frobenius Operator