Abstract:We consider a class of finite time horizon nonlinear stochastic optimal control problem, where the control acts additively on the dynamics and the control cost is quadratic. This framework is flexible and has found applications in many domains. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging Monte Carlo task. The focus of this article is to propose a new Monte Carlo approach that significantly improves upon existing methodology. Our proposed methodology first tackles the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state space model associated with the control problem, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce computational cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}(\epsilon^2)$ mean squared error with a computational cost of $\mathcal{O}(\epsilon^{-2}\log(\epsilon)^2)$. In contrast, a computational cost of $\mathcal{O}(\epsilon^{-3})$ is required for existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples, which validate our theory.

Nonstationary discrete-time deterministic and stochastic control systems: Bounded and unbounded cases

Nonstationary Discrete-Time Deterministic And Stochastic Control Systems With Infinite Horizon

First Passage Problems for Nonstationary Discrete-Time Stochastic Control Systems

Stochastic Optimal Control of Quasi Non-Integrable Hamiltonian Systems with Stochastic Maximum Principle

Discrete-Time Hybrid Control Processes with Unbounded Costs

Brief Minimax control for discrete-time time-varying stochastic systems

A Minimax Stochastic Optimal Control for Bounded-Uncertain Systems

Minimax Control for Discrete-Time Time-Varying Stochastic Systems

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Stability analysis of optimal control problems with time-dependent costs

Mixed Deterministic and Random Optimal Control of Linear Stochastic Systems with Quadratic Costs

Infinite dimensional open-loop linear quadratic stochastic optimal control problems and related games

Stochastic Singular Linear Systems and Related Linear-Quadratic Optimal Control Problems under Finite and Infinite Horizons

Extended Stationary Problem About a Stochastic Control of Singular Type

A Multilevel Approach for Stochastic Nonlinear Optimal Control

An Optimal Solution to Infinite Horizon Nonlinear Control Problems: Part II

Optimal control on finite graphs: asymptotic optimal controls and ergodic constant in the case of entropic costs

Optimal Control of Discrete-Time Nonlinear Systems

Robust Optimal Guaranteed Cost Control for 2D Discrete Systems

Finite-Horizon Optimal Control of Discrete-Time Switched Linear Systems

Time-Inconsistent Stochastic Optimal Control Problems and Backward Stochastic Volterra Integral Equations