Abstract:We consider a class of finite time horizon nonlinear stochastic optimal control problem, where the control acts additively on the dynamics and the control cost is quadratic. This framework is flexible and has found applications in many domains. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging Monte Carlo task. The focus of this article is to propose a new Monte Carlo approach that significantly improves upon existing methodology. Our proposed methodology first tackles the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state space model associated with the control problem, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce computational cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}(\epsilon^2)$ mean squared error with a computational cost of $\mathcal{O}(\epsilon^{-2}\log(\epsilon)^2)$. In contrast, a computational cost of $\mathcal{O}(\epsilon^{-3})$ is required for existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples, which validate our theory.

Almost Optimal Agnostic Control of Unknown Linear Dynamics

Optimal Agnostic Control of Unknown Linear Dynamics in a Bounded Parameter Range

Controlling Unknown Linear Dynamics with Almost Optimal Regret

Optimal Guaranteed Cost Control for a Class of Linear Uncertain Time-Delay Systems

A Learning-Based Optimal Tracking Controller for Continuous Linear Systems with Unknown Dynamics: Theory and Case Study

A Minimax Stochastic Optimal Control for Bounded-Uncertain Systems

Worst-Case Control and Learning Using Partial Observations Over an Infinite Time-Horizon

Stochastic Optimal Control as Approximate Input Inference

Regret Optimal Control for Uncertain Stochastic Systems

Optimal Adaptive Control of Linear Stochastic Systems with Quadratic Cost Function

Learning-based Optimal Control for Linear Systems with Model Uncertainties

Learning-Based Optimal Control with Performance Guarantees for Unknown Systems with Latent States

Episodic Bayesian Optimal Control with Unknown Randomness Distributions

A Multilevel Approach for Stochastic Nonlinear Optimal Control

Regret-optimal control in dynamic environments

Probabilistic inverse optimal control for non-linear partially observable systems disentangles perceptual uncertainty and behavioral costs

Tracking optimal feedback control under uncertain parameters

Convexity and monotonicity in nonlinear optimal control under uncertainty

Safe Non-Stochastic Control of Control-Affine Systems: An Online Convex Optimization Approach

Regret-Optimal Control under Partial Observability

Suboptimality analysis of receding horizon quadratic control with unknown linear systems and its applications in learning-based control