Abstract:Motion planning and control problems are embedded and essential in almost all robotics applications. These problems are often formulated as stochastic optimal control problems and solved using dynamic programming algorithms. Unfortunately, most existing algorithms that guarantee convergence to optimal solutions suffer from the curse of dimensionality: the run time of the algorithm grows exponentially with the dimension of the state space of the system. We propose novel dynamic programming algorithms that alleviate the curse of dimensionality in problems that exhibit certain low-rank structure. The proposed algorithms are based on continuous tensor decompositions recently developed by the authors. Essentially, the algorithms represent high-dimensional functions (e.g., the value function) in a compressed format, and directly perform dynamic programming computations (e.g., value iteration, policy iteration) in this format. Under certain technical assumptions, the new algorithms guarantee convergence towards optimal solutions with arbitrary precision. Furthermore, the run times of the new algorithms scale polynomially with the state dimension and polynomially with the ranks of the value function. This approach realizes substantial computational savings in "compressible" problem instances, where value functions admit low-rank approximations. We demonstrate the new algorithms in a wide range of problems, including a simulated six-dimensional agile quadcopter maneuvering example and a seven-dimensional aircraft perching example. In some of these examples, we estimate computational savings of up to ten orders of magnitude over standard value iteration algorithms. We further demonstrate the algorithms running in real time on board a quadcopter during a flight experiment under motion capture.

Mean field type control with species dependent dynamics via structured tensor optimization

Graph-structured tensor optimization for nonlinear density control and mean field games

Kernel Expansions for High-Dimensional Mean-Field Control with Non-local Interactions

Mean Field Control by Stochastic Koopman Operator via a Spectral Method

Mean Field Type Control Problems, Some Hilbert-space-valued FBSDEs, and Related Equations

Optimal control of distributed ensembles with application to Bloch equations

Optimal control problems with generalized mean-field dynamics and viscosity solution to Master Bellman equation

Backstepping Mean-Field Density Control for Large-Scale Heterogeneous Nonlinear Stochastic Systems

Transporting Robotic Swarms via Mean-Field Feedback Control

Feedback Interconnected Mean-Field Density Estimation and Control

Solving high-dimensional Hamilton-Jacobi-Bellman equation with functional hierarchical tensor

Entropic Optimal Planning for Path-Dependent Mean Field Games

Mean-field control of non exchangeable systems

Ensemble Control of Stochastic Linear Systems

Extended mean-field control problems with multi-dimensional singular controls

Decentralized Optimal Tracking Control for Large-scale Multi-Agent Systems under Complex Environment: A Constrained Mean Field Game with Reinforcement Learning Approach

Deep Graphic FBSDEs for Opinion Dynamics Stochastic Control

Optimal movement control models of Langevin and Hamiltonian types

Existence of Optimal Stationary Singular Controls and Mean Field Game Equilibria

High-Dimensional Stochastic Optimal Control using Continuous Tensor Decompositions

A Class of Discrete-time Mean-field Stochastic Linear-quadratic Optimal Control Problems with Financial Application