Abstract:Controlling systems of ordinary differential equations (ODEs) is ubiquitous in science and engineering. For finding an optimal feedback controller, the value function and associated fundamental equations such as the Bellman equation and the Hamilton-Jacobi-Bellman (HJB) equation are essential. The numerical treatment of these equations poses formidable challenges due to their non-linearity and their (possibly) high-dimensionality. In this paper we consider a finite horizon control system with associated Bellman equation. After a time-discretization, we obtain a sequence of short time horizon problems which we call local optimal control problems. For solving the local optimal control problems we apply two different methods, one being the well-known policy iteration, where a fixed-point iteration is required for every time step. The other algorithm borrows ideas from Model Predictive Control (MPC), by solving the local optimal control problem via open-loop control methods on a short time horizon, allowing us to replace the fixed-point iteration by an adjoint method. For high-dimensional systems we apply low rank hierarchical tensor product approximation/tree-based tensor formats, in particular tensor trains (TT tensors) and multi-polynomials, together with high-dimensional quadrature, e.g. Monte-Carlo. We prove a linear error propagation with respect to the time discretization and give numerical evidence by controlling a diffusion equation with unstable reaction term and an Allen-Kahn equation.

Optimal bounds for POD approximations of infinite horizon control problems based on time derivatives

An Approximation for the Boundary Optimal Control Problem of a Heat Equation Defined in a Variable Domain

POD-based Reduced Order Optimal Control of Parabolic PDE Systems Via Diffusivity-Interior-boundary Actuation

A multilinear HJB-POD method for the optimal control of PDEs

A Multilinear HJB-POD Method for the Optimal Control of PDEs on a Tree Structure

POD Suboptimal Control of Evolution Problems: Theory and Applications

Finite‐Horizon Ε‐optimal Tracking Control of Discrete‐Time Linear Systems Using Iterative Approximate Dynamic Programming

Solving Finite-Horizon HJB for Optimal Control of Continuous-Time Systems

Policy-Iteration-Based Finite-Horizon Approximate Dynamic Programming for Continuous-Time Nonlinear Optimal Control

Fast Approximate Dynamic Programming for Input-Affine Dynamics

Approximate dynamic programming for continuous state and control problems

Theoretical and Numerical Analysis of Approximate Dynamic Programming with Approximation Errors

Numerical Solutions to the Bellman Equation of Optimal Control

Approximating optimal feedback controllers of finite horizon control problems using hierarchical tensor formats

POD-ROM methods: from a finite set of snapshots to continuous-in-time approximations

Hamiltonian-Driven Adaptive Dynamic Programming With Approximation Errors

Polynomial Approximation of Value Functions and Nonlinear Controller Design with Performance Bounds

A Multilevel Approach for Stochastic Nonlinear Optimal Control

H ∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems

A New Approach to Finite-Horizon Optimal Control for Discrete-Time Affine Nonlinear Systems via a Pseudolinear Method

Dynamic Programming Principle and Hamilton-Jacobi-Bellman Equation for Optimal Control Problems with Uncertainty