Abstract:We present a finite-horizon optimization algorithm that extends the established concept of Dual Dynamic Programming (DDP) in two ways. First, in contrast to the linear costs, dynamics, and constraints of standard DDP, we consider problems in which all of these can be polynomial functions. Second, we allow the state trajectory to be described by probability distributions rather than point values, and return approximate value functions fitted to these. The algorithm is in part an adaptation of sum-of-squares techniques used in the approximate dynamic programming literature. It alternates between a forward simulation through the horizon, in which the moments of the state distribution are propagated through a succession of single-stage problems, and a backward recursion, in which a new polynomial function is derived for each stage using the moments of the state as fixed data. The value function approximation returned for a given stage is the point-wise maximum of all polynomials derived for that stage. This contrasts with the piecewise affine functions derived in conventional DDP. We prove key convergence properties of the new algorithm, and validate it in simulation on two case studies related to the optimal operation of energy storage devices with nonlinear characteristics. The first is a small borehole storage problem, for which multiple value function approximations can be compared. The second is a larger problem, for which conventional discretized dynamic programming is intractable.

Higher-Order Nonlinear Discrete Approximate Iteration on the Continuing Dynamic Programming

Approximate dynamic programming for continuous state and control problems

Model-free Adaptive Dynamic Programming for Optimal Control of Discrete-time Affine Nonlinear System

Policy Approximation in Policy Iteration Approximate Dynamic Programming for Discrete-Time Nonlinear Systems.

Convergence of Dynamic Programming Models

Umbilical cord serum activin A levels are increased in pre-eclampsia with impaired blood flow in the uteroplacental and fetal circulation.

Approximate Dynamic Programming with Feasibility Guarantees

On Iterative Adaptive Dynamic Programming

A Simultaneous Strategy For Dynamic Optimization Based On Symbolic Derivation

A Convex Optimization Approach to Dynamic Programming in Continuous State and Action Spaces

A Novel Approximate Dynamic Programming Structure for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Efficient approximate dynamic programming based on design and analysis of computer experiments for infinite-horizon optimization

Inexact cuts for Deterministic and Stochastic Dual Dynamic Programming applied to convex nonlinear optimization problems

Dual dynamic programming for stochastic programs over an infinite horizon

Error bound analysis of policy iteration based approximate dynamic programming for deterministic discrete-time nonlinear systems

Boosting Dynamic Programming with Neural Networks for Solving NP-hard Problems.

Discrete‐Time Optimal Control of State‐Constrained Nonlinear Systems Using Approximate Dynamic Programming

Dynamic Programming for Structured Continuous Markov Decision Problems

Gradient-Bounded Dynamic Programming with Submodular and Concave Extensible Value Functions

Direct Heuristic Dynamic Programming with Augmented States

A Moment and Sum-of-Squares Extension of Dual Dynamic Programming with Application to Nonlinear Energy Storage Problems