An infinite-horizon optimal control problem and the stability of the adjoint variable (in Russian)

Dmitry Khlopin

DOI: https://doi.org/10.48550/arXiv.1012.3592

2010-12-16

Optimization and Control

Abstract:In an infinite-horizon optimal control problem the stability of the adjoint variable implying the vanishing of the adjoint variable at infinity along optimal solution.

What problem does this paper attempt to address?

The transversality conditions in infinite horizon problems and the stability of adjoint variable

Khlopin Dmitry

DOI: https://doi.org/10.48550/arXiv.1108.2562

2011-08-12

Abstract:This paper investigates the necessary conditions of optimality for uni- formly overtaking optimal control on infinite horizon with free right endpoint. Clarke's form of the Pontryagin Maximum Principle is proved without the as- sumption on boundedness of total variation of adjoint variable. The transversality condition for adjoint variable is shown to become necessary if the adjoint variable is partially Lyapunov stable. The modifications of this condition are proposed for the case of unbounded adjoint variable. The Cauchy-type formula for the adjoint variable proposed by S. M. Aseev and A. V. Kryazhimskii is shown to complement relations of the Pontryagin Maximum Principle up to the complete set of necessary conditions of optimality if the improper integral in the formula converges conditionally and continuously depends on the original position. The results are extended to an unbounded objective functional (described by a non- convergent improper integral), unbounded constraint on the control, and uniformly sporadically catching up optimal control.

Optimization and Control,Systems and Control
Linear quadratic optimal control problems for stochastic evolution equations in infinite horizon

Qi Lü

DOI: https://doi.org/10.3934/mcrf.2024017

2024-05-15

Mathematical Control and Related Fields

Abstract:We study linear quadratic optimal control problems for stochastic evolution equations in an infinite horizon with constant coefficients. We first give a characterization of the non-emptiness of the admissible control set for all initial states by an operator-valued algebraic Lyapunov equation. Then we demonstrate the equivalence between the existence of an optimal control with suitable a prori bound and the existence of an optimal feedback control. This stands in contrast to the stochastic linear quadratic optimal control problems in a finite time horizon. Finally, we prove that the optimal feedback control can be determined via a generalized operator-valued algebraic Riccati equation.

mathematics, applied
Infinite horizon optimal control of forward-backward stochastic differential equations with delay

Nacira Agram,Bernt Øksendal

DOI: https://doi.org/10.48550/arXiv.1302.1723

2013-12-06

Abstract:We consider a problem of optimal control of an infinite horizon system governed by forward-backward stochastic differential equations with delay. Sufficient and necessary maximum principles for optimal control under partial information in infinite horizon are derived. We illustrate our results by an application to a problem of optimal consumption with respect to recursive utility from a cash flow with delay.

Optimization and Control
Necessary Conditions In Infinite-Horizon Control Problem That Need No Asymptotic Assumptions

Dmitry V. Khlopin

DOI: https://doi.org/10.1007/s11228-023-00672-5

2024-09-17

Abstract:We consider an infinite-horizon optimal control problem with an asymptotic terminal constraint. For the the weakly overtaking criterion and the overtaking criterion, necessary boundary conditions on co-state arcs are deduced, these conditions need no assumptions about the asymptotic behavior of the motion, co-state arc, cost functional, and its derivatives. In the absence of an asymptotic terminal constraint, these boundary conditions with the Pontryagin Maximum Principle allow raising the co-state arcs, corresponding to some asymptotic subdifferentials of the cost functional (fixing the optimal control) at infinity. If this set is a singleton, these conditions coincide with the co-state arc representation proposed by Aseev and Kryazhimskii. These results are illustrated by several examples.

Optimization and Control
Pontryagin Principles in Infinite Horizon in Presence of Asymptotical Constraints

Joel Blot,Thoi Ngo

DOI: https://doi.org/10.48550/arXiv.1511.02090

2015-11-06

Dynamical Systems

Abstract:We establish necessary conditions of optimality for discrete-time infinite-horizon optimal control in presence of constraints at infinity. These necessary conditions are in form of weak and strong Pontryagin principles. We use a functional analytic framework and multipliers rules in Banach (sequence) spaces. We establish new properties on Nemytskii operators in sequence spaces. We also provide sufficient conditions of optimality. MSC 2010: 49J21, 65K05, 39A99.
An Optimal Solution to Infinite Horizon Nonlinear Control Problems: Part II

Mohamed Naveed Gul Mohamed,Aayushman Sharma,Raman Goyal,Suman Chakravorty

2024-03-26

Abstract:This paper considers the infinite horizon optimal control problem for nonlinear systems. Under the condition of nonlinear controllability of the system to any terminal set containing the origin and forward invariance of the terminal set, we establish a regularized solution approach consisting of a ``finite free final time" optimal transfer problem to the terminal set which renders the set globally asymptotically stable. Further, we show that the approximations converge to the optimal infinite horizon cost as the size of the terminal set decreases to zero. We also perform the analysis for the discounted problem and show that the terminal set is asymptotically stable only for a subset of the state space and not globally. The theory is empirically evaluated on various nonholonomic robotic systems to show that the cost of our approximate problem converges and the transfer time into the terminal set is dependent on the initial state of the system, necessitating the free final time formulation.

Optimization and Control,Systems and Control
Infinite Horizon Sparse Optimal Control

Dante Kalise,Karl Kunisch,Zhiping Rao

DOI: https://doi.org/10.1007/s10957-016-1016-9

2016-10-05

Journal of Optimization Theory and Applications

Abstract:A class of infinite horizon optimal control problems involving nonsmooth cost functionals is discussed. The existence of optimal controls is studied for both the convex case and the nonconvex case, and the sparsity structure of the optimal controls promoted by the nonsmooth penalties is analyzed. A dynamic programming approach is proposed to numerically approximate the corresponding sparse optimal controllers.

mathematics, applied,operations research & management science
On receding-horizon approximation in time-varying optimal control

Jintao Sun,Michael Cantoni

2023-09-05

Abstract:The closed-loop stability and infinite-horizon performance of receding-horizon approximations are studied for non-stationary linear-quadratic regulator (LQR) problems. The approach is based on a lifted reformulation of the optimal control problem, under assumed uniform controllability and observability, leading to a strict contraction property of the corresponding Riccati operator. Leveraging this contraction property, a stabilizing linear time-varying state-feedback approximation of the infinite-horizon optimal control policy is constructed to meet a performance-loss specification. Its synthesis involves only finite preview of the time-varying problem data at each time step, over a sufficiently long prediction horizon.

Systems and Control,Optimization and Control
Linear-quadratic optimal control for infinite-dimensional input-state-output systems

Timo Reis,Manuel Schaller

2024-01-21

Abstract:We examine the minimization of a quadratic cost functional composed of the output and the final state of abstract infinite-dimensional evolution equations in view of existence of solutions and optimality conditions. While the initial value is prescribed, we are minimizing over all inputs within a specified convex subset of square integrable controls with values in a Hilbert space. The considered class of infinite-dimensional systems is based on the system node approach introduced by Staffans. Thus, our developed approach includes optimal control of a wide variety of linear partial differential equations with boundary control and observation that are not well-posed in the sense that the output continuously depends on the input and the initial value. We provide an application of particular optimal control problems arising in energy-optimal control of port-Hamiltonian systems. Last, we illustrate the our abstract theory by two examples including a non-well-posed heat equation with Dirichlet boundary control and a wave equation on an L-shaped domain with boundary control of the stress in normal direction.

Optimization and Control,Functional Analysis
Asymptotic problems in optimal control with a vanishing Lagrangian and unbounded data

Monica Motta,Caterina Sartori

DOI: https://doi.org/10.48550/arXiv.1406.7655

2014-06-30

Abstract:In this paper we give a representation formula for the limit of the fnite horizon problem as the horizon becomes infinite, with a nonnegative Lagrangian and unbounded data. It is related to the limit of the discounted infinite horizon problem, as the discount factor goes to zero. We give sufficient conditions to characterize the limit function as unique nonnegative solution of the associated HJB equation. We also briefly discuss the ergodic problem.

Optimization and Control
Stochastic Singular Linear Systems and Related Linear-Quadratic Optimal Control Problems under Finite and Infinite Horizons

Mengzhen Li,Tianyang Nie,Zhen Wu

2024-09-03

Abstract:In this paper, we study the necessary and sufficient conditions for ensuring the well-posedness of the stochastic singular systems. Moreover, we investigate the stochastic singular linear-quadratic control problems, considering both finite and infinite horizons, and transform each of these problems into their corresponding normal linear-quadratic control problem. To guarantee the finiteness of the infinite-horizon linear-quadratic control problem, we establish the Popov-Belevitch-Hautus rank criterion for accessing the controllability of the stochastic system. Furthermore, we derive the feedback form of the optimal control. Finally, we provide solutions for illustrated examples of the stochastic singular linear-quadratic control problem in both finite and infinite horizons.

Optimization and Control
LOCAL AND GLOBAL STABILITY OF INFINITE-HORIZON VARIATIONAL PROBLEM

LT Gong,PS Fei

DOI: https://doi.org/10.1016/s0252-9602(17)30715-4

1998-01-01

Abstract:This paper presents an existence theorem of the optimal solution for the infinite-horizon variational problem with the inclusion constraints on the state variables and velocity variables, give the estimation of the optimal solution. Under some conditions, the authors obtain the local stability and global stability of the stationary solution.
Infinite Horizon Backward Stochastic Difference Equations and Related Stochastic Recursive Control Problems

Yuanyuan Ji,Qi Zhang

DOI: https://doi.org/10.1137/23m160270x

IF: 2.2

2024-10-09

SIAM Journal on Control and Optimization

Abstract:SIAM Journal on Control and Optimization, Volume 62, Issue 5, Page 2750-2775, October 2024. This paper is concerned with the discrete-time infinite horizon backward stochastic differential equation, i.e., the infinite horizon backward stochastic difference equation. We first prove the existence and uniqueness of the solution of this equation. Then the discrete-time stochastic recursive control problem is studied, in which the cost functional is set to be the solution of the infinite horizon backward stochastic difference equation. By introducing a proper discrete-time infinite horizon dual equation, we prove the stochastic maximum principle and the verification theorem for this recursive control problem. Finally we apply the derived stochastic maximum principle to the optimal consumption problem arising from a type of long-term trust fund.

mathematics, applied,automation & control systems
On extension for infinite horizon game of pursuit-evasion

Dmitry Khlopin

DOI: https://doi.org/10.48550/arXiv.1012.3579

2010-12-16

Abstract:The extension of a conflict control problem with infinite horizon is constructed. This extension is the projective limit of restricted games. Relations between "sensitivity to target set" and the existence of the optimal control are studied. Special attention is paid to the pursuit-evasion game with "joint control/relaxed control".

Optimization and Control
Infinite dimensional open-loop linear quadratic stochastic optimal control problems and related games

Guangdong Jing

2024-03-24

Abstract:We investigate the linear quadratic stochastic optimal control problems in infinite dimension without Markovian restriction for coefficients. The necessary and sufficient conditions for open-loop optimal controls are presented. We prove the Fréchet differentiable of the cost functional with respect to the control variable, and the Fréchet derivatives are characterized in detail by operators derived from dual analysis, which are proven to be the stationary conditions. Transposition methods are adopted to deal with the adjoint equations. As applications, we employ the results to study open-loop Nash equilibria for two-person stochastic differential games.

Optimization and Control,Probability
Characterizing All Optimal Controls for an Indefinite Stochastic Linear Quadratic Control Problem

HZ Wu,XY Zhou

DOI: https://doi.org/10.1109/tac.2002.800650

IF: 6.549

2002-01-01

IEEE Transactions on Automatic Control

Abstract:This note is concerned with a stochastic linear quadratic (LQ) control problem in the infinite-time horizon, with indefinite state and control weighting matrices in the cost function. It is shown that the solvability of this problem is equivalent to the existence of a so-called static stabilizing solution to a generalized algebraic Riccati equation. Moreover, another algebraic Riccati equation is introduced and all the possible optimal controls, including the ones in state feedback form, of the underlying LQ problem are explicitly obtained in terms of the two Riccati equations.
Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the related Hamiltonian Systems

Qingmeng Wei,Yaqi Xu,Zhiyong Yu

2023-11-13

Abstract:In this work, we focus on an infinite horizon mean-field linear-quadratic stochastic control problem with jumps. Firstly, the infinite horizon linear mean-field stochastic differential equations and backward stochastic differential equations with jumps are studied to support the research of the control problem. The global integrability properties of their solution processes are studied by introducing a kind of so-called dissipation conditions suitable for the systems involving the mean-field terms and jumps. For the control problem, we conclude a sufficient and necessary condition of open-loop optimal control by the variational approach. Besides, a kind of infinite horizon fully coupled linear mean-field forward-backward stochastic differential equations with jumps is studied by using the method of continuation. Such a research makes the characterization of the open-loop optimal controls more straightforward and complete.

Optimization and Control,Probability
Linear-quadratic optimal control for abstract differential-algebraic equations

Hannes Gernandt,Timo Reis

2024-07-10

Abstract:In this paper, we extend a classical approach to linear quadratic (LQ) optimal control via Popov operators to abstract linear differential-algebraic equations (ADAEs) in Hilbert spaces. To ensure existence of solutions, we assume that the underlying differential-algebraic equation has index one in the pseudo-resolvent sense. This leads to the existence of a degenerate semigroup that can be used to define a Popov operator for our system. It is shown that under a suitable coercivity assumption for the Popov operator the optimal costs can be described by a bounded Riccati operator and that the optimal control input is of feedback form. Furthermore, we characterize exponential stability of ADAEs which is required to solve the infinite horizon LQ problem.

Optimization and Control
Optimal control of diffusion processes: $\infty$-order variational analysis and numerical solution

Roman Chertovskih,Nikolay Pogodaev,Maxim Staritsyn,A. Pedro Aguiar

2024-03-04

Abstract:We tackle a nonlinear optimal control problem for a stochastic differential equation in Euclidean space and its state-linear counterpart for the Fokker-Planck-Kolmogorov equation in the space of probabilities. Our approach is founded on a novel concept of local optimality surpassing Pontryagin's minimum, originally crafted for deterministic optimal ensemble control problems. A key practical outcome is a rapidly converging numerical algorithm, which proves its feasibility for problems involving Markovian and open-loop strategies.

Optimization and Control

An infinite-horizon optimal control problem and the stability of the adjoint variable (in Russian)

The transversality conditions in infinite horizon problems and the stability of adjoint variable

Linear quadratic optimal control problems for stochastic evolution equations in infinite horizon

Infinite horizon optimal control of forward-backward stochastic differential equations with delay

Necessary Conditions In Infinite-Horizon Control Problem That Need No Asymptotic Assumptions

Pontryagin Principles in Infinite Horizon in Presence of Asymptotical Constraints

An Optimal Solution to Infinite Horizon Nonlinear Control Problems: Part II

Infinite Horizon Sparse Optimal Control

On receding-horizon approximation in time-varying optimal control

Linear-quadratic optimal control for infinite-dimensional input-state-output systems

Asymptotic problems in optimal control with a vanishing Lagrangian and unbounded data

Stochastic Singular Linear Systems and Related Linear-Quadratic Optimal Control Problems under Finite and Infinite Horizons

LOCAL AND GLOBAL STABILITY OF INFINITE-HORIZON VARIATIONAL PROBLEM

Infinite Horizon Backward Stochastic Difference Equations and Related Stochastic Recursive Control Problems

On extension for infinite horizon game of pursuit-evasion

Infinite dimensional open-loop linear quadratic stochastic optimal control problems and related games

Characterizing All Optimal Controls for an Indefinite Stochastic Linear Quadratic Control Problem

Infinite Horizon Mean-Field Linear Quadratic Optimal Control Problems with Jumps and the related Hamiltonian Systems

Linear-quadratic optimal control for abstract differential-algebraic equations

Optimal control of diffusion processes: $\infty$-order variational analysis and numerical solution