A Maximum Principle for Optimal Control of Stochastic Evolution Equations

Kai Du,Qingxin Meng

DOI: https://doi.org/10.48550/arXiv.1206.3649

2013-12-25

Abstract:A general maximum principle is proved for optimal controls of abstract semilinear stochastic evolution equations. The control variable, as well as linear unbounded operators, acts in both drift and diffusion terms, and the control set need not be convex.

Optimization and Control

What problem does this paper attempt to address?

Stochastic Optimal Control of Quasi Non-Integrable Hamiltonian Systems with Stochastic Maximum Principle

X. D. Gu,W. Q. Zhu,W. Xu

DOI: https://doi.org/10.1007/s11071-012-0494-z

IF: 5.741

2012-01-01

Nonlinear Dynamics

Abstract:A new procedure for designing optimal control of quasi non-integrable Hamiltonian systems under stochastic excitations is proposed based on the stochastic averaging method for quasi non-integrable Hamiltonian systems and the stochastic maximum principle. First, the control problem consisting of 2n-dimensional equations governing the controlled quasi non-integrable system and performance index is converted into a partially averaged one consisting of one-dimensional equation of the controlled system and performance index by using the stochastic averaging method. Then, the adjoint equation and the maximum condition of the partially averaged control problem are derived based on the stochastic maximum principle. The optimal control forces are determined from the maximum condition and solving the forward–backward stochastic differential equations (FBSDE). For infinite time-interval ergodic control, the adjoint variable is a stationary process and the FBSDE is reduced to a partial differential equation. Finally, the response statistics of optimally controlled system is predicted by solving the Fokker–Plank equation (FPE) associated with the fully averaged Itô equation of the controlled system. An example of two degree-of-freedom (DOF) quasi non-integrable Hamiltonian system is worked out to illustrate the proposed procedure and its effectiveness.
Maximum principle for optimal control of stochastic evolution equations with recursive utilities

Guomin Liu,Shanjian Tang

2024-02-04

Abstract:We consider the optimal control problem of stochastic evolution equations in a Hilbert space under a recursive utility, which is described as the solution of a backward stochastic differential equation (BSDE). A very general maximum principle is given for the optimal control, allowing the control domain not to be convex and the generator of the BSDE to vary with the second unknown variable $z$. The associated second-order adjoint process is characterized as a unique solution of a conditionally expected operator-valued backward stochastic integral equation.

Optimization and Control,Probability
Optimal bounded control of quasi-nonintegrable Hamiltonian systems using stochastic maximum principle

X. D. Gu,W. Q. Zhu

DOI: https://doi.org/10.1007/s11071-013-1188-x

IF: 5.741

2013-01-01

Nonlinear Dynamics

Abstract:A new procedure for designing optimal bounded control of quasi-nonintegrable Hamiltonian systems with actuator saturation is proposed based on the stochastic averaging method for quasi-nonintegrable Hamiltonian systems and the stochastic maximum principle. First, the stochastic averaging method for controlled quasi-nonintegrable Hamiltonian systems is introduced. The original control problem is converted into one for a partially averaged equation of system energy together with a partially averaged performance index. Then, the adjoint equation and the maximum condition of the partially averaged control problem are derived based on the stochastic maximum principle. The bounded optimal control forces are obtained from the maximum condition and solving the forward–backward stochastic differential equations (FBSDE). For infinite time-interval ergodic control, the adjoint variable is stationary process, and the FBSDE is reduced to an ordinary differential equation. Finally, the stationary probability density of the Hamiltonian and other response statistics of optimally controlled system are obtained by solving the Fokker–Plank–Kolmogorov equation associated with the fully averaged Itô equation of the controlled system. For comparison, the bang–bang control is also presented. An example of two degree-of-freedom quasi-nonintegrable Hamiltonian system is worked out to illustrate the proposed procedure and its effectiveness. Numerical results show that the proposed control strategy has higher control efficiency and less discontinuous control force than the corresponding bang–bang control at the price of slightly less control effectiveness.
Maximum principle for stochastic control of SDEs with measurable drifts

Olivier Menoukeu-Pamen,Ludovic Tangpi

DOI: https://doi.org/10.48550/arXiv.2101.06205

2021-01-16

Abstract:In this paper, we consider stochastic optimal control of systems driven by stochastic differential equations with irregular drift coefficient. We establish a necessary and sufficient stochastic maximum principle. To achieve this, we first derive an explicit representation of the first variation process (in Sobolev sense ) of the controlled diffusion. Since the drift coefficient is not smooth, the representation is given in terms of the local time of the state process. Then we construct a sequence of optimal control problems with smooth coefficients by an approximation argument. Finally, we use Ekeland's variational principle to obtain an approximating adjoint process from which we derive the maximum principle by passing to the limit.

Optimization and Control
Maximum principle for stochastic optimal control problem under convex expectation

Xiaojuan Li,Mingshang Hu

2024-08-20

Abstract:In this paper, we study a stochastic optimal control problem under a type of consistent convex expectation dominated by G-expectation. By the separation theorem for convex sets, we get the representation theorems for this convex expectation and conditional convex expectation. Based on these results, we obtain the variational equation for cost functional by weak convergence and discretization methods. Furthermore, we establish the maximum principle which is sufficient under usual convex assumptions. Finally, we study the linear quadratic control problem by using the obtained maximum principle.

Optimization and Control
A Second-Order Maximum Principle for Singular Optimal Stochastic Controls

Shanjian Tang

DOI: https://doi.org/10.3934/dcdsb.2010.14.1581

2010-01-01

Abstract:A singular optimal stochastic control problem is studied. A second-order maximum principle is presented. The second-order adjoint processes are involved, though the diffusion of the control system is control independent. The range theorem of vector-valued measures is used to prove the maximum principle. Examples are given to illustrate the applications.
Stochastic Maximum Principle for Optimal Control of SPDEs

Marco Fuhrman,Ying Hu,Gianmario Tessitore

DOI: https://doi.org/10.1007/s00245-013-9203-7

2013-05-18

Abstract:We prove a version of the maximum principle, in the sense of Pontryagin, for the optimal control of a stochastic partial differential equation driven by a finite dimensional Wiener process. The equation is formulated in a semi-abstract form that allows direct applications to a large class of controlled stochastic parabolic equations. We allow for a diffusion coefficient dependent on the control parameter, and the space of control actions is general, so that in particular we need to introduce two adjoint processes. The second adjoint process takes values in a suitable space of operators on L4.

mathematics, applied
Necessary Conditions for Optimal Control of Stochastic Systems with Random Jumps

SJ TANG,XJ LI

DOI: https://doi.org/10.1137/s0363012992233858

IF: 2.2

1994-01-01

SIAM Journal on Control and Optimization

Abstract:A maximum principle is proved for optimal controls of stochastic systems with random jumps. The control is allowed to enter into both diffusion and jump terms. The form of the maximum principle turns out to be quite different from the one corresponding to the pure diffusion system (the word ''pure'' here means the absence of the jump term). In calculating the first-order coefficient for the cost variation, only a property for Lebesgue integrals of scalar-valued functions in the real number space R is used. This shows that there is no essential difference between deterministic and stochastic systems as far as the derivation of maximum principles is concerned.
Stochastic Maximum Principle for Square-Integrable Optimal Control of Linear Stochastic Systems

Shanjian Tang,Xueqi Wang

DOI: https://doi.org/10.1007/s11401-024-0032-6

2024-01-01

Chinese Annals of Mathematics Series B

Abstract:The authors give a stochastic maximum principle for square-integrable optimal control of linear stochastic systems. The control domain is not necessarily convex and the cost functional can have a quadratic growth. In particular, they give a stochastic maximum principle for the linear quadratic optimal control problem.
Maximum principle for discrete-time stochastic optimal control problem and stochastic game

Zhen Wu,Feng Zhang

DOI: https://doi.org/10.3934/mcrf.2021031

2022-01-01

Mathematical Control and Related Fields

Abstract:This paper is first concerned with one kind of discrete-time stochastic optimal control problem with convex control domains, for which necessary condition in the form of Pontryagin's maximum principle and sufficient condition of optimality are derived. The results are then extended to two kinds of discrete-time stochastic games. Two illustrative examples are studied, for which the explicit optimal strategies are given. This paper establishes a rigorous version of discrete-time stochastic maximum principle in a clear and concise way and paves a road for further related topics.

mathematics, applied
Maximum principle for optimal control of interacting particle system: stochastic flow model

Andrey A. Dorogovtsev,Yuecai Han,Kateryna Hlyniana,Yuhang Li

2024-01-16

Abstract:In this paper, we consider the stochastic optimal control problem for the interacting particle system. We obtain the stochastic maximum principle of the optimal control system by introducing a generalized backward stochastic differential equation with interaction. The existence and uniqueness of the solution of this type of equation is proved. We derive the necessary condition that the optimal control should satisfy. As an application, the linear quadratic case is investigated to illustrate the main results.

Probability
A Minimax Stochastic Optimal Control for Bounded-Uncertain Systems

Z. G. Ying

DOI: https://doi.org/10.1177/1077546309103282

IF: 2.633

2010-01-01

Journal of Vibration and Control

Abstract:A minimax stochastic optimal control strategy for bounded-uncertain stochastic systems is proposed. The minimax dynamical programming equation for an uncertain stochastic control system is firstly derived based on the optimality principle and Itô differential rule. A new type of bang-bang constraint on the bounded uncertain disturbance is proposed to form a class of minimax stochastic optimal control problems. Then the worst disturbance and minimax optimal control are obtained for the bang-bang-type uncertain system under stochastic excitations. According to this method, the quasi linear control law is obtained for linear stochastic systems with bounded uncertainty and the state-dependent quasi Riccati equation is derived from the minimax dynamical programming equation. Furthermore, a minimax stochastic optimal control strategy for uncertain stochastic quasi Hamiltonian systems is developed based on the stochastic averaging method and minimax dynamical programming equation. The worst disturbance and minimax optimal control for the stochastically averaged system are obtained by the similar procedure. The proposed and developed minimax stochastic optimal control strategies are illustrated with an example of a single-degree-of-freedom uncertain stochastic control system.
Second Order Optimality Conditions for Optimal Control Problems of Stochastic Evolution Equations

Qi Lu,Haisen Zhang,Xu Zhang

DOI: https://doi.org/10.48550/arXiv.1811.07337

2018-11-18

Abstract:In this paper, we establish some second order necessary/sufficient optimality conditions for optimal control problems of stochastic evolution equations in infinite dimensions. The control acts on both the drift and diffusion terms and the control region is convex. The concepts of relaxed and $V$-transposition solutions (introduced in our previous works) to operator-valued backward stochastic evolution equations are employed to derive these optimality conditions. The correction part of the second order adjoint equation, which does not appear in the (first order) Pontryagin-type stochastic maximum principle, plays a fundamental role in our second order optimality conditions.

Optimization and Control
Maximum principle for discrete-time stochastic optimal control problem under distribution uncertainty

Mingshang Hu,Shaolin Ji,Xiaojuan Li

DOI: https://doi.org/10.48550/arXiv.2206.12846

2022-06-26

Abstract:In this paper, we study a discrete-time stochastic optimal control problem under distribution uncertainty with convex control domain. By weak convergence method and Sion's minimax theorem, we obtain the variational inequality for cost functional under a reference probability $P^{\ast}$. Moreover, under the square integrability condition for noise and control, we establish the discrete-time stochastic maximum principle under $P^{\ast}$. Finally, we introduce a backward algorithm to calculate the reference probability $P^{\ast}$ and the optimal control $u^{\ast}$.

Optimization and Control
Maximum Principle for Stochastic Control System with Elephant Memory and Jump Diffusion

Siqi Feng,Lei Gao,Guangchen Wang,Hua Xiao

DOI: https://doi.org/10.1007/s11424-024-3163-7

2024-06-13

Journal of Systems Science and Complexity

Abstract:Motivated by a duopoly game problem, the authors study an optimal control problem where the system is driven by Brownian motion and Poisson point process and has elephant memory for the control variable and the state variable. Firstly, the authors establish the unique solvability of an anticipated backward stochastic differential equation, derive a stochastic maximum principle, and prove a verification theorem for the aforementioned optimal control problem. Furthermore, the authors generalize these results to nonzero-sum stochastic differential game problems. Finally, the authors apply the theoretical results to the duopoly game problem and obtain the corresponding Nash equilibrium solution.

mathematics, interdisciplinary applications
Stochastic Maximum Principle for Subdiffusions and Its Applications

Shuaiqi Zhang,Zhen-Qing Chen

DOI: https://doi.org/10.1137/23m157168x

IF: 2.2

2024-03-15

SIAM Journal on Control and Optimization

Abstract:SIAM Journal on Control and Optimization, Volume 62, Issue 2, Page 953-981, April 2024. In this paper, we study optimal stochastic control problems for stochastic systems driven by non-Markov subdiffusion [math], which have mixed features of deterministic and stochastic controls. Here [math] is the standard Brownian motion on [math], and [math] is the inverse of a subordinator [math] with drift [math] that is independent of [math]. We obtain stochastic maximum principles (SMPs) for these systems using both convex and spiking variational methods, depending on whether or not the domain is convex. To derive SMPs, we first establish a martingale representation theorem for subdiffusions [math], and then use it to derive the existence and uniqueness result for the solutions of backward stochastic differential equations (BSDEs) driven by subdiffusions, which may be of independent interest. We also derive sufficient SMPs. Application to a linear quadratic system is given to illustrate the main results of this paper.

mathematics, applied,automation & control systems
Relationships Between the Maximum Principle and Dynamic Programming for Infinite Dimensional Stochastic Control Systems

Liangying Chen,Qi Lü

DOI: https://doi.org/10.48550/arXiv.2112.14636

2021-12-30

Abstract:Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationships between Pontryagin type maximum principle and dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables appearing into both the drift and the diffusion terms. To do so, we first establish dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth and nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations.

Optimization and Control
A global maximum principle for stochastic optimal control problems with delay and applications

Weijun Meng,Jingtao Shi

DOI: https://doi.org/10.1016/j.sysconle.2021.104909

2021-04-01

Abstract:In this paper, an open problem is solved, for the stochastic optimal control problem with delay where the control domain is nonconvex and the diffusion term contains both control and its delayed term. Inspired by previous results about delayed stochastic control systems, Peng's global stochastic maximum principle is generalized to the time delayed case. A special backward stochastic differential equation is introduced to deal with the cross terms, when applying the duality technique. Comparing with the classical result, the maximum condition contains an indicator function, which in fact is the characteristic of the stochastic optimal control problem with delay. Furthermore, to illustrate the applications of our theoretical results, three dynamic optimization problems are addressed based on the global maximum principle.

automation & control systems,operations research & management science
Stochastic Linear Quadratic Optimal Control with General Control Domain

Shaolin Ji,Xiaole Xue

DOI: https://doi.org/10.48550/arXiv.1710.11302

2017-10-31

Optimization and Control

Abstract:This paper considers the stochastic linear quadratic optimal control problem in which the control domain is nonconvex. By the functional analysis and convex perturbation methods, we establish a novel maximum principle. The application of the proposed maximum principle is illustrated through a work-out example.
Maximum principle for partially observed stochastic recursive optimal control problems involving impulse controls

Zhen Wu,Yan Zhang

DOI: https://doi.org/10.1002/oca.2920

2022-06-21

Optimal Control Applications and Methods

Abstract:This article focuses on a stochastic recursive optimal control problem involving impulse control under partially observed information and obtains the related maximum principle. Unlike the classical literature, both the regular control (considering the nonconvexity of the domain) and the impulse control (where the domain is convex) are considered in the framework. In virtue of two types of variational methods, spike variation and convex variation, the Pontryagin maximum principle is established. In addition, to demonstrate the validity of the results, this article investigates a differential game problem as an application.

automation & control systems,operations research & management science,mathematics, applied

A Maximum Principle for Optimal Control of Stochastic Evolution Equations

Stochastic Optimal Control of Quasi Non-Integrable Hamiltonian Systems with Stochastic Maximum Principle

Maximum principle for optimal control of stochastic evolution equations with recursive utilities

Optimal bounded control of quasi-nonintegrable Hamiltonian systems using stochastic maximum principle

Maximum principle for stochastic control of SDEs with measurable drifts

Maximum principle for stochastic optimal control problem under convex expectation

A Second-Order Maximum Principle for Singular Optimal Stochastic Controls

Stochastic Maximum Principle for Optimal Control of SPDEs

Necessary Conditions for Optimal Control of Stochastic Systems with Random Jumps

Stochastic Maximum Principle for Square-Integrable Optimal Control of Linear Stochastic Systems

Maximum principle for discrete-time stochastic optimal control problem and stochastic game

Maximum principle for optimal control of interacting particle system: stochastic flow model

A Minimax Stochastic Optimal Control for Bounded-Uncertain Systems

Second Order Optimality Conditions for Optimal Control Problems of Stochastic Evolution Equations

Maximum principle for discrete-time stochastic optimal control problem under distribution uncertainty

Maximum Principle for Stochastic Control System with Elephant Memory and Jump Diffusion

Stochastic Maximum Principle for Subdiffusions and Its Applications

Relationships Between the Maximum Principle and Dynamic Programming for Infinite Dimensional Stochastic Control Systems

A global maximum principle for stochastic optimal control problems with delay and applications

Stochastic Linear Quadratic Optimal Control with General Control Domain

Maximum principle for partially observed stochastic recursive optimal control problems involving impulse controls