Generalization of a Class of Singular Stochastic Control Problem

Xiang Liu

2000-01-01

Abstract:In this paper, we generalize a problem of singular stochastic control with stopping, and prove the existance of the optimal control. We find that the optimal strategies can be taken in different structure forms, depending on parameter values.

What problem does this paper attempt to address?

Generalized guaranteed cost control with D-stability and multiple output constraints.

Minqing Xiao,Hongye Su,Weihua Xu

DOI: https://doi.org/10.1016/j.amc.2012.06.010

IF: 4.397

2012-01-01

Applied Mathematics and Computation

Abstract:This paper is concerned with the problem of robust generalized guaranteed cost control with D-stability and multiple output constraints for a class of linear uncertain systems. Being a combination of output performance indices, a generalized cost function is considered to the linear polytopic uncertain systems described in a unified framework. The aim is to design a state feedback controller, such that the closed-loop system is robust D-stable, and the upper bound of the generalized cost function is as small as possible subject to multiple output constraints. Based on parameter-dependent Lyapunov functions, convex conditions for the existence of such controllers are presented in terms of linear matrix inequality. The proposed approach shows a unified treatment of the linear systems in the differential, shift, and delta domains. Numerical examples are provided to illustrate the effectiveness of the design method.
Robust Control For Singular Systems With State Delay And Parameter Uncertainty

Renquan Lu,Anke Xue,Hongye Su,Xiaofu Ji,Jian Chu

2006-01-01

Abstract:This paper considers the problem of robust control for singular systems with state delay and parameter uncertainty. The time delay appears in both state and control input and is assumed to be length-bounded otherwise arbitrarily fast time-varying, while the parameter uncertainty under consideration is norm bounded and possibly time-varying. We first obtain a condition for a singular time-delay system to be regular, impulse free and stable, based on which, the problems of robust stability and robust stabilization are solved. An linear matrix inequality (LMI) design method is develop and an explicit expression for the desired stabilizing control is also given. The proposed synthesis method is illustrated by a numerical example.
Stochastic stabilization of quasi-generalized Hamiltonian systems

quan ling,xiaoling jin,yong wang,zhilong huang

DOI: https://doi.org/10.1177/1077546313486503

IF: 2.633

2015-01-01

Journal of Vibration and Control

Abstract:A procedure for designing a feedback control to asymptotically stabilize, with probability one, quasi-generalized Hamiltonian systems subject to stochastically parametric excitations is proposed. First, the motion equations of controlled systems are reduced to lower-dimensional averaged Ito stochastic differential equations by using the stochastic averaging method. Second, a dynamic programming equation for the averaged system with an appropriate performance index (with undetermined parameters in cost function) is established based on the dynamic programming principle, and the optimal control law is derived from a minimization condition with respect to control. Third, the Lyapunov function method is adopted to evaluate the stability boundary of asymptotic stability with probability one for the uncontrolled/controlled systems. Finally, the parameters in cost function are selected to guarantee the sufficient stability of the controlled systems. Numerical results for a nine-dimensional mathematical system and a three-dimensional practical system, which describes a structure including viscoelastic element, illustrate the effectiveness of the feedback control strategy, and stability domains can be obviously enlarged when imposing the feedback controls on the original systems.
Singular LQ Sub-Optimal Control Problem for a Class of Discrete-Time Singular Systems with Time-Delay

Baolin Zhang,Feifei Zheng,Feilong Cao

DOI: https://doi.org/10.1109/ccdc.2009.5191610

2009-01-01

Abstract:Singular linear quadratic (LQ) sub-optimal control problem for a class of discrete-time singular systems with time-delay is investigated. First, based on the algebra equivalent transformation, the original singular LQ problem is transformed into a nonsingular LQ problem for standard discrete-time systems with time-delay. Under some general conditions, the optimal control law of the original LQ problem is given via successive approximation approach. The optimal control law can be synthesized as a sum of state feedback term and a time-delay compensation term which is a limit of a sequence of adjoint vectors.
Stability and Stabilization of a Class of Switched Stochastic Systems with Saturation Control

Guo Yingxin,Ge Shuzhi Sam,University of Chinese Academy of Sciences,Xu Chao

DOI: https://doi.org/10.1007/s11432-020-3002-7

2021-01-01

Science China Information Sciences

Abstract:A switching system always comprises of several subsystems and a rule supervising the switching between the subsystems. A major problem that is often inherent to all dynamical systems is actuator saturation. Saturation is a nonlinear property that nonlinearly maps small input signals to the output, which may affect the system properties and even destroy them. In this study, stability and stabilization of a class of switched stochastic systems with saturation control was investigated. First, the variation parameter method was used to present the integral form of switched stochastic systems. Second, to guarantee that the zero solution is globally exponentially stable in mean square, two sufficient conditions were obtained using direct computation with Gronwall inequality and indirect method with matrix theory, respectively. Further, another simple sufficient condition was obtained for the stability of the systems using the row norm, column norm, and Frobenius norm. Finally, two examples were used to illustrate the preciseness and effectiveness of the results. Moreover, various control designs were observed to stabilize the systems, which differ from the technique of linear matrix inequalities.
Exploratory Optimal Stopping: A Singular Control Formulation

Jodi Dianetti,Giorgio Ferrari,Renyuan Xu

2024-10-02

Abstract:This paper explores continuous-time and state-space optimal stopping problems from a reinforcement learning perspective. We begin by formulating the stopping problem using randomized stopping times, where the decision maker's control is represented by the probability of stopping within a given time--specifically, a bounded, non-decreasing, càdlàg control process. To encourage exploration and facilitate learning, we introduce a regularized version of the problem by penalizing it with the cumulative residual entropy of the randomized stopping time. The regularized problem takes the form of an (n+1)-dimensional degenerate singular stochastic control with finite-fuel. We address this through the dynamic programming principle, which enables us to identify the unique optimal exploratory strategy. For the specific case of a real option problem, we derive a semi-explicit solution to the regularized problem, allowing us to assess the impact of entropy regularization and analyze the vanishing entropy limit. Finally, we propose a reinforcement learning algorithm based on policy iteration. We show both policy improvement and policy convergence results for our proposed algorithm.

Optimization and Control,Machine Learning,Mathematical Finance
Logic-Based Switching Control for Stabilization of Stochastic Feedforward Nonlinear Systems

Miao Yu

DOI: https://doi.org/10.1109/tac.2019.2954800

IF: 6.549

2019-01-01

IEEE Transactions on Automatic Control

Abstract:This article addresses the stabilization problem for a class of stochastic feedforward systems with unknown nonlinearities. Due to the unknown nonlinearities, a nested saturation control law is designed with a priori unknown saturation bounds. Then, a group of logic-based switching conditions are provided for tuning the saturation bounds adaptively. Global asymptotical stability in the probability of the system states is proven. The key features of the control approach lie in that both explicit bounds and specific forms of the nonlinearities could be unknown for designing and deploying the controller.
A Second-Order Maximum Principle for Singular Optimal Stochastic Controls

Shanjian Tang

DOI: https://doi.org/10.3934/dcdsb.2010.14.1581

2010-01-01

Abstract:A singular optimal stochastic control problem is studied. A second-order maximum principle is presented. The second-order adjoint processes are involved, though the diffusion of the control system is control independent. The range theorem of vector-valued measures is used to prove the maximum principle. Examples are given to illustrate the applications.
Controllability Gramian for Stochastic Game-Based Systems

Feng Liu,Zhiyong Yu

DOI: https://doi.org/10.1109/tac.2022.3232181

IF: 6.549

2022-01-01

IEEE Transactions on Automatic Control

Abstract:This article is concerned with the exact controllability of a linear stochastic game-based control system (SGBCS) with time-variant random coefficients. Some necessary conditions and sufficient conditions for the exact terminal controllability of the SGBCS are obtained. Then, it is proved that the exact controllability of the SGBCS, the nonsingularity of a generalized version of the Gramian matrix, the validity of an observability inequality for the adjoint equation, and the unique solvability of a family of minimization problems are all equivalent. An admissible control for the regulator is constructed in closed form, which can steer the macrostate from any existing initial vector to any desired terminal random variable.

automation & control systems,engineering, electrical & electronic
Near Optimal Control for a Class of Stochastic Hybrid Systems.

Chunyue Song,Ping Li

DOI: https://doi.org/10.1016/j.automatica.2010.05.024

IF: 6.4

2010-01-01

Automatica

Abstract:To address a computationally intractable optimal control problem for a class of stochastic hybrid systems, this paper proposes a near optimal state feedback control scheme, which is constructed by using a statistical prediction method based on approximate numerical solution that samples over the entire state space. A numerical example illustrates the potential of the approach.
General Necessary Conditions for Partially Observed Optimal Stochastic Controls

XJ Li,SJ Tang

DOI: https://doi.org/10.2307/3215225

IF: 1.1161

1995-01-01

Journal of Applied Probability

Abstract:The partially observed control problem is considered for stochastic processes with control entering into the diffusion and the observation. The maximum principle is proved for the partially observable optimal control. A pure probabilistic approach is used, and the adjoint processes are characterized as solutions of related backward stochastic differential equations in finite-dimensional spaces. Most of the derivation is identified with that of the completely observable case.
A singular control model with application to the goodwill problem

Andrew Jack,Timothy C. Johnson,Mihail Zervos

DOI: https://doi.org/10.1016/j.spa.2008.01.001

IF: 1.43

2008-11-01

Stochastic Processes and their Applications

Abstract:We consider a stochastic system whose uncontrolled state dynamics are modelled by a general one-dimensional Itô diffusion. The control effort that can be applied to this system takes the form that is associated with the so-called monotone follower problem of singular stochastic control. The control problem that we address aims at maximising a performance criterion that rewards high values of the utility derived from the system’s controlled state but penalises any expenditure of control effort. This problem has been motivated by applications such as the so-called goodwill problem in which the system’s state is used to represent the image that a product has in a market, while control expenditure is associated with raising the product’s image, e.g., through advertising. We obtain the solution to the optimisation problem that we consider in a closed analytic form under rather general assumptions. Also, our analysis establishes a number of results that are concerned with analytic as well as probabilistic expressions for the first derivative of the solution to a second-order linear non-homogeneous ordinary differential equation. These results have independent interest and can potentially be of use to the solution of other one-dimensional stochastic control problems.

statistics & probability
Singular stochastic control problems motivated by the optimal sustainable exploitation of an ecosystem

Gechun Liang,Zhesheng Liu,Mihail Zervos

2024-10-06

Abstract:We derive the explicit solutions to singular stochastic control problems of the monotone follower type with (a) an expected discounted criterion, (b) an expected ergodic criterion and (c) a pathwise ergodic criterion. These problems have been motivated by the optimal sustainable exploitation of an ecosystem, such as a natural fishery. Under general assumptions on the diffusion coefficients, the discounting rate function, the running payoff function and the marginal profit of control action, we show that the optimal strategies are of a threshold type. We solve the three problems by first constructing suitable solutions to their associated HJB equations, which take the form of quasi-variational inequalities with gradient constraints. In the cases of the ergodic control problems, we also use a suitable new variational argument. Furthermore, we establish the convergence of the solution of the discounted control problem to the one of the ergodic control problems as the discounting rate function tends to 0 in an Abelian sense.

Optimization and Control,Probability
Stochastic Singular Linear Systems and Related Linear-Quadratic Optimal Control Problems under Finite and Infinite Horizons

Mengzhen Li,Tianyang Nie,Zhen Wu

2024-09-03

Abstract:In this paper, we study the necessary and sufficient conditions for ensuring the well-posedness of the stochastic singular systems. Moreover, we investigate the stochastic singular linear-quadratic control problems, considering both finite and infinite horizons, and transform each of these problems into their corresponding normal linear-quadratic control problem. To guarantee the finiteness of the infinite-horizon linear-quadratic control problem, we establish the Popov-Belevitch-Hautus rank criterion for accessing the controllability of the stochastic system. Furthermore, we derive the feedback form of the optimal control. Finally, we provide solutions for illustrated examples of the stochastic singular linear-quadratic control problem in both finite and infinite horizons.

Optimization and Control
Stochastic Stability and Stabilization for a Class of Singular Hybrid Systems

杨莹,李俊民,陈国培

DOI: https://doi.org/10.3969/j.issn.1001-2400.2010.05.016

2010-01-01

Journal of Xidian University

Abstract:Based on the method combining the techniques of the multiple Lyapunov function and stochastic generalized Lyapunov function,a necessary and sufficient condition is derived for discrete-time singular hybrid systems with the time-homogenous finite state Markov chain without using the restricted equivalent property of singular systems.The condition is given in terms of coupled generalized Lyapunov equations(CGLEs) such that the solution of the systems is stochastically stable.The equations can be solved by changing them into strict linear matrix inequalities(SLMIs).The result is extended to solve the stabilization problem and the design of a state-feedback controller.A numerical example shows the effectiveness of the proposed approach.
Zero-Sum Stopper Versus Singular-Controller Games with Constrained Control Directions

Andrea Bovo,Tiziano De Angelis,Jan Palczewski

DOI: https://doi.org/10.1137/23m1579558

IF: 2.2

2024-07-19

SIAM Journal on Control and Optimization

Abstract:SIAM Journal on Control and Optimization, Volume 62, Issue 4, Page 2203-2228, August 2024. We consider a class of zero-sum stopper versus singular-controller games in which the controller can only act on a subset [math] of the [math] coordinates of a controlled diffusion. Due to the constraint on the control directions these games fall outside the framework of recently studied variational methods. In this paper we develop an approximation procedure, based on [math]-stability estimates for the controlled diffusion process and almost sure convergence of suitable stopping times. That allows us to prove existence of the game's value and to obtain an optimal strategy for the stopper under continuity and growth conditions on the payoff functions. This class of games is a natural extension of (single-agent) singular control problems, studied in the literature, with similar constraints on the admissible controls.

mathematics, applied,automation & control systems
Ensemble Control of Stochastic Linear Systems

Ji Qi,Anatoly Zlotnik,Jr-Shin Li

DOI: https://doi.org/10.48550/arXiv.1201.1857

2012-01-10

Abstract:In this paper, we consider the problem of steering a family of independent, structurally identical, finite-dimensional stochastic linear systems with variation in system parameters between initial and target states of interest by using an open-loop control function. Our exploration of this class of control problems, which falls under the rising subject of ensemble control, is motivated by pulse design problems in quantum control. Here we extend the concept of ensemble control to stochastic systems with additive diffusion and jump processes, which we model using Brownian motion and Poisson counters, respectively, and consider optimal steering problems. We derive a Fredholm integral equation that is used to solve for the optimal control, which minimizes both the mean square error (MSE) and the error in the mean of the terminal state. In addition, we present several example control problems for which optimal solutions are computed by numerically approximating the singular system of the associated Fredholm operator. We use Monte Carlo simulations to illustrate the performance of the resulting controls. Our work has immediate practical applications to the control of dynamical systems with additive noise and parameter dispersion, and also makes an important contribution to stochastic control theory.

Optimization and Control,Dynamical Systems
Control theory for stochastic distributed parameter systems, an engineering perspective

Qi Lü,Xu Zhang

DOI: https://doi.org/10.1016/j.arcontrol.2021.04.002

IF: 9.4

2021-01-01

Annual Reviews in Control

Abstract:The main purpose of this paper is to survey some recent progresses on control theory for stochastic distributed parameter systems, i.e., systems governed by stochastic differential equations in infinite dimensions, typically by stochastic partial differential equations. We will explain the new phenomenon and difficulties in the study of controllability and optimal control problems for one dimensional stochastic parabolic equations and stochastic hyperbolic equations. In particular, we shall see that both the formulation of corresponding stochastic control problems and the tools to solve them may differ considerably from their deterministic/finite-dimensional counterparts. More importantly, one has to develop new tools, say, the stochastic transposition method introduced in our previous works, to solve some problems in this field.

automation & control systems
Stabilization of Multi-Input Stochastic Nonlinear Systems

XP Liu,GX Gu,KM Zhou

DOI: https://doi.org/10.1109/cdc.1998.758465

1998-01-01

Abstract:This paper studies the problem of globally asymptotic stability in probability for a class of multi-input stochastic nonlinear systems. We assume that the system under consideration is in a special form, which is much more general than a strict feedback form. We then show by recursively using backstepping and augmentation that the globally asymptotic stability in probability of the system can be achieved via a static state feedback.
Solution to the Generalized Champagne Problem on Simultaneous Stabilization of Linear Systems

Guan Qiang,Wang Long,Xia BiCan,Yang Lu,Yu WenSheng,Zeng ZhenBing

DOI: https://doi.org/10.1007/s11432-007-0053-2

2007-01-01

Abstract:The well-known Generalized Champagne Problem on simultaneous stabilization of linear systems is solved by using complex analysis and Blondel's technique. We give a complete answer to the open problem proposed by Patel et al., which automatically includes the solution to the original Champagne Problem. Based on the recent development in automated inequality-type theorem proving, a new stabilizing controller design method is established. Our numerical examples significantly improve the relevant results in the literature.

Generalization of a Class of Singular Stochastic Control Problem

Generalized guaranteed cost control with D-stability and multiple output constraints.

Robust Control For Singular Systems With State Delay And Parameter Uncertainty

Stochastic stabilization of quasi-generalized Hamiltonian systems

Singular LQ Sub-Optimal Control Problem for a Class of Discrete-Time Singular Systems with Time-Delay

Stability and Stabilization of a Class of Switched Stochastic Systems with Saturation Control

Exploratory Optimal Stopping: A Singular Control Formulation

Logic-Based Switching Control for Stabilization of Stochastic Feedforward Nonlinear Systems

A Second-Order Maximum Principle for Singular Optimal Stochastic Controls

Controllability Gramian for Stochastic Game-Based Systems

Near Optimal Control for a Class of Stochastic Hybrid Systems.

General Necessary Conditions for Partially Observed Optimal Stochastic Controls

A singular control model with application to the goodwill problem

Singular stochastic control problems motivated by the optimal sustainable exploitation of an ecosystem

Stochastic Singular Linear Systems and Related Linear-Quadratic Optimal Control Problems under Finite and Infinite Horizons

Stochastic Stability and Stabilization for a Class of Singular Hybrid Systems

Zero-Sum Stopper Versus Singular-Controller Games with Constrained Control Directions

Ensemble Control of Stochastic Linear Systems

Control theory for stochastic distributed parameter systems, an engineering perspective

Stabilization of Multi-Input Stochastic Nonlinear Systems

Solution to the Generalized Champagne Problem on Simultaneous Stabilization of Linear Systems