Abstract:In this paper, the infinite horizon optimal control problems for singular diffusion processes are considered from the viewpoints of Markov decision processes and perturbation analysis, where the singularity of diffusion means that the covariance matrix of the system noise is allowed to be degenerate. A formula of performance difference under two different controls is derived and leads to a comparison theorem. By the comparison theorem, starting from a control, a so-called better control can be selected. Therefore, a control policy iteration algorithm is developed, by which the performance improves step by step and converges to the optimal one. When this applies to the stochastic affine nonlinear regulator and stochastic linear quadratic optimal control problems, better control can be constructed in a closed form. It is also shown that when the considered stochastic systems degenerate to the deterministic ones, the proposed algorithm reduces to the adaptive dynamic programming algorithm [J. J. Murray, C. J. Cox, G. G. Lendaris, and R. Saeks, Adaptive dynamic programming, IEEE Trans. Systems Man Cybernet., 32 (2002), pp. 140-153] for the affine nonlinear systems and to the well-known Kleinman algorithm [D. L. Kleinman, On an iterative technique for Riccati equation computation, IEEE Trans. Automat. Control, 13 (1968), pp. 114-115] for the linear quadratic optimal control problem.

Optimal Singular Control Problem in Infinite Horizon for Stochastic Processes with Regime-Switching

From the Optimal Singular Stochastic Control to the Optimal Stopping for Regime-Switching Processes.

Singular Linear Quadratic Optimal Control for Singular Stochastic Discrete‐time Systems

Stochastic Singular Linear Systems and Related Linear-Quadratic Optimal Control Problems under Finite and Infinite Horizons

The existence of optimal feedback controls for stochastic dynamical systems with regime-switching

Stochastic linear-quadratic optimal control problems with non-Markovian regime switching

Stochastic linear-quadratic optimal control problems with multi-dimensional state and non-Markovian regime switching

Stochastic linear quadratic optimal control problems with regime-switching jumps in infinite horizon

TIME-INCONSISTENT OPTIMAL CONTROL PROBLEMS WITH REGIME-SWITCHING

Stability Analysis and Optimal Control of Stochastic Singular Systems.

Singular Optimal Controls of Stochastic Recursive Systems and Hamilton–Jacobi–Bellman Inequality

Optimal control of a class of deterministic switched systems

Finite and infinite horizon indefinite linear quadratic optimal control for discrete-time singular Markov jump systems

Singular Optimal Controls for Stochastic Recursive Systems under Convex Control Constraint

A stochastic hybrid optimal control with regime switching and recursive cost functional

Singular stochastic control problems motivated by the optimal sustainable exploitation of an ecosystem

Constrained stochastic linear quadratic control under regime switching with controlled jump size

The Existence of Time-Optimal Control for Singular Systems

Singular Optimal Control for Uncertain Systems

The Time-Optimal Control Of Singular Systems

Policy Iteration Algorithm for Singular Controlled Diffusion Processes