Abstract:The optimization problems of Markov control processes (MCPs) with exact knowledge of system parameters, in the form of transition probabilities or infinitesimal transition rates, can be solved by using the concept of Markov performance potential which plays an important role in the sensitivity analysis of MCPs. In this paper, by using an equivalent infinitesimal generator, we first introduce a definition of discounted Poisson equations for semi-Markov control processes (SMCPs), which is similar to that for MCPs, and the performance potentials of SMCPs are defined as solution of the equation. Some related optimization techniques based on performance potentials for MCPs may be extended to the optimization of SMCPs if the system parameters are known with certainty. Unfortunately, exact values of the distributions of the sojourn times at some states or the transition probabilities of the embedded Markov chain for a large-scale SMCP are generally difficult or impossible to obtain, which leads to the uncertainty of the semi-Markov kernel, and thereby to the uncertainty of equivalent infinitesimal transition rates. Similar to the optimization of uncertain MCPs, a potential-based policy iteration method is proposed in this work to search for the optimal robust control policy for SMCPs with uncertain infinitesimal transition rates that are represented as compact sets. In addition, convergence of the algorithm is discussed.

Approximate Policy Iteration for Robust Stochastic Control of Multi-agent Markov Decision Processes

Robust optimal policies for team Markov games

Model-Based Robot Learning Control with Uncertainty Directed Exploration

Optimal Control of Robust Team Stochastic Games

Approximate Linear Programming for Decentralized Policy Iteration in Cooperative Multi-agent Markov Decision Processes

Nonfragile Near-Optimal Control of Stochastic Time-Varying Multiagent Systems with Control- and State-Dependent Noises

Adaptive Optimal Robust Control for Uncertain Nonlinear Systems Using Neural Network Approximation in Policy Iteration

Online Pareto Optimal Control of Mean-Field Stochastic Multi-Player Systems Using Policy Iteration

Approximating Pareto Frontier Through Bayesian-optimization-directed Robust Multi-objective Reinforcement Learning

Approximate Dynamic Programming for Nonlinear-Constrained Optimizations

Temporal Difference-Based Policy Iteration for Optimal Control of Stochastic Systems

Policy Gradient Algorithms for Robust MDPs with Non-Rectangular Uncertainty Sets

Finite-horizon Approximate Optimal Consensus Control for Discrete-time Nonlinear Multi-Agent Systems with Unknown Dynamics

Learning Robust Policies for Uncertain Parametric Markov Decision Processes

Online Pareto optimal control of mean-ﬁeld stochastic multi-player systems using policy iteration

An Off‐policy Iteration Algorithm for Robust Stabilization of Constrained‐input Uncertain Nonlinear Systems

A learning optimal control scheme for robust stabilization of a class of uncertain nonlinear systems

Simulation-Based Optimistic Policy Iteration For Multi-Agent MDPs with Kullback-Leibler Control Cost

The Optimal Robust Control Policy for Uncertain Semi-Markov Control Processes

Policy Iteration and Event-Triggered Robust Adaptive Dynamic Programming for Large-Scale Systems

Policy Iteration Based Feedback Control