Abstract:In this paper, we consider the risk probability minimization problem for infinite discounted continuous-time Markov decision processes (CTMDPs) with unbounded transition rates. First, we introduce a class of policies depending on histories with the additional reward levels. Then, we construct the corresponding probability spaces, and also establish the non-explosion of the state process. Secondly, under suitable conditions we prove that the value function is a solution to the optimality equation for the probability criterion by an iteration technique, and obtain a value iteration algorithm to compute (at least approximate) the value function. Furthermore, under an additional condition we establish the uniqueness of the solution to the optimality equation and the existence of an optimal policy. Finally, we illustrate our results with two examples. The first one is used to verify our conditions for CTMDPs with unbounded transition rates, the second one for the numerical calculation of the value function and an optimal policy.

Discount Optimality for Unbounded Rewards

Discount Optimality for Nonnegative Costs

Finite-horizon Optimality for Continuous-Time Markov Decision Processes with Unbounded Transition Rates

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

First Passage Optimality for Continuous-Time Markov Decision Processes with Varying Discount Factors and History-Dependent Policies

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Average Optimality for Unbounded Rewards

Discounted Optimality for Continuous-Time Markov Decision Processes in Polish Spaces

Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Continuous-Time Markov Decision Processes with Discounted Rewards: the Case of Polish Spaces

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Constrained Optimality for Discount Criteria

Continuous-Time Controlled Markov Chains with Discounted Rewards

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

The Finiteness of the Reward Function and the Optimal Value Function in Markov Decision Processes

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

A survey of recent results on continuous-time Markov decision processes

Another Set of Conditions for Strongn(n = −1, 0) Discount Optimality in Markov Decision Processes

Continuous Time Markov Decision Processes with Nonuniformly Bounded Transition Rate: Expected Total Rewards