Abstract:This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded , and the costs are assumed to be bounded below . By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we prove the existence of a discounted cost optimal stationary policy by providing a value iteration technique. Moreover, by developing a new average cost, minimum nonnegative solution method, we prove the existence of an average cost optimal stationary policy under some reasonably mild conditions. Finally, we use some examples to illustrate applications of our results. Except that the costs are assumed to be bounded below, the conditions for the existence of discounted cost (or average cost) optimal policies are much weaker than those in the previous literature, and the minimum nonnegative solution approach is new .

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

Discounted Continuous-Time Constrained Markov Decision Processes in Polish Spaces

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Denumerable Continuous-Time Markov Decision Processes with Multiconstraints on Average Costs

Constrained Total Undiscounted Continuous-Time Markov Decision Processes

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

A survey of recent results on continuous-time Markov decision processes

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Constrained Continuous-Time Markov Decision Processes with Average Criteria

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Risk-sensitive Average Continuous-Time Markov Decision Processes with Unbounded Transition and Cost Rates.

Multiconstrained Finite-Horizon Piecewise Deterministic Markov Decision Processes with Unbounded Transition Rates

Risk-sensitive Continuous-Time Markov Decision Processes with Unbounded Rates and Borel Spaces

Finite-horizon Optimality for Continuous-Time Markov Decision Processes with Unbounded Transition Rates

Constrained Markov Decision Processes with Non-constant Discount Factor

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

Constrained Markov Decision Processes with First Passage Criteria

Convergence of Markov Decision Processes with Constraints and State-Action Dependent Discount Factors

Markov Decision Problems with Unbounded Transition Rates under Discounted-Cost Performance Criteria