Abstract:This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded , and the costs are assumed to be bounded below . By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we prove the existence of a discounted cost optimal stationary policy by providing a value iteration technique. Moreover, by developing a new average cost, minimum nonnegative solution method, we prove the existence of an average cost optimal stationary policy under some reasonably mild conditions. Finally, we use some examples to illustrate applications of our results. Except that the costs are assumed to be bounded below, the conditions for the existence of discounted cost (or average cost) optimal policies are much weaker than those in the previous literature, and the minimum nonnegative solution approach is new .

Discount Optimality for Nonnegative Costs

Average Optimality for Nonnegative Costs

Discount Optimality for Unbounded Rewards

Constrained Optimality for Discount Criteria

Another Set of Conditions for Strongn(n = −1, 0) Discount Optimality in Markov Decision Processes

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

First Passage Models for Denumerable Semi-Markov Decision Processes with Nonnegative Discounted Costs

A Note on Optimality Conditions for Continuous-Time Markov Decision Processes with Average Cost Criterion

Optimality Conditions for CTMDP with Average Cost Criterion

Discrete-Time Hybrid Control Processes with Unbounded Costs

Discounted Optimality for Continuous-Time Markov Decision Processes in Polish Spaces

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Average Optimality for Markov Decision Processes in Borel Spaces: a New Condition and Approach

Constrained Optimality for Average Criteria

Optimal Stationary Policies for Semi-Markov Control Processes with Discounted-Cost Criteria

Optimal Stopping Time on Discounted Semi-Markov Processes

Markov Decision Problems with Unbounded Transition Rates under Discounted-Cost Performance Criteria

Performance Optimization of Semi-Markov Decision Processes with Discounted-cost Criteria.

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Finite-horizon Optimality for Continuous-Time Markov Decision Processes with Unbounded Transition Rates

First Passage Optimality for Continuous-Time Markov Decision Processes with Varying Discount Factors and History-Dependent Policies