Abstract:This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded , and the costs are assumed to be bounded below . By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we prove the existence of a discounted cost optimal stationary policy by providing a value iteration technique. Moreover, by developing a new average cost, minimum nonnegative solution method, we prove the existence of an average cost optimal stationary policy under some reasonably mild conditions. Finally, we use some examples to illustrate applications of our results. Except that the costs are assumed to be bounded below, the conditions for the existence of discounted cost (or average cost) optimal policies are much weaker than those in the previous literature, and the minimum nonnegative solution approach is new .

Unbounded Cost Markov Decision Processes with Limsup and Liminf Average Criteria: New Conditions

Constrained Continuous-Time Markov Decision Processes with Average Criteria

A Note on Optimality Conditions for Continuous-Time Markov Decision Processes with Average Cost Criterion

Denumerable Continuous-Time Markov Decision Processes with Multiconstraints on Average Costs

Markov Decision Problems with Unbounded Transition Rates under Discounted-Cost Performance Criteria

A New Condition for the Existence of Optimal Stationary Policies in Denumerable State Average Cost Continuous Time Markov Decision Processes with Unbounded Cost and Transition Rates

Markov Decision Processes with Variance Minimization: A New Condition and Approach

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

Average cost Markov decision processes with countable state spaces

Another Set of Conditions for Markov Decision Processes with Average Sample-Path Costs

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Optimality Conditions for CTMDP with Average Cost Criterion

Limiting Average Criteria for Nonstationary Markov Decision Processes

A Note on the Existence of Optimal Stationary Policies for Average Markov Decision Processes with Countable States

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Average Optimality for Markov Decision Processes in Borel Spaces: a New Condition and Approach

New Average Optimality Conditions for Semi-Markov Decision Processes in Borel Spaces.

Nonstationary Denumerable State Markov Decision Processes – with Average Variance Criterion

Constrained Total Undiscounted Continuous-Time Markov Decision Processes

Constrained Markov Decision Processes with First Passage Criteria