Abstract:This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded , and the costs are assumed to be bounded below . By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we prove the existence of a discounted cost optimal stationary policy by providing a value iteration technique. Moreover, by developing a new average cost, minimum nonnegative solution method, we prove the existence of an average cost optimal stationary policy under some reasonably mild conditions. Finally, we use some examples to illustrate applications of our results. Except that the costs are assumed to be bounded below, the conditions for the existence of discounted cost (or average cost) optimal policies are much weaker than those in the previous literature, and the minimum nonnegative solution approach is new .

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Continuous-Time Controlled Markov Chains with Discounted Rewards

Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Continuous-Time Markov Decision Processes with Discounted Rewards: the Case of Polish Spaces

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Continuous Time Markov Decision Processes with Nonuniformly Bounded Transition Rate: Expected Total Rewards

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

Continuous Time Markov Decision Processes with Discounted Moment Criterion

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

First Passage Optimality for Continuous-Time Markov Decision Processes with Varying Discount Factors and History-Dependent Policies

Constrained Continuous-Time Markov Decision Processes with Average Criteria

Discounted Optimality for Continuous-Time Markov Decision Processes in Polish Spaces

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

Risk-sensitive Continuous-Time Markov Decision Processes with Unbounded Rates and Borel Spaces

Finite-horizon Optimality for Continuous-Time Markov Decision Processes with Unbounded Transition Rates

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

Discounted Continuous-Time Constrained Markov Decision Processes in Polish Spaces