Abstract:This paper deals with continuous-time Markov decision processes in Polish spaces, under the discounted and average cost criteria. All underlying Markov processes are determined by given transition rates which are allowed to be unbounded , and the costs are assumed to be bounded below . By introducing an occupation measure of a randomized Markov policy and analyzing properties of occupation measures, we first show that the family of all randomized stationary policies is ‘sufficient’ within the class of all randomized Markov policies. Then, under the semicontinuity and compactness conditions, we prove the existence of a discounted cost optimal stationary policy by providing a value iteration technique. Moreover, by developing a new average cost, minimum nonnegative solution method, we prove the existence of an average cost optimal stationary policy under some reasonably mild conditions. Finally, we use some examples to illustrate applications of our results. Except that the costs are assumed to be bounded below, the conditions for the existence of discounted cost (or average cost) optimal policies are much weaker than those in the previous literature, and the minimum nonnegative solution approach is new .

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

Risk-sensitive Continuous-Time Markov Decision Processes with Unbounded Rates and Borel Spaces

Finite Horizon Risk-Sensitive Continuous-Time Markov Decision Processes with Unbounded Transition and Cost Rates

Risk-sensitive Average Continuous-Time Markov Decision Processes with Unbounded Transition and Cost Rates.

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

Finite-horizon Optimality for Continuous-Time Markov Decision Processes with Unbounded Transition Rates

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

Risk Probability Minimization Problems for Continuous-Time Markov Decision Processes on Finite Horizon

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Risk-sensitive Infinite-Horizon Discounted Piecewise Deterministic Markov Decision Processes

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Continuous-Time Markov Decision Processes with Discounted Rewards: the Case of Polish Spaces

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

First Passage Optimality for Continuous-Time Markov Decision Processes with Varying Discount Factors and History-Dependent Policies

Discounted Continuous-Time Constrained Markov Decision Processes in Polish Spaces

Continuous Time Markov Decision Processes with Nonuniformly Bounded Transition Rate: Expected Total Rewards

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Nonstationary Markov Decision Processes with Risk Probability Criteria