Abstract:We consider continuous-time Markov decision processes in Polish spaces. The performance of a control policy is measured by the expected discounted reward criterion associated with state-dependent discount factors. All underlying Markov processes are determined by the given transition rates which are allowed to be unbounded , and the reward rates may have neither upper nor lower bounds . By using the dynamic programming approach , we establish the discounted reward optimality equation (DROE) and the existence and uniqueness of its solutions. Under suitable conditions, we also obtain a discounted optimal stationary policy which is optimal in the class of all randomized stationary policies. Moreover, when the transition rates are uniformly bounded, we provide an algorithm to compute (or at least to approximate) the discounted reward optimal value function as well as a discounted optimal stationary policy. Finally, we use an example to illustrate our results. Specially, we first derive an explicit and exact solution to the DROE and an explicit expression of a discounted optimal stationary policy for such an example.

Continuous Time Markov Decision Processes with Nonuniformly Bounded Transition Rate: Expected Total Rewards

Continuous Time Markov Decision Processes with Expected Discounted Total Rewards

Total Reward Criteria for Unconstrained/constrained Continuous-Time Markov Decision Processes.

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Continuous-Time Markov Decision Processes with Discounted Rewards: the Case of Polish Spaces

Continuous-Time Controlled Markov Chains with Discounted Rewards

Continuous-Time Markov Decision Processes with State-Dependent Discount Factors

Finite-horizon Optimality for Continuous-Time Markov Decision Processes with Unbounded Transition Rates

Risk-Sensitive Discounted Continuous-Time Markov Decision Processes with Unbounded Rates.

The Finiteness of the Reward Function and the Optimal Value Function in Markov Decision Processes

Risk-sensitive Continuous-Time Markov Decision Processes with Unbounded Rates and Borel Spaces

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

Constrained Denumerable State Non-Stationary MDPs with Expected Total Reward Criterion

Constrained Continuous-Time Markov Decision Processes with Average Criteria

Constrained Total Undiscounted Continuous-Time Markov Decision Processes

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

Continuous Time Markov Decision Processes with Discounted Moment Criterion

Analysis for Some Properties of Discrete Time Markov Decision Processes

Nonstationary Markov Decision Processes with Risk Probability Criteria