Abstract:This paper is devoted to studying constrained continuous-time Markov decision processes (MDPs) in the class of randomized policies depending on state histories. The transition rates may be unbounded, the reward and costs are admitted to be unbounded from above and from below, and the state and action spaces are Polish spaces. The optimality criterion to be maximized is the expected discounted rewards, and the constraints can be imposed on the expected discounted costs. First, we give conditions for the nonexplosion of underlying processes and the finiteness of the expected discounted rewards/costs. Second, using a technique of occupation measures, we prove that the constrained optimality of continuous-time MDPs can be transformed to an equivalent (optimality) problem over a class of probability measures. Based on the equivalent problem and a so-called $\bar{w}$-weak convergence of probability measures developed in this paper, we show the existence of a constrained optimal policy. Third, by providing a linear programming formulation of the equivalent problem, we show the solvability of constrained optimal policies. Finally, we use two computable examples to illustrate our main results.

Constrained Markov Decision Processes with First Passage Criteria

Constrained Continuous-Time Markov Decision Processes with Average Criteria

First Passage Markov Decision Processes with Constraints and Varying Discount Factors

First Passage Optimality for Continuous-Time Markov Decision Processes with Varying Discount Factors and History-Dependent Policies

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Convergence of Markov Decision Processes with Constraints and State-Action Dependent Discount Factors

Constrained Total Undiscounted Continuous-Time Markov Decision Processes

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

First Passage Models for Denumerable Semi-Markov Decision Processes with Nonnegative Discounted Costs

Discounted Continuous-Time Constrained Markov Decision Processes in Polish Spaces

Constrained Denumerable State Non-Stationary MDPs with Expected Total Reward Criterion

Constrained Markov Decision Processes with Non-constant Discount Factor

The Risk Probability Criterion for Discounted Continuous-Time Markov Decision Processes

First Passage Risk Probability Minimization for Piecewise Deterministic Markov Decision Processes

Constrained Semi-Markov Decision Processes with Ratio and Time Expected Average Criteria in Polish Spaces

On the First Passage G-Mean-variance Optimality for Discounted Continuous-Time Markov Decision Processes.

Total Reward Criteria for Unconstrained/constrained Continuous-Time Markov Decision Processes.

Unbounded Cost Markov Decision Processes with Limsup and Liminf Average Criteria: New Conditions

Multiconstrained Finite-Horizon Piecewise Deterministic Markov Decision Processes with Unbounded Transition Rates

Markov Decision Processes with State-Dependent Discount Factors and Unbounded Rewards/costs.

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion