Abstract:This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first provide two optimality inequalities with opposed directions, and also give suitable conditions under which the existence of solutions to the two optimality inequalities is ensured. Then, from the two optimality inequalities we prove the existence of optimal (deterministic) stationary policies by using the Dynkin formula. Moreover, we present a "semi martingale characterization" of an optimal stationary policy. Finally, we use a generalized Potlach process with control to illustrate the difference between our conditions and those in the previous literature, and then further apply our results to average optimal control problems of generalized birth-death systems, upwardly skip-free processes and two queueing systems. The approach developed in this paper is slightly different from the "optimality inequality approach" widely used in the previous literature.

Constrained Continuous-Time Markov Decision Processes with Average Criteria

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Denumerable Continuous-Time Markov Decision Processes with Multiconstraints on Average Costs

Constrained Optimization for Average Cost Continuous-Time Markov Decision Processes.

A Note on Optimality Conditions for Continuous-Time Markov Decision Processes with Average Cost Criterion

Total Reward Criteria for Unconstrained/constrained Continuous-Time Markov Decision Processes.

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

Optimality of Mixed Policies for Average Continuous-Time Markov Decision Processes with Constraints

Constrained Markov Decision Processes with First Passage Criteria

Constrained Total Undiscounted Continuous-Time Markov Decision Processes

Average Optimality For Continuous-Time Markov Decision Processes In Polish Spaces

Linear Programming and Constrained Average Optimality for General Continuous-Time Markov Decision Processes in History-Dependent Policies.

Unbounded Cost Markov Decision Processes with Limsup and Liminf Average Criteria: New Conditions

Constrained Semi-Markov Decision Processes with Ratio and Time Expected Average Criteria in Polish Spaces

Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates

Drift and Monotonicity Conditions for Continuous-Time Controlled Markov Chains with an Average Criterion.

Discounted Continuous-Time Markov Decision Processes with Constraints: Unbounded Transition and Loss Rates

Absorbing Continuous-Time Markov Decision Processes with Total Cost Criteria

Denumerable-state Continuous-Time Markov Decision Processes with Unbounded Transition and Reward Rates under the Discounted Criterion

Nonstationary Denumerable State Markov Decision Processes – with Average Variance Criterion

Discounted Continuous-Time Constrained Markov Decision Processes in Polish Spaces