Abstract:This paper is devoted to studying the average optimality in continuous-time Markov decision processes with fairly general state and action spaces. The criterion to be maximized is expected average rewards. The transition rates of underlying continuous-time jump Markov processes are allowed to be unbounded, and the reward rates may have neither upper nor lower bounds. We first provide two optimality inequalities with opposed directions, and also give suitable conditions under which the existence of solutions to the two optimality inequalities is ensured. Then, from the two optimality inequalities we prove the existence of optimal (deterministic) stationary policies by using the Dynkin formula. Moreover, we present a "semi martingale characterization" of an optimal stationary policy. Finally, we use a generalized Potlach process with control to illustrate the difference between our conditions and those in the previous literature, and then further apply our results to average optimal control problems of generalized birth-death systems, upwardly skip-free processes and two queueing systems. The approach developed in this paper is slightly different from the "optimality inequality approach" widely used in the previous literature.

Another Set of Verifiable Conditions for Average Markov Decision Processes with Borel Spaces

Verifiable conditions for average optimality of continuous-time Markov decision processes.

Average Optimality for Markov Decision Processes in Borel Spaces: a New Condition and Approach

New Average Optimality Conditions for Semi-Markov Decision Processes in Borel Spaces.

Nonhomogeneous Markov Decision Processes with Borel State Space-The Average Criterion with Nonuniformly Bounded Rewards.

On Average Optimality for Non-Stationary Markov Decision Processes in Borel Spaces

The Average Variance Criterion for Nonstationary MDP with Borel State Space

A Semimartingale Characterization of Average Optimal Stationary Policies for Markov Decision Processes

Markov Decision Processes with Variance Minimization: A New Condition and Approach

Average cost Markov decision processes with countable state spaces

A Note on the Existence of Optimal Stationary Policies for Average Markov Decision Processes with Countable States

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

A Note on Optimality Conditions for Continuous-Time Markov Decision Processes with Average Cost Criterion

Limiting Average Criteria for Nonstationary Markov Decision Processes

Another Set of Conditions for Markov Decision Processes with Average Sample-Path Costs

Average Optimality in Markov Decision Processes with Unbounded Rewards

Nonstationary Denumerable State Markov Decision Processes – with Average Variance Criterion

A New Condition for the Existence of Optimal Stationary Policies in Denumerable State Average Cost Continuous Time Markov Decision Processes with Unbounded Cost and Transition Rates

Drift and Monotonicity Conditions for Continuous-Time Controlled Markov Chains with an Average Criterion.

Average Optimality For Continuous-Time Markov Decision Processes In Polish Spaces

An Average-Value-at-risk Criterion for Markov Decision Processes with Unbounded Costs