Abstract:In this paper, we study the optimal control of service rates in a queueing system with a Markovian arrival process (MAP) and exponential service times. The service rate is allowed to be state dependent, i.e., we can adjust the service rate according to the queue length and the phase of the MAP. The cost function consists of holding cost and operating cost. The goal is to find the optimal service rates that minimize the long-run average total cost. To achieve that, we use the matrix-analytic methods (MAM) together with the sensitivity-based optimization (SBO) theory. A performance difference formula is derived, which can quantify the difference of the long-run average total cost under any two different settings of service rates. Based on the difference formula, we show that the long-run average total cost is monotone in the service rate and the optimal control is a bang-bang control. We also show that, under some mild conditions, the optimal control policy of service rates is of a quasi-threshold-type. By utilizing the MAM theory, we propose a recursive algorithm to compute the value function related quantities. An iterative algorithm to efficiently find the optimal policy, which is similar to policy iteration, is proposed based on the SBO theory. We further study some special cases of the problem, such as the optimality of the threshold-type policy for the M/M/1 queue. Finally, a number of numerical examples are presented to demonstrate the main results and explore the impact of the phase of the MAP on the optimization in the MAP/M/1 queue.

A New Condition for the Existence of Optimal Stationary Policies in Denumerable State Average Cost Continuous Time Markov Decision Processes with Unbounded Cost and Transition Rates

A Note on the Existence of Optimal Stationary Policies for Average Markov Decision Processes with Countable States

A Note on Optimality Conditions for Continuous-Time Markov Decision Processes with Average Cost Criterion

New Discount and Average Optimality Conditions for Continuous-Time Markov Decision Processes

Average Optimality and Constrained Average Optimality for Controlled Queuing Systems

Constrained Continuous-Time Markov Decision Processes with Average Criteria

Unbounded Cost Markov Decision Processes with Limsup and Liminf Average Criteria: New Conditions

Drift and Monotonicity Conditions for Continuous-Time Controlled Markov Chains with an Average Criterion.

Optimal Stationary Policies for a Class of Countable Markov Control Processes

Denumerable Continuous-Time Markov Decision Processes with Multiconstraints on Average Costs

Verifiable conditions for average optimality of continuous-time Markov decision processes.

Average cost Markov decision processes with countable state spaces

A Semimartingale Characterization of Average Optimal Stationary Policies for Markov Decision Processes

Another Set of Conditions for Markov Decision Processes with Average Sample-Path Costs

Another Set of Verifiable Conditions for Average Markov Decision Processes with Borel Spaces

Constrained Continuous-Time Markov Control Processes with Discounted Criteria

Optimal Control of State-Dependent Service Rates in a MAP/M/1 Queue

Risk-sensitive Average Continuous-Time Markov Decision Processes with Unbounded Transition and Cost Rates.

Average Optimality for Markov Decision Processes in Borel Spaces: a New Condition and Approach

Another Set of Conditions for Strongn(n = −1, 0) Discount Optimality in Markov Decision Processes

Optimality Conditions for CTMDP with Average Cost Criterion