Abstract:We consider the dynamic scheduling problem in a single-server multi-class queueing system, where the accumulated holding costs of a customer is a convexly increasing function of the time that the customer spends in the system. This problem was already addressed by van Mieghem in his seminal paper (Ann Appl Probab 5:808–833, 1995). He launched the so-called generalized rule and proved its asymptotic optimality in the heavy traffic limit. The optimal scheduling problem with convex holding costs have also been studied in many other papers. However, the holding costs in these papers are typically count-convex, i.e., the aggregate holding cost rate related to a class of customers increases convexly as a function of the current number of customers in this class, which is clearly different from the time-convex holding costs introduced in van Mieghem (Ann Appl Probab 5:808–833, 1995) and studied also in the present paper. We utilize the Whittle index approach to develop a novel heuristic policy for the dynamic scheduling problem with time-convex holding costs. Instead of the original open version of the continuous-time scheduling problem, we apply the Whittle index approach to the corresponding discrete-time closed version of the problem. As the main theoretical contribution, we prove that, for IHR service times, the discounted discrete-time problem is indexable and also give an explicit expression for the Whittle index itself. A special challenge in our model is the two-dimensional state space with an infinite number of possible states. By utilizing the discrete-time results, we develop a novel index policy for the original continuous-time problem and demonstrate, by numerical simulations, that this Whittle index policy outperforms the generalized rule. Thus, while the generalized rule is asymptotically optimal in the heavy traffic limit, it can still be improved in normal traffic conditions.

Opportunistic Scheduling Revisited Using Restless Bandits: Indexability and Index Policy

Opportunistic Scheduling as Restless Bandits

Uncertainty-of-Information Scheduling: A Restless Multi-armed Bandit Framework

Remote Estimation of Gauss-Markov Processes over Multiple Channels: A Whittle Index Policy

Approximating the Optimal Competitive Ratio for an Ancient Online Scheduling Problem

Optimizing AoI at Query in Multiuser Wireless Uplink Networks: A Whittle Index Approach

Closed-Form Whittle’s Index-Enabled Random Access for Timely Status Update

Towards Efficient Dynamic Uplink Scheduling over Multiple Unknown Channels

Transmission scheduling of P2P real-time communication based on restless multi-armed bandit

Indexability of Finite State Restless Multi-Armed Bandit and Rollout Policy

Dynamic allocation indices for restless projects and queueing admission control: a polyhedral approach

PCL-Indexability and Whittle Index for Restless Bandits with General Observation Models

A unifying computations of Whittle's Index for Markovian bandits

Whittle Index Policy for Dynamic Multichannel Allocation in Remote State Estimation

Open Bandit Processes and Optimal Scheduling of Queueing Networks

Optimality of Myopic Policy for Restless Multiarmed Bandit with Imperfect Observation

A Whittle Index Approach to Minimizing Functions of Age of Information

Whittle index approach to the multi-class queueing systems with convex holding costs and IHR service times

Low-Complexity Algorithm for Restless Bandits with Imperfect Observations

Optimal Myopic Policy for Restless Bandit: A Perspective of Eigendecomposition

Index policies for optimal mean-variance trade-off of inter-delivery times in real-time sensor networks