Abstract:We study a general problem of allocating limited resources to heterogeneous customers over time under model uncertainty. Each type of customer can be serviced using different actions, each of which stochastically consumes some combination of resources and returns different rewards for the resources consumed. We consider a general model in which the resource consumption distribution associated with each customer type–action combination is not known but is consistent and can be learned over time. In addition, the sequence of customer types to arrive over time is arbitrary and completely unknown. We overcome both the challenges of model uncertainty and customer heterogeneity by judiciously synthesizing two algorithmic frameworks from the literature: inventory balancing, which “reserves” a portion of each resource for high-reward customer types that could later arrive based on competitive ratio analysis, and online learning, which “explores” the resource consumption distributions for each customer type under different actions based on regret analysis. We define an auxiliary problem, which allows for existing competitive ratio and regret bounds to be seamlessly integrated. Furthermore, we propose a new variant of upper confidence bound (UCB), dubbed lazyUCB, which conducts less exploration in a bid to focus on “exploitation” in view of the resource scarcity. Finally, we construct an information-theoretic family of counterexamples to show that our integrated framework achieves the best possible performance guarantee. We demonstrate the efficacy of our algorithms on both synthetic instances generated for the online matching with stochastic rewards problem under unknown probabilities and a publicly available hotel data set. Our framework is highly practical in that it requires no historical data (no fitted customer choice models or forecasting of customer arrival patterns) and can be used to initialize allocation strategies in fast-changing environments. This paper was accepted by J. George Shanthikumar, Management Science Special Section on Data-Driven Prescriptive Analytics.

On Exploring Service Failures by Joint Learning in Rational Databases

A Novel Fuzzy Classification to Enhance Software Regression Testing

Data Quality Aware Hierarchical Federated Reinforcement Learning Framework for Dynamic Treatment Regimes

Analyzing Risk of Service Failures in Heavy Haul Rail Lines: A Hybrid Approach for Imbalanced Data

Data models for service failure prediction in supply-chain networks

Actionable and Interpretable Fault Localization for Recurring Failures in Online Service Systems

Reliability Prediction for Service Oriented System Via Matrix Factorization in a Collaborative Way

Service failure monitoring via multivariate multiple linear regression profile schemes with dimensionality reduction

Inventory Balancing with Online Learning

A Latent Class Regression Approach to IT Maintenance Outsourcing Service Management

SLIM: a Scalable Light-weight Root Cause Analysis for Imbalanced Data in Microservice

Learning Sparse Functional Factors for Large-Scale Service Clustering.

Multi-Focal Learning And Its Application To Customer Service Support

Checking account activity and credit default risk of enterprises: An application of statistical learning methods

Dynamic Adjustment of Disk Redundancy and Scrubbing Strategy with Reinforcement Learning

Data-driven hierarchical learning and real-time decision-making of equipment scheduling and location assignment in automatic high-density storage systems

A Joint Learning Framework for Restaurant Survival Prediction and Explanation.

Robust Failure Diagnosis of Microservice System through Multimodal Data

Log clustering based problem identification for online service systems

Practical and White-Box Anomaly Detection through Unsupervised and Active Learning

Learning in the Model Space for Fault Diagnosis