Abstract:Contextual combinatorial cascading bandit ($C^{3}$ -bandit) is a powerful multi-armed bandit framework that balances the tradeoff between exploration and exploitation in the learning process. It well captures users’ click behavior and has been applied in a broad spectrum of real-world applications such as recommender systems and search engines. However, such a framework does not provide a performance guarantee of the initial exploration phase. To that end, we propose conservative contextual combinatorial cascading bandit ($C^{4}$ -bandit) model, aiming to address the aforementioned crucial modeling issues. In this problem, the learning agent is given some contexts and recommends a list of items not worse than the baseline strategy, and then observes the reward by some stopping rule. The objective is now to maximize the reward while simultaneously satisfying the safety constraint, i.e. guaranteeing the algorithm to perform at least as well as a baseline strategy. To tackle this new problem, we extend an online learning algorithm, called Upper Confidence Bound (UCB), to deal with a critical tradeoff between exploitation and exploration and employ the conservative mechanism to properly handle the safety constraints. By carefully integrating these two techniques, we develop a new algorithm, called $C^{4}$ -UCB for this problem. Further, we rigorously prove the n-step upper bound in two situations: known baseline reward and unknown baseline reward. The regret in both situations is only enlarged by an additive constant term compared to results of $C^{3}$ -bandit. Finally, experiments on synthetic and realistic datasets demonstrate its advantages.

contextual: Evaluating Contextual Multi-Armed Bandit Problems in R

Deep Contextual Multi-armed Bandits

Selectively Contextual Bandits

Contextual Bandits with Similarity Information

Contextual Bandits with Random Projection

Contextual Restless Multi-Armed Bandits with Application to Demand Response Decision-Making

AutoML for Contextual Bandits

A Hierarchical Nearest Neighbour Approach to Contextual Bandits

Adapting multi-armed bandits policies to contextual bandits scenarios

Partially Observable Contextual Bandits with Linear Payoffs

Post-Contextual-Bandit Inference

Conservative Contextual Combinatorial Cascading Bandit

A Survey on Practical Applications of Multi-Armed and Contextual Bandits

Conversational Contextual Bandit: Algorithm and Application

Exploring Offline Policy Evaluation for the Continuous-Armed Bandit Problem

Robust Contextual Linear Bandits.

A Survey on Contextual Multi-armed Bandits

A Bayesian Approach for Subset Selection in Contextual Bandits.

Online learning with Corrupted context: Corrupted Contextual Bandits

Towards Domain Adaptive Neural Contextual Bandits

Contextual Multi-armed Bandit Algorithm for Semiparametric Reward Model