Abstract:We define a general framework for a large class of combinatorial multi-armed bandit (CMAB) problems, where subsets of base arms with unknown distributions form super arms. In each round, a super arm is played and the base arms contained in the super arm are played and their outcomes are observed. We further consider the extension in which more based arms could be probabilistically triggered based on the outcomes of already triggered arms. The reward of the super arm depends on the outcomes of all played arms, and it only needs to satisfy two mild assumptions, which allow a large class of nonlinear reward instances. We assume the availability of an offline (α,β)-approximation oracle that takes the means of the outcome distributions of arms and outputs a super arm that with probability β generates an α fraction of the optimal expected reward. The objective of an online learning algorithm for CMAB is to minimize (α,β)-approximation regret, which is the difference between the αβ fraction of the expected reward when always playing the optimal super arm, and the expected reward of playing super arms according to the algorithm. We provide CUCB algorithm that achieves O(log n) distribution-dependent regret, where n is the number of rounds played, and we further provide distribution-independent bounds for a large class of reward functions. Our regret analysis is tight in that it matches the bound of UCB1 algorithm (up to a constant factor) for the classical MAB problem, and it significantly improves the regret bound in a earlier paper on combinatorial bandits with linear rewards. We apply our CMAB framework to two new applications, probabilistic maximum coverage and social influence maximization, both having nonlinear reward structures. In particular, application to social influence maximization requires our extension on probabilistically triggered arms.

Decision Making for Multi-Armed Bandit Problem Utilizing Two Parallel Uncorrelated Wideband Complex Chaos

Photonic Decision-Making for Arbitrary-Number-armed Bandit Problem Utilizing Parallel Chaos Generation.

Decentralized Stochastic Multi-Player Multi-Armed Walking Bandits

A Parallel Algorithm for 𝒳-Armed Bandits

A Parallel Algorithm for $\Mathcal{x}$-Armed Bandits

Solving multi-armed bandit problems using a chaotic microresonator comb

Portfolio Choices with Orthogonal Bandit Learning

Networked Bandits With Disjoint Linear Payoffs

Risk-Aware Multi-Armed Bandit Problem with Application to Portfolio Selection

Finite-time Analysis of Globally Nonstationary Multi-Armed Bandits

Understanding the stochastic dynamics of sequential decision-making processes: A path-integral analysis of multi-armed bandits

Top Feasible-Arm Selections in Constrained Multi-Armed Bandit

Auction-Based Combinatorial Multi-Armed Bandit Mechanisms with Strategic Arms

Multi-dueling Bandits with Dependent Arms

Combinatorial Multi-Armed Bandit and Its Extension to Probabilistically Triggered Arms

Multi-Action Restless Bandits with Weakly Coupled Constraints: Simultaneous Learning and Control

Bandits with Concave Aggregated Reward

Combinatorial Multi-Armed Bandit: General Framework and Applications.

Distributed Bandits with Heterogeneous Agents

Cost-Efficient Online Decision Making: A Combinatorial Multi-Armed Bandit Approach

Quantum exploration algorithms for multi-armed bandits