Abstract:The multi-armed bandit (MAB) models have always received lots of attention from multiple research communities due to their broad application domains. The optimal selection problem with unknown rewards in advance, such as ad recommendation in social networks, spectrum access in the cognitive radio field, etc., can be efficiently solved by using MAB models. In an MAB model, given $N$ arms whose rewards are unknown in advance, the player selects exactly one arm in each round, and his goal is to maximize the cumulative rewards over a fixed horizon. Further, a more general model called combinatorial MAB (i.e., CMAB), where $K$ arms can be played simultaneously in each round, is put forward. However, the existing CMAB models neglect the strategic behaviors of the $N$ arms, which indicates that one arm might report false information to increase its own profits. In fact, in many applications such as user selection in crowdsensing, the arms are not the feelingless machines but the rational individuals. To this end, we combine the upper confidence bound (UCB) with auction theory to develop a new algorithm called auction-based UCB (AUCB). We divide the auction-based CMAB problem into two sub-problems: winning arm selection and payment computation problems. For AUCB, we derive an upper bound on regret and prove the truthfulness in one round, individual rationality, and computational efficiency. In addition, we consider an extended situation that some arms may be unavailable in some rounds and the arms will bid inconsistently in different rounds. We devise another algorithm called eAUCB to solve this problem. Extensive simulations are conducted to show the significant performance of the proposed algorithms.

Photonic Decision-Making for Arbitrary-Number-armed Bandit Problem Utilizing Parallel Chaos Generation.

Decision Making for Multi-Armed Bandit Problem Utilizing Two Parallel Uncorrelated Wideband Complex Chaos

Solving multi-armed bandit problems using a chaotic microresonator comb

Conflict-free collective stochastic decision making by orbital angular momentum of photons through quantum interference

Scalable photonic reinforcement learning by time-division multiplexing of laser chaos

Harnessing microcomb-based parallel chaos for random number generation and optical decision making

A Parallel Algorithm for 𝒳-Armed Bandits

A Parallel Algorithm for $\Mathcal{x}$-Armed Bandits

Decentralized Stochastic Multi-Player Multi-Armed Walking Bandits

Arm order recognition in multi-armed bandit problem with laser chaos time series

Understanding the stochastic dynamics of sequential decision-making processes: A path-integral analysis of multi-armed bandits

Exploring Best Arm With Top Reward-Cost Ratio In Stochastic Bandits

Auction-Based Combinatorial Multi-Armed Bandit Mechanisms with Strategic Arms

Blocking Bandits

Combination of Auction Theory and Multi-Armed Bandits: Model, Algorithm, and Application

Restless Bandit Problem with Rewards Generated by a Linear Gaussian Dynamical System

Covert Online Decision Making: From Sequential Hypothesis Testing to Stochastic Bandits

Parallelizing Exploration-Exploitation Tradeoffs with Gaussian Process Bandit Optimization

Networked Bandits With Disjoint Linear Payoffs

Dynamic channel selection in wireless communications via a multi-armed bandit algorithm using laser chaos time series

Old Dog Learns New Tricks: Randomized UCB for Bandit Problems