Abstract:Thompson sampling (TS) is one of the most popular and earliest algorithms to solve stochastic multi-armed bandit problems. We consider a variant of TS, named $\alpha$-TS, where we use a fractional or $\alpha$-posterior ($\alpha\in(0,1)$) instead of the standard posterior distribution. To compute an $\alpha$-posterior, the likelihood in the definition of the standard posterior is tempered with a factor $\alpha$. For $\alpha$-TS we obtain both instance-dependent $\mathcal{O}\left(\sum_{k \neq i^*} \Delta_k\left(\frac{\log(T)}{C(\alpha)\Delta_k^2} + \frac{1}{2} \right)\right)$ and instance-independent $\mathcal{O}(\sqrt{KT\log K})$ frequentist regret bounds under very mild conditions on the prior and reward distributions, where $\Delta_k$ is the gap between the true mean rewards of the $k^{th}$ and the best arms, and $C(\alpha)$ is a known constant. Both the sub-Gaussian and exponential family models satisfy our general conditions on the reward distribution. Our conditions on the prior distribution just require its density to be positive, continuous, and bounded. We also establish another instance-dependent regret upper bound that matches (up to constants) to that of improved UCB [Auer and Ortner, 2010]. Our regret analysis carefully combines recent theoretical developments in the non-asymptotic concentration analysis and Bernstein-von Mises type results for the $\alpha$-posterior distribution. Moreover, our analysis does not require additional structural properties such as closed-form posteriors or conjugate priors.

Thompson Sampling under Bernoulli Rewards with Local Differential Privacy

Thompson Sampling Itself is Differentially Private

Thompson Sampling for Budgeted Multi-Armed Bandits

Optimal Rates of (Locally) Differentially Private Heavy-tailed Multi-Armed Bandits

Thompson Sampling Algorithms for Mean-Variance Bandits

Cascading Bandit under Differential Privacy

Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays

On the Prior Sensitivity of Thompson Sampling

Self-accelerated Thompson Sampling with Near-Optimal Regret Upper Bound

Generalized Linear Bandits with Local Differential Privacy

Generalized Regret Analysis of Thompson Sampling using Fractional Posteriors

Robust Thompson Sampling Algorithms Against Reward Poisoning Attacks

A General Recipe for the Analysis of Randomized Multi-Armed Bandit Algorithms

Efficient and Adaptive Posterior Sampling Algorithms for Bandits

A Thompson Sampling Algorithm with Logarithmic Regret for Unimodal Gaussian Bandit.

A Differentially Private Approach for Budgeted Combinatorial Multi-armed Bandits

A Unifying Theory of Thompson Sampling for Continuous Risk-Averse Bandits

Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis

(Locally) Differentially Private Combinatorial Semi-Bandits

Algorithms for Differentially Private Multi-Armed Bandits

Thompson Sampling for Unimodal Bandits