Abstract:Thompson sampling (TS) is one of the most popular and earliest algorithms to solve stochastic multi-armed bandit problems. We consider a variant of TS, named $\alpha$-TS, where we use a fractional or $\alpha$-posterior ($\alpha\in(0,1)$) instead of the standard posterior distribution. To compute an $\alpha$-posterior, the likelihood in the definition of the standard posterior is tempered with a factor $\alpha$. For $\alpha$-TS we obtain both instance-dependent $\mathcal{O}\left(\sum_{k \neq i^*} \Delta_k\left(\frac{\log(T)}{C(\alpha)\Delta_k^2} + \frac{1}{2} \right)\right)$ and instance-independent $\mathcal{O}(\sqrt{KT\log K})$ frequentist regret bounds under very mild conditions on the prior and reward distributions, where $\Delta_k$ is the gap between the true mean rewards of the $k^{th}$ and the best arms, and $C(\alpha)$ is a known constant. Both the sub-Gaussian and exponential family models satisfy our general conditions on the reward distribution. Our conditions on the prior distribution just require its density to be positive, continuous, and bounded. We also establish another instance-dependent regret upper bound that matches (up to constants) to that of improved UCB [Auer and Ortner, 2010]. Our regret analysis carefully combines recent theoretical developments in the non-asymptotic concentration analysis and Bernstein-von Mises type results for the $\alpha$-posterior distribution. Moreover, our analysis does not require additional structural properties such as closed-form posteriors or conjugate priors.

A Unifying Theory of Thompson Sampling for Continuous Risk-Averse Bandits

Thompson Sampling Algorithms for Mean-Variance Bandits

Optimal Regret Analysis of Thompson Sampling in Stochastic Multi-armed Bandit Problem with Multiple Plays

Thompson Sampling for Unimodal Bandits

Efficient and Adaptive Posterior Sampling Algorithms for Bandits

Adaptive Portfolio by Solving Multi-armed Bandit Via Thompson Sampling

Thompson Sampling for Budgeted Multi-Armed Bandits

A Note on Information-Directed Sampling and Thompson Sampling

Generalized Regret Analysis of Thompson Sampling using Fractional Posteriors

On the Prior Sensitivity of Thompson Sampling

A General Recipe for the Analysis of Randomized Multi-Armed Bandit Algorithms

Self-accelerated Thompson Sampling with Near-Optimal Regret Upper Bound

A Thompson Sampling Algorithm with Logarithmic Regret for Unimodal Gaussian Bandit.

Thompson Sampling for Bandit Learning in Matching Markets

Risk-averse Contextual Multi-armed Bandit Problem with Linear Payoffs

Optimality of Thompson Sampling with Noninformative Priors for Pareto Bandits

Feel-good thompson sampling for contextual bandits and reinforcement learning

Thompson Sampling For Combinatorial Bandits: Polynomial Regret and Mismatched Sampling Paradox

Safe Linear Thompson Sampling with Side Information

Efficient Inference Without Trading-off Regret in Bandits: An Allocation Probability Test for Thompson Sampling