Abstract:We study high-dimensional multi-armed contextual bandits with batched feedback where the $T$ steps of online interactions are divided into $L$ batches. In specific, each batch collects data according to a policy that depends on previous batches and the rewards are revealed only at the end of the batch. Such a feedback structure is popular in applications such as personalized medicine and online advertisement, where the online data often do not arrive in a fully serial manner. We consider high-dimensional and linear settings where the reward function of the bandit model admits either a sparse or low-rank structure and ask how small a number of batches are needed for a comparable performance with fully dynamic data in which $L = T$. For these settings, we design a provably sample-efficient algorithm which achieves a $ \mathcal{\tilde O}(s_0^2 \log^2 T)$ regret in the sparse case and $ \mathcal{\tilde O} ( r ^2 \log^2 T)$ regret in the low-rank case, using only $L = \mathcal{O}( \log T)$ batches. Here $s_0$ and $r$ are the sparsity and rank of the reward parameter in sparse and low-rank cases, respectively, and $ \mathcal{\tilde O}(\cdot)$ omits logarithmic factors involving the feature dimensions. In other words, our algorithm achieves regret bounds comparable to those in fully sequential setting with only $\mathcal{O}( \log T)$ batches. Our algorithm features a novel batch allocation method that adjusts the batch sizes according to the estimation accuracy within each batch and cumulative regret. Furthermore, we also conduct experiments with synthetic and real-world data to validate our theory.

Differentially Private High Dimensional Bandits

FLIPHAT: Joint Differential Privacy for High Dimensional Sparse Linear Bandits

Generalized Linear Bandits with Local Differential Privacy

(Locally) Differentially Private Combinatorial Semi-Bandits

Locally Differentially Private (Contextual) Bandits Learning

Differentially Private Variational Inference for Non-conjugate Models

Concentrated Differential Privacy for Bandits

Algorithms for Differentially Private Multi-Armed Bandits

Near-Optimal Algorithms for Differentially Private Online Learning in a Stochastic Environment

Optimal Rates of (Locally) Differentially Private Heavy-tailed Multi-Armed Bandits

Private Estimation and Inference in High-Dimensional Regression with FDR Control

Privacy-Preserving Multi-Party Contextual Bandits

Low-Rank Bandits via Tight Two-to-Infinity Singular Subspace Recovery

An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm

Provably Efficient High-Dimensional Bandit Learning with Batched Feedbacks

Distributed Differential Privacy in Multi-Armed Bandits

On the Optimal Regret of Locally Private Linear Contextual Bandit

Differentially Private Sliced Inverse Regression: Minimax Optimality and Algorithm

Differentially Private Learning Beyond the Classical Dimensionality Regime

High-dimensional Contextual Bandit Problem without Sparsity

On the Computational Complexity of Private High-dimensional Model Selection