Abstract:This paper introduces a federated learning framework tailored for online combinatorial optimization with bandit feedback. In this setting, agents select subsets of arms, observe noisy rewards for these subsets without accessing individual arm information, and can cooperate and share information at specific intervals. Our framework transforms any offline resilient single-agent $(\alpha-\epsilon)$-approximation algorithm, having a complexity of $\tilde{\mathcal{O}}(\frac{\psi}{\epsilon^\beta})$, where the logarithm is omitted, for some function $\psi$ and constant $\beta$, into an online multi-agent algorithm with $m$ communicating agents and an $\alpha$-regret of no more than $\tilde{\mathcal{O}}(m^{-\frac{1}{3+\beta}} \psi^\frac{1}{3+\beta} T^\frac{2+\beta}{3+\beta})$. This approach not only eliminates the $\epsilon$ approximation error but also ensures sublinear growth with respect to the time horizon $T$ and demonstrates a linear speedup with an increasing number of communicating agents. Additionally, the algorithm is notably communication-efficient, requiring only a sublinear number of communication rounds, quantified as $\tilde{\mathcal{O}}\left(\psi T^\frac{\beta}{\beta+1}\right)$. Furthermore, the framework has been successfully applied to online stochastic submodular maximization using various offline algorithms, yielding the first results for both single-agent and multi-agent settings and recovering specialized single-agent theoretical guarantees. We empirically validate our approach to a stochastic data summarization problem, illustrating the effectiveness of the proposed framework, even in single-agent scenarios.

Cooperative Stochastic Bandits with Asynchronous Agents and Constrained Feedback

Distributed Bandits with Heterogeneous Agents

Cooperative Multi-agent Bandits: Distributed Algorithms with Optimal Individual Regret and Constant Communication Costs

Collaborative Multi-agent Stochastic Linear Bandits

Cooperative Multi-Agent Constrained Stochastic Linear Bandits

Multi-agent Multi-armed Bandits with Stochastic Sharable Arm Capacities

Cooperative Stochastic Multi-agent Multi-armed Bandits Robust to Adversarial Corruptions

Multi-Agent Stochastic Bandits Robust to Adversarial Corruptions

Distributed Stochastic Bandit Learning with Delayed Context Observation

Doubly Adversarial Federated Bandits

On Regret-optimal Cooperative Nonstochastic Multi-armed Bandits

Federated Combinatorial Multi-Agent Multi-Armed Bandits

Online Stochastic Optimization under Correlated Bandit Feedback

Multi-Agent Best Arm Identification in Stochastic Linear Bandits

Decentralized Stochastic Multi-Player Multi-Armed Walking Bandits

Best arm identification in multi-armed bandits with delayed feedback

Adaptive Algorithms for Multi-armed Bandit with Composite and Anonymous Feedback

Active clustering with bandit feedback

Byzantine-Resilient Decentralized Multi-Armed Bandits

Optimal Cooperative Multiplayer Learning Bandits with Noisy Rewards and No Communication

Networked Bandits With Disjoint Linear Payoffs