Abstract:In this paper, we study Federated Bandit, a decentralized Multi-Armed Bandit problem with a set of N agents, who can only communicate their local data with neighbors described by a connected graph G. Each agent makes a sequence of decisions on selecting an arm from M candidates, yet they only have access to local and potentially biased feedback/evaluation of the true reward for each action taken. Learning only locally will lead agents to sub-optimal actions while converging to a no-regret strategy requires a collection of distributed data. Motivated by the proposal of federated learning, we aim for a solution with which agents will never share their local observations with a central entity, and will be allowed to only share a private copy of his/her own information with their neighbors. We first propose a decentralized bandit algorithm \textttGossip\_UCB, which is a coupling of variants of both the classical gossiping algorithm and the celebrated Upper Confidence Bound (UCB) bandit algorithm. We show that \textttGossip\_UCB successfully adapts local bandit learning into a global gossiping process for sharing information among connected agents, and achieves guaranteed regret at the order of O(\max\ \textttpoly (N,M) łog T, \textttpoly (N,M)łog_łambda_2^-1 N\ ) for all N agents, where łambda_2\in(0,1) is the second largest eigenvalue of the expected gossip matrix, which is a function of G. We then propose \textttFed\_UCB, a differentially private version of \textttGossip\_UCB, in which the agents preserve ε-differential privacy of their local data while achieving O(\max \\frac\textttpoly (N,M) ε łog^2.5 T, \textttpoly (N,M) (łog_łambda_2^-1 N + łog T) \ ) regret.

Distributed Bandits with Heterogeneous Agents

Distributed Multiarmed Bandits

A Distributed Algorithm for Sequential Decision Making in Multi-Armed Bandit with Homogeneous Rewards

A Distributed Algorithm for Multi-Armed Bandit with Homogeneous Rewards over Directed Graphs

Cooperative Multi-agent Bandits: Distributed Algorithms with Optimal Individual Regret and Constant Communication Costs

Cooperative Stochastic Bandits with Asynchronous Agents and Constrained Feedback

Distributed Optimization via Kernelized Multi-armed Bandits

Combining Diverse Information for Coordinated Action: Stochastic Bandit Algorithms for Heterogeneous Agents

Multi-agent Multi-armed Bandits with Stochastic Sharable Arm Capacities

Decentralized Upper Confidence Bound Algorithms for Homogeneous Multi-Agent Multi-Armed Bandits

Distributed Multi-Armed Bandit over Arbitrary Undirected Graphs.

Heterogeneous Stochastic Interactions for Multiple Agents in a Multi-armed Bandit Problem

Fair Multi-Agent Bandits

Federated Bandit

Cooperative Multi-Agent Graph Bandits: UCB Algorithm and Regret Analysis

Federated Combinatorial Multi-Agent Multi-Armed Bandits

Decentralized Stochastic Multi-Player Multi-Armed Walking Bandits

Collaborative Multi-agent Stochastic Linear Bandits

Distributed Robust Bandits with Efficient Communication.

Networked Bandits With Disjoint Linear Payoffs

You Can Trade Your Experience in Distributed Multi-Agent Multi-Armed Bandits.