Abstract:Online experimentation with interference is a common challenge in modern applications such as e-commerce and adaptive clinical trials in medicine. For example, in online marketplaces, the revenue of a good depends on discounts applied to competing goods. Statistical inference with interference is widely studied in the offline setting, but far less is known about how to adaptively assign treatments to minimize regret. We address this gap by studying a multi-armed bandit (MAB) problem where a learner (e-commerce platform) sequentially assigns one of possible $\mathcal{A}$ actions (discounts) to $N$ units (goods) over $T$ rounds to minimize regret (maximize revenue). Unlike traditional MAB problems, the reward of each unit depends on the treatments assigned to other units, i.e., there is interference across the underlying network of units. With $\mathcal{A}$ actions and $N$ units, minimizing regret is combinatorially difficult since the action space grows as $\mathcal{A}^N$. To overcome this issue, we study a sparse network interference model, where the reward of a unit is only affected by the treatments assigned to $s$ neighboring units. We use tools from discrete Fourier analysis to develop a sparse linear representation of the unit-specific reward $r_n: [\mathcal{A}]^N \rightarrow \mathbb{R} $, and propose simple, linear regression-based algorithms to minimize regret. Importantly, our algorithms achieve provably low regret both when the learner observes the interference neighborhood for all units and when it is unknown. This significantly generalizes other works on this topic which impose strict conditions on the strength of interference on a known network, and also compare regret to a markedly weaker optimal action. Empirically, we corroborate our theoretical findings via numerical simulations.

Neural Network-Based Bandit: A Medium Access Control for the IIoT Alarm Scenario

The Bandit Whisperer: Communication Learning for Restless Bandits

Distributed Resource Allocation for URLLC in IIoT Scenarios: A Multi-Armed Bandit Approach

Fair Distributed Cooperative Bandit Learning on Networks for Intelligent Internet of Things Systems (Technical Report)

Risk-Aware Continuous Control with Neural Contextual Bandits

Multi-Armed Bandits with Network Interference

An Intelligent Bayesian Inference Based Learning Automaton Approach for Traffic Management in Radio Access Network

Analyzing Random Access Collisions in Massive IoT Networks.

Distributed NOMA-Based Multi-Armed Bandit Approach for Channel Access in Cognitive Radio Networks

Multi-Armed Bandit-Based User Network Node Selection

LSTM-ACB-Based RA for IoT Mixed Traffic

Two-Stage Resource Allocation in Reconfigurable Intelligent Surface Assisted Hybrid Networks Via Multi-Player Bandits

Edge-Centric Bandit Learning for Task-Offloading Allocations in Multi-RAT Heterogeneous Networks

Faster Activity and Data Detection in Massive Random Access: A Multiarmed Bandit Approach

Traffic Management in IoT Backbone Networks Using GNN and MAB with SDN Orchestration

Multiagent Reinforcement Learning Meets Random Access in Massive Cellular Internet of Things

Sparsity-Aware Intelligent Massive Random Access Control in Open RAN: A Reinforcement Learning Based Approach

Reliability-Optimized User Admission Control for URLLC Traffic: A Neural Contextual Bandit Approach

Improving Portfolio Optimization Results with Bandit Networks

Potential and Pitfalls of Multi-Armed Bandits for Decentralized Spatial Reuse in WLANs

A Gang of Bandits