Abstract:This paper considers the distributed online bandit optimization problem with nonconvex loss functions over a time-varying digraph. This problem can be viewed as a repeated game between a group of online players and an adversary. At each round, each player selects a decision from the constraint set, and then the adversary assigns an arbitrary, possibly nonconvex, loss function to this player. Only the loss value at the current round, rather than the entire loss function or any other information (e.g. gradient), is privately revealed to the player. Players aim to minimize a sequence of global loss functions, which are the sum of local losses. We observe that traditional multi-point bandit algorithms are unsuitable for online optimization, where the data for the loss function are not all a priori, while the one-point bandit algorithms suffer from poor regret guarantees. To address these issues, we propose a novel one-point residual feedback distributed online algorithm. This algorithm estimates the gradient using residuals from two points, effectively reducing the regret bound while maintaining $\mathcal{O}(1)$ sampling complexity per iteration. We employ a rigorous metric, dynamic regret, to evaluate the algorithm's performance. By appropriately selecting the step size and smoothing parameters, we demonstrate that the expected dynamic regret of our algorithm is comparable to existing algorithms that use two-point feedback, provided the deviation in the objective function sequence and the path length of the minimization grows sublinearly. Finally, we validate the effectiveness of the proposed algorithm through numerical simulations.

A Second-Order Method for Stochastic Bandit Convex Optimisation

Online Newton Method for Bandit Convex Optimisation

Risk-Averse Stochastic Convex Bandit

Second Order Methods for Bandit Optimization and Control

Tight Rates for Bandit Control Beyond Quadratics

Nearly Optimal Regret for Stochastic Linear Bandits with Heavy-Tailed Payoffs

Regret Analysis for Continuous Dueling Bandit

Bandit Convex Optimisation

Nash Regret Guarantees for Linear Bandits

Distributed Online Stochastic-Constrained Convex Optimization With Bandit Feedback

Projection-Free Bandit Convex Optimization over Strongly Convex Sets

Improved Regret for Bandit Convex Optimization with Delayed Feedback

A Simple Approach For Non-Stationary Linear Bandits

Linear bandits with polylogarithmic minimax regret

Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits.

Bandit Convex Optimization in Non-stationary Environments.

Improved Algorithms for Stochastic Linear Bandits Using Tail Bounds for Martingale Mixtures

Minimax Optimal Submodular Optimization with Bandit Feedback

Improved Regret Bounds of (Multinomial) Logistic Bandits via Regret-to-Confidence-Set Conversion

Adaptive Regret for Bandits Made Possible: Two Queries Suffice

Distributed Online Bandit Nonconvex Optimization with One-Point Residual Feedback via Dynamic Regret