Abstract:This brief studies a variation of the stochastic multiarmed bandit (MAB) problems, where the agent knows the a priori knowledge named the near-optimal mean reward (NoMR). In common MAB problems, an agent tries to find the optimal arm without knowing the optimal mean reward. However, in more practical applications, the agent can usually get an estimation of the optimal mean reward defined as NoMR. For instance, in an online Web advertising system based on MAB methods, a user's near-optimal average click rate (NoMR) can be roughly estimated from his/her demographic characteristics. As a result, application of the NoMR is efficient at improving the algorithm's performance. First, we formalize the stochastic MAB problem by knowing the NoMR that is in between the suboptimal mean reward and the optimal mean reward. Second, we use the cumulative regret as the performance metric for our problem, and we get that this problem's lower bound of the cumulative regret is Omega(1/Delta), where Delta is the difference between the suboptimal mean reward and the optimal mean reward. Compared with the conventional MAB problem with the increasing logarithmic lower bound of the regret, our regret lower bound is uniform with the learning step. Third, a novel algorithm, NOMR-BANDIT, is set forth to solve this problem. In NOMR-BANDIT, the NoMR is used to design an efficient exploration strategy. In addition, we analyzed the regret's upper bound in NOMR-BANDIT and concluded that it also has a uniform upper bound of O(1/Delta), which is in the same order as the lower bound. Consequently, NOMR-BANDIT is an optimal algorithm of this problem. To enhance our method's generalization, CASCADE-BANDIT based on NOMR-BANDIT is proposed to solve the problem, where NoMR is less than the suboptimal mean reward. CASCADE-BANDIT has an upper bound of O(Delta log n), where n represents the learning step, and the order of O(Delta log n) is the same with that of the conventional MAB methods. Finally, extensive experimental results demonstrated that the established NOMR-BANDIT is more efficient than the compared bandit solutions. After sufficient iterations, NOMR-BANDIT saved 10%-80% more cumulative regret than the state of the art.

Thresholding Bandit with Optimal Aggregate Regret

Adaptive Regret for Bandits Made Possible: Two Queries Suffice

Asynchronous Parallel Empirical Variance Guided Algorithms for the Thresholding Bandit Problem

Optimal Algorithms for Lipschitz Bandits with Heavy-tailed Rewards

Optimal Regret Is Achievable with Bounded Approximate Inference Error: An Enhanced Bayesian Upper Confidence Bound Framework

The best of both worlds: stochastic and adversarial bandits

UCB algorithms for multi-armed bandits: Precise regret and adaptive inference

Near-Optimal Regret in Linear MDPs with Aggregate Bandit Feedback

Fast and Regret Optimal Best Arm Identification: Fundamental Limits and Low-Complexity Algorithms

Satisficing Exploration in Bandit Optimization

Optimal Thresholding Linear Bandit

Non-stationary Bandits with Habituation and Recovery Dynamics and Knapsack Constraints

An Optimal Algorithm for the Stochastic Bandits While Knowing the Near-Optimal Mean Reward

Corralling a Larger Band of Bandits: A Case Study on Switching Regret for Linear Bandits

LinearAPT: An Adaptive Algorithm for the Fixed-Budget Thresholding Linear Bandit Problem

Breaking the Moments Condition Barrier: No-Regret Algorithm for Bandits with Super Heavy-Tailed Payoffs

A Simple Approach For Non-Stationary Linear Bandits

Tight Rates for Bandit Control Beyond Quadratics

Nearly Optimal Regret for Stochastic Linear Bandits with Heavy-Tailed Payoffs

The Fragility of Optimized Bandit Algorithms

A PTAS for the Bayesian Thresholding Bandit Problem.