Abstract:We consider pure-exploration problems in the context of stochastic sequential adaptive experiments with a finite set of alternative options. The goal of the decision-maker is to accurately answer a query question regarding the alternatives with high confidence with minimal measurement efforts. A typical query question is to identify the alternative with the best performance, leading to ranking and selection problems, or best-arm identification in the machine learning literature. We focus on the fixed-precision setting and derive a sufficient condition for optimality in terms of a notion of strong convergence to the optimal allocation of samples. Using dual variables, we characterize the necessary and sufficient conditions for an allocation to be optimal. The use of dual variables allow us to bypass the combinatorial structure of the optimality conditions that relies solely on primal variables. Remarkably, these optimality conditions enable an extension of top-two algorithm design principle, initially proposed for best-arm identification. Furthermore, our optimality conditions give rise to a straightforward yet efficient selection rule, termed information-directed selection, which adaptively picks from a candidate set based on information gain of the candidates. We outline the broad contexts where our algorithmic approach can be implemented. We establish that, paired with information-directed selection, top-two Thompson sampling is (asymptotically) optimal for Gaussian best-arm identification, solving a glaring open problem in the pure exploration literature. Our algorithm is optimal for $\epsilon$-best-arm identification and thresholding bandit problems. Our analysis also leads to a general principle to guide adaptations of Thompson sampling for pure-exploration problems. Numerical experiments highlight the exceptional efficiency of our proposed algorithms relative to existing ones.

Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification

Optimal Adaptive Experimental Design for Estimating Treatment Effect

Privacy Preserving Adaptive Experiment Design

Adaptive Designs for Sequential Experiments

Regret Bounds and Experimental Design for Estimate-then-Optimize

Minimax-Regret Sample Selection in Randomized Experiments

Optimization-Driven Adaptive Experimentation

DTR Bandit: Learning to Make Response-Adaptive Decisions With Low Regret

Dual-Directed Algorithm Design for Efficient Pure Exploration

Regret Minimization and Statistical Inference in Online Decision Making with High-dimensional Covariates

Approximation to the optimal allocation for response adaptive designs

Simple Bayesian Algorithms for Best Arm Identification

Robust Optimal Design of Two-Armed Trials with Side Information

Adaptive Experimentation with Delayed Binary Feedback

Minimax Optimal Simple Regret in Two-Armed Best-Arm Identification

Adaptive A/B Tests and Simultaneous Treatment Parameter Optimization

Sequential Optimum Test with Multi-armed Bandits for Online Experimentation

Minimizing Dynamic Regret and Adaptive Regret Simultaneously

Adaptive Experimentation at Scale: A Computational Framework for Flexible Batches

Experimenting on Markov Decision Processes with Local Treatments

Adaptive Experiments Toward Learning Treatment Effect Heterogeneity