Abstract:We consider the framework of methods for unconstrained minimization that are, in each iteration, restricted to a model that is only a valid approximation to the objective function on some affine subspace containing an incumbent point. These methods are of practical interest in computational settings where derivative information is either expensive or impossible to obtain. Recent attention has been paid in the literature to employing randomized matrix sketching for generating the affine subspaces within this framework. We consider a relatively straightforward, deterministic augmentation of such a generic subspace optimization method. In particular, we consider a sequential optimization framework where actions consist of one-dimensional linear subspaces and rewards consist of (approximations to) the magnitudes of directional derivatives computed in the direction of the action subspace. Reward maximization in this context is consistent with maximizing lower bounds on descent guaranteed by first-order Taylor models. This sequential optimization problem can be analyzed through the lens of dynamic regret. We modify an existing linear upper confidence bound (UCB) bandit method and prove sublinear dynamic regret in the subspace optimization setting. We demonstrate the efficacy of employing this linear UCB method in a setting where forward-mode algorithmic differentiation can provide directional derivatives in arbitrary directions and in a derivative-free setting. For the derivative-free setting, we propose SS-POUNDers, an extension of the derivative-free optimization method POUNDers that employs the linear UCB mechanism to identify promising subspaces. Our numerical experiments suggest a preference, in either computational setting, for employing a linear UCB mechanism within a subspace optimization method.

Augmenting Subspace Optimization Methods with Linear Bandits

Linear Submodular Maximization with Bandit Feedback

Per-Round Knapsack-Constrained Linear Submodular Bandits

Linear Submodular Bandits With A Knapsack Constraint

Non-stationary Bandits with Habituation and Recovery Dynamics and Knapsack Constraints

Bandits with Preference Feedback: A Stackelberg Game Perspective

Geometry-Aware Approaches for Balancing Performance and Theoretical Guarantees in Linear Bandits

Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces

Low-Rank Bandits via Tight Two-to-Infinity Singular Subspace Recovery

Adaptive and Oblivious Randomized Subspace Methods for High-Dimensional Optimization: Sharp Analysis and Lower Bounds

Noise-Adaptive Confidence Sets for Linear Bandits and Application to Bayesian Optimization

Variance-Aware Sparse Linear Bandits.

Subspace Methods for Nonlinear Optimization

Bandits with Mean Bounds

Learning to Optimize via Posterior Sampling

Old Dog Learns New Tricks: Randomized UCB for Bandit Problems

BOF-UCB: A Bayesian-Optimistic Frequentist Algorithm for Non-Stationary Contextual Bandits

Online SuBmodular + SuPermodular (BP) Maximization with Bandit Feedback

Leveraging Offline Data in Linear Latent Bandits

Indexed Minimum Empirical Divergence-Based Algorithms for Linear Bandits

Nearly Minimax-Optimal Regret for Linearly Parameterized Bandits.