Abstract:Optimizing an expensive, black-box function <math>f(·)</math> is challenging when its input space is high-dimensional. Sequential design frameworks first model <math>f(·)</math> with a surrogate function and then optimize an acquisition function to determine input settings to evaluate next. Optimization of both <math>f(·)</math> and the acquisition function benefit from effective dimension reduction. Global variable selection detects and removes input variables that do not affect <math>f(·)</math> across the input space. Further dimension reduction may be possible if we consider local variable selection around the current optimum estimate. We develop a sequential design algorithm called sequential optimization in locally important dimensions (SOLID) that incorporates global and local variable selection to optimize a continuous, differentiable function. SOLID performs local variable selection by comparing the surrogate's predictions in a localized region around the estimated optimum with the p alternative predictions made by removing each input variable. The search space of the acquisition function is further restricted to focus only on the variables that are deemed locally active, leading to greater emphasis on refining the surrogate model in locally active dimensions. A simulation study across multiple test functions and an application to the Sarcos robot dataset show that SOLID outperforms conventional approaches. <a class="ext-link" href="https://doi.org/10.1080/00401706.2020.1714738">Supplementary materials</a> for this article are available online.

Sequential Optimization in Locally Important Dimensions

Optimization-Driven Adaptive Experimentation

Sequential model based optimization of partially defined functions under unknown constraints

BINOCULARS for Efficient, Nonmyopic Sequential Experimental Design

A sequential procedure for neighborhood selection-of-the-best in optimization via simulation

Discrete optimization via simulation: algorithms and error control

Learning Multiple Initial Solutions to Optimization Problems

A $p$-step-ahead sequential adaptive algorithm for D-optimal nonlinear regression design

Global Sensitivity Analysis for Optimization with Variable Selection

Logspace Sequential Quadratic Programming for Design Optimization

Fast Rates for Contextual Linear Optimization

Sequential infinite-dimensional Bayesian optimal experimental design with derivative-informed latent attention neural operator

A novel batch-selection strategy for parallel global optimization

Large-Scale Discrete Constrained Black-Box Optimization Using Radial Basis Functions

An Adaptive Algorithm based on High-Dimensional Function Approximation to obtain Optimal Designs

Sensitivity Analysis for Computationally Expensive Models using Optimization and Objective-oriented Surrogate Approximations

Ensemble of surrogate based global optimization methods using hierarchical design space reduction

DADO -- Low-Cost Query Strategies for Deep Active Design Optimization

Best Subset Selection via a Modern Optimization Lens

An Adaptive Sampling Algorithm for Simulation-Based Optimization With Descriptive Complexity Preference

CSOLNP: Numerical Optimization Engine for Solving Non-linearly Constrained Problems