Abstract:Bayesian Optimization (BO) is a popular approach to optimizing expensive-to-evaluate black-box functions. Despite the success of BO, its performance may decrease exponentially as the dimensionality increases. A common framework to tackle this problem is to assume that the objective function depends on a limited set of features that lie on a low-dimensional manifold embedded in the high-dimensional ambient space. The latent space can be linear or more generally nonlinear. To learn feature mapping, existing works usually use an encode-decoder framework which is either computationally expensive or susceptible to overfittting when the labeled data is limited. This paper proposes a new approach for BO in high dimensions by exploiting a new representation of the objective function. Our approach combines a random linear projection to reduce the dimensionality, with a representation learning of the nonlinear manifold. When the geometry of the latent manifold is available, a solution to exploit this geometry is proposed for representation learning. In contrast, we use a neural network. To mitigate overfitting by using the neural network, we train the feature mapping in a geometry-aware semi-supervised manner. Our approach enables efficient optimizing of BO's acquisition function in the low-dimensional space, with the advantage of projecting back to the original high-dimensional space compared to existing works in the same setting. Finally, we show empirically that our algorithm outperforms other high-dimensional BO baselines in various synthetic functions and real applications.

Bayesian Optimization in a Billion Dimensions via Random Embeddings

High-dimensional Bayesian optimization using low-dimensional feature spaces

High-Dimensional Bayesian Optimization Using Both Random and Supervised Embeddings

High dimensional Bayesian Optimization via Condensing-Expansion Projection

High Dimensional Bayesian Optimization Via Supervised Dimension Reduction

High-Dimensional Bayesian Optimization via Random Projection of Manifold Subspaces

An Adaptive Dimension Reduction Estimation Method for High-dimensional Bayesian Optimization

Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings

BOIDS: High-dimensional Bayesian Optimization via Incumbent-guided Direction Lines and Subspace Embeddings

Increasing the Scope as You Learn: Adaptive Bayesian Optimization in Nested Subspaces

Are Random Decompositions all we need in High Dimensional Bayesian Optimisation?

Embedded Bandits for Large-Scale Black-Box Optimization

EMT-ReMO: Evolutionary Multitasking for High-Dimensional Multi-Objective Optimization Via Random Embedding

Derivative-Free Optimization of High-Dimensional Non-Convex Functions by Sequential Random Embeddings

Vanilla Bayesian Optimization Performs Great in High Dimensions

SCORE: A 1D Reparameterization Technique to Break Bayesian Optimization's Curse of Dimensionality

Large -Scale Optimization Via Evolutionary Multitasking Assisted Random Embedding

Parameter Optimization using high-dimensional Bayesian Optimization

Semi-supervised Embedding Learning for High-dimensional Bayesian Optimization

Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes

High Dimensional Bayesian Optimization Assisted by Principal Component Analysis