Abstract:Bayesian optimization (BO) is well known to be sample efficient for solving black-box problems. However, BO algorithms may get stuck in suboptimal solutions even with plenty of samples. Intrinsically, such a suboptimal problem of BO can attribute to the poor surrogate accuracy of the trained Gaussian process (GP), particularly that in the regions where the optimal solutions locate. Hence, we propose to build multiple GP models instead of a single GP surrogate to complement each other, thus resolving the suboptimal problem of BO. Nevertheless, according to the bias-variance tradeoff equation, the individual prediction errors can increase when increasing the diversity of models, which may lead even worse overall surrogate accuracy. On the other hand, based on the theory of the Rademacher complexity, it has been proven that exploiting the agreement of models on unlabeled information can reduce the complexity of hypothesis space, therefore achieving the required surrogate accuracy with fewer samples. Such value of model agreement has been extensively demonstrated for co-training style algorithms to boost model accuracy with a small portion of samples. Inspired by the above, we propose a novel BO algorithm labeled as co-learning BO (CLBO), which exploits both model diversity and agreement on unlabeled information to improve the overall surrogate accuracy with limited samples, therefore achieving more efficient global optimization. Through tests on five numerical toy problems and three engineering benchmarks, the effectiveness of the proposed CLBO has been well demonstrated.

Multiple Adaptive Bayesian Linear Regression for Scalable Bayesian Optimization with Warm Start

High-dimensional Bayesian optimization using low-dimensional feature spaces

Co-Learning Bayesian Optimization.

Large Language Models to Enhance Bayesian Optimization

Deep Kernel Learning-Based Bayesian Optimization with Adaptive Kernel Functions

Simulation Based Bayesian Optimization

Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes

Robust Bayesian Optimization via Localized Online Conformal Prediction

On Provably Robust Meta-Bayesian Optimization

Adaptive Local Bayesian Optimization Over Multiple Discrete Variables

Scalable Bayesian Optimization Using Vecchia Approximations of Gaussian Processes

Bayesian Optimization through Gaussian Cox Process Models for Spatio-temporal Data

Scalable First-Order Bayesian Optimization via Structured Automatic Differentiation

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multimodal Functions

Batch Bayesian Optimization Via Adaptive Local Search.

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multi-Modal Functions

Bayesian Optimization under Stochastic Delayed Feedback

Extrapolative Bayesian Optimization with Gaussian Process and Neural Network Ensemble Surrogate Models

Large-Batch, Iteration-Efficient Neural Bayesian Design Optimization

Bayesian Optimization with High-Dimensional Outputs

Bayesian Optimization Based on K-Optimality