Abstract:Bayesian optimization (BO) has proven to be an effective paradigm for the global optimization of expensive-to-sample systems. One of the main advantages of BO is its use of Gaussian processes (GPs) to characterize model uncertainty which can be leveraged to guide the learning and search process. However, BO typically treats systems as black-boxes and this limits the ability to exploit structural knowledge (e.g., physics and sparse interconnections). Composite functions of the form $f(x, y(x))$, wherein GP modeling is shifted from the performance function $f$ to an intermediate function $y$, offer an avenue for exploiting structural knowledge. However, the use of composite functions in a BO framework is complicated by the need to generate a probability density for $f$ from the Gaussian density of $y$ calculated by the GP (e.g., when $f$ is nonlinear it is not possible to obtain a closed-form expression). Previous work has handled this issue using sampling techniques; these are easy to implement and flexible but are computationally intensive. In this work, we introduce a new paradigm which allows for the efficient use of composite functions in BO; this uses adaptive linearizations of $f$ to obtain closed-form expressions for the statistical moments of the composite function. We show that this simple approach (which we call BOIS) enables the exploitation of structural knowledge, such as that arising in interconnected systems as well as systems that embed multiple GP models and combinations of physics and GP models. Using a chemical process optimization case study, we benchmark the effectiveness of BOIS against standard BO and sampling approaches. Our results indicate that BOIS achieves performance gains and accurately captures the statistics of composite functions.

Federated Bayesian Optimization Via Compressed Sensing

Federated Many-Task Bayesian Optimization

Differentially Private Federated Bayesian Optimization with Distributed Exploration

Collaborative and Federated Black-box Optimization: A Bayesian Optimization Perspective

SimFBO: Towards Simple, Flexible and Communication-efficient Federated Bilevel Learning

Privacy-preserving Federated Bayesian Optimization with Learnable Noise

Private Outsourced Bayesian Optimization

BOIS: Bayesian Optimization of Interconnected Systems

A Forward-Backward Bregman Splitting Scheme for Regularized Distributed Optimization Problems

Optimized Compressed Sensing for Communication Efficient Federated Learning.

High-dimensional Bayesian optimization using low-dimensional feature spaces

Distilling Ensemble Surrogates for Federated Data-Driven Many-Task Optimization

Multi-Fidelity Cost-Aware Bayesian Optimization

Communication-Efficient Federated Non-Linear Bandit Optimization

Federated Bayesian Optimization for Privacy-Preserving Neural Architecture Search

How Bayesian Should Bayesian Optimisation Be?

Joint Composite Latent Space Bayesian Optimization

A Field Guide to Federated Optimization

Constrained Bayesian Optimization Under Partial Observations: Balanced Improvements and Provable Convergence

Robust Bayesian Optimization via Localized Online Conformal Prediction

A Bayesian Optimization Framework for Finding Local Optima in Expensive Multimodal Functions