Abstract:Bayesian optimization (BO) has proven to be an effective paradigm for the global optimization of expensive-to-sample systems. One of the main advantages of BO is its use of Gaussian processes (GPs) to characterize model uncertainty which can be leveraged to guide the learning and search process. However, BO typically treats systems as black-boxes and this limits the ability to exploit structural knowledge (e.g., physics and sparse interconnections). Composite functions of the form $f(x, y(x))$, wherein GP modeling is shifted from the performance function $f$ to an intermediate function $y$, offer an avenue for exploiting structural knowledge. However, the use of composite functions in a BO framework is complicated by the need to generate a probability density for $f$ from the Gaussian density of $y$ calculated by the GP (e.g., when $f$ is nonlinear it is not possible to obtain a closed-form expression). Previous work has handled this issue using sampling techniques; these are easy to implement and flexible but are computationally intensive. In this work, we introduce a new paradigm which allows for the efficient use of composite functions in BO; this uses adaptive linearizations of $f$ to obtain closed-form expressions for the statistical moments of the composite function. We show that this simple approach (which we call BOIS) enables the exploitation of structural knowledge, such as that arising in interconnected systems as well as systems that embed multiple GP models and combinations of physics and GP models. Using a chemical process optimization case study, we benchmark the effectiveness of BOIS against standard BO and sampling approaches. Our results indicate that BOIS achieves performance gains and accurately captures the statistics of composite functions.

Optimizing High-Dimensional Physics Simulations via Composite Bayesian Optimization

Bayesian Optimization with High-Dimensional Outputs

High-dimensional Bayesian optimization using low-dimensional feature spaces

Simulation Based Bayesian Optimization

Parameter Optimization using high-dimensional Bayesian Optimization

Surrogate-Based Simulation Optimization

A Bayesian Optimization through Sequential Monte Carlo and Statistical Physics-Inspired Techniques

Joint Composite Latent Space Bayesian Optimization

Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes

Bayesian Optimization over High-Dimensional Combinatorial Spaces via Dictionary-based Embeddings

Bayesian Optimization Algorithms for Accelerator Physics

Deep Gaussian process for multi-objective Bayesian optimization

BOIS: Bayesian Optimization of Interconnected Systems

Bayesian Variational Optimization for Combinatorial Spaces

Bayesian task embedding for few-shot Bayesian optimization

Physics-Aware Multifidelity Bayesian Optimization: a Generalized Formulation

Knowledge-based modeling of simulation behavior for Bayesian optimization

Bayesian Kernelized Tensor Factorization as Surrogate for Bayesian Optimization

Bayesian Optimization using Deep Gaussian Processes

Simultaneous and Meshfree Topology Optimization with Physics-informed Gaussian Processes

Bayesian optimization of atomic structures with prior probabilities from universal interatomic potentials