High-Throughput Multi-Objective Bayesian Optimization using Gradients.

Yiming Yao,Fei Liu,Qingfu Zhang

DOI: https://doi.org/10.1109/CEC60901.2024.10612098

2024-01-01

Abstract:When gradient information is available, multi-objective Bayesian optimization (MOBO) algorithm using gradient-enhanced Gaussian process (GradGP) models has been proven to be an efficient method. However, the intractable training and inference complexity hinders its scalability in high-throughput scenarios with large batch sizes and increasing data volume. To address this issue, we develop an efficient high-throughput MOBO algorithm using gradient information named MOEAID-DSVGP, which integrates stochastic variational GP models using directional derivatives (DSVGP) within the decomposition-based MOBO framework. In this algorithm, DSVGP model is trained for each objective function by optimizing a small set of inducing inputs and inducing directions as variational parameters, the complexity is independent of sample size and input dimension. MOEAID is employed as the inner optimizer for high-throughput solution generation. Experimental studies with varying batch sizes and input dimensions show that compared to GradGP-based MOBO method, the proposed algorithm exhibits superior scalability and allows more optimization iterations to yield better outcome in most cases. We further investigate and analyze the influence of increasing the number of inducing directions on the performance improvement in problems with different features.