Respecting the limit:Bayesian optimization with a bound on the optimal value

Hanyang Wang,Juergen Branke,Matthias Poloczek
2024-11-07
Abstract:In many real-world optimization problems, we have prior information about what objective function values are achievable. In this paper, we study the scenario that we have either exact knowledge of the minimum value or a, possibly inexact, lower bound on its value. We propose bound-aware Bayesian optimization (BABO), a Bayesian optimization method that uses a new surrogate model and acquisition function to utilize such prior information. We present SlogGP, a new surrogate model that incorporates bound information and adapts the Expected Improvement (EI) acquisition function accordingly. Empirical results on a variety of benchmarks demonstrate the benefit of taking prior information about the optimal value into account, and that the proposed approach significantly outperforms existing techniques. Furthermore, we notice that even in the absence of prior information on the bound, the proposed SlogGP surrogate model still performs better than the standard GP model in most cases, which we explain by its larger expressiveness.
Machine Learning
What problem does this paper attempt to address?
The problem that this paper attempts to solve is that in many practical optimization problems, we usually have prior information about the reachable range of the objective function value, especially the lower bound of the minimum value. However, the existing Bayesian optimization methods usually assume that the objective function is a black box and do not utilize this prior information. Therefore, this paper proposes a new method - Bound - aware Bayesian Optimization (BABO), aiming to improve the optimization efficiency by using the known lower bound of the minimum value. Specifically, the paper focuses on the minimization problem, and it is known that the minimum value \( f^* \) of the objective function has a lower bound \( f_b \), that is, \( f^* \geq f_b \). The author proposes a new surrogate model - Shifted Logarithmic Gaussian Process (SlogGP), and the corresponding acquisition function - Shifted Logarithmic Truncated Expected Improvement (SlogTEI). These new methods can effectively utilize the known lower - bound information, thereby improving the sample efficiency in the optimization process. In addition, even in the absence of prior lower - bound information, the SlogGP surrogate model still performs better than the standard Gaussian Process (GP) model, mainly because SlogGP has greater expressive power. The paper verifies the effectiveness of the proposed method through a series of synthetic functions and two practical application experiments, and the results show that BABO significantly outperforms the existing techniques.