Abstract:Bayesian optimization (BO) and its batch extensions are successful for optimizing expensive black-box functions. However, these traditional BO approaches are not yet ideal for optimizing less expensive functions when the computational cost of BO can dominate the cost of evaluating the blackbox function. Examples of these less expensive functions are cheap machine learning models, inexpensive physical experiment through simulators, and acquisition function optimization in Bayesian optimization. In this paper, we consider a batch BO setting for situations where function evaluations are less expensive. Our model is based on a new exploration strategy using geometric distance that provides an alternative way for exploration, selecting a point far from the observed locations. Using that intuition, we propose to use Sobol sequence to guide exploration that will get rid of running multiple global optimization steps as used in previous works. Based on the proposed distance exploration, we present an efficient batch BO approach. We demonstrate that our approach outperforms other baselines and global optimization methods when the function evaluations are less expensive.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to improve the efficiency and performance of Batch Bayesian Optimization (BO) when the cost of function evaluation is low**.
Specifically, traditional Bayesian optimization methods and their batch - processing extensions are mainly used to optimize expensive black - box functions, where the cost of function evaluation is much higher than the computational cost. However, in many practical applications, the cost of function evaluation is not high (for example, simple machine - learning models, low - cost physical experiment simulations, etc.). At this time, the computational cost of batch - processing Bayesian optimization may exceed the cost of function evaluation, resulting in a decline in optimization efficiency. Therefore, the paper proposes a new batch - processing Bayesian optimization method, aiming to optimize these functions with low evaluation costs and ensure that the computational cost does not become a bottleneck.
### Main contributions of the paper:
1. **For the first time, studied the application of Bayesian optimization in functions with low evaluation costs**.
2. **Proposed a batch - processing Bayesian optimization method based on Distance Exploration (DE)**, which selects new points far from existing observation points through geometric distance, thereby improving the exploration efficiency.
3. **Verified the effectiveness of this method in benchmark tests and practical applications**, especially in the case of low evaluation costs, showing performance superior to other baseline methods.
### Method overview:
- **UCB - DE algorithm**: First, use the standard Bayesian optimization method (such as GP - UCB) to select the first point in the batch, and then use the Sobol sequence to generate the remaining batch points. The Sobol sequence is a low - discrepancy sequence that can evenly distribute points in space, thereby avoiding repeated sampling of known areas.
- **Distance Exploration (DE) strategy**: Select the remaining points in the batch by maximizing the minimum distance between the new points and the existing observation points, ensuring that the new points are as far away from the existing observation points as possible, thereby improving the exploration efficiency.
- **Computational complexity advantage**: Compared with other batch - processing Bayesian optimization methods, UCB - DE only needs to perform one global optimization, greatly reducing the computational cost.
### Experimental results:
The paper verified the effectiveness of UCB - DE through multiple benchmark functions and practical applications (such as physical heat - treatment design, machine - learning model optimization, etc.). Especially in the case of low evaluation costs, UCB - DE can find better solutions in a shorter time and has higher computational efficiency.
In summary, this paper proposes an efficient batch - processing Bayesian optimization method for functions with low evaluation costs, solving the computational bottleneck problem of traditional methods in this scenario.