Mike Diessner,Kevin J. Wilson,Richard D. Whalley
Abstract:NUBO, short for Newcastle University Bayesian Optimization, is a Bayesian optimization framework for optimizing expensive-to-evaluate black-box functions, such as physical experiments and computer simulators. Bayesian optimization is a cost-efficient optimization strategy that uses surrogate modeling via Gaussian processes to represent an objective function and acquisition functions to guide the selection of candidate points to approximate the global optimum of the objective function. NUBO focuses on transparency and user experience to make Bayesian optimization accessible to researchers from all disciplines. Clean and understandable code, precise references, and thorough documentation ensure transparency, while a modular and flexible design, easy-to-write syntax, and careful selection of Bayesian optimization algorithms ensure a good user experience. NUBO allows users to tailor Bayesian optimization to their problem by writing a custom optimization loop using the provided building blocks. It supports sequential single-point, parallel multi-point, and asynchronous optimization of bounded, constrained, and mixed (discrete and continuous) parameter input spaces. Only algorithms and methods extensively tested and validated to perform well are included in NUBO. This ensures that the package remains compact and does not overwhelm the user with an unnecessarily large number of options. The package is written in Python but does not require expert knowledge of Python to optimize simulators and experiments. NUBO is distributed as open-source software under the BSD 3-Clause license.
What problem does this paper attempt to address?
### What problems does this paper attempt to solve?
This paper aims to solve the problem of optimizing expensive black - box functions. Such functions usually have the following characteristics:
1. **Unknown or unanalyzable mathematical expressions**: They cannot be solved by traditional analytical methods.
2. **High evaluation cost**: Each evaluation of these functions requires a large amount of resources, such as material costs, computing resources or time.
To solve these problems, the paper proposes a Python package named NUBO (Newcastle University Bayesian Optimization). NUBO is based on Bayesian Optimization (BO), which is an effective strategy for finding the global optimal solution with the least number of function evaluations. Specifically, NUBO solves the following problems:
- **Optimizing expensive black - box functions**: For example, the objective functions in physical experiments and computer simulations, whose evaluation costs are very high.
- **Providing transparent and easy - to - use tools**: Ensure that researchers in different disciplines can understand and use Bayesian optimization techniques without having a deep background in statistics or computer science.
- **Supporting parallel and asynchronous optimization**: In addition to the traditional single - point sequential optimization, it also supports multi - point parallel optimization and asynchronous optimization to accelerate the optimization process.
- **Simplifying implementation**: Through concise code and documentation, ensure that users can easily get started and avoid being troubled by complex algorithm details.
### Basic principles of Bayesian optimization
Bayesian optimization approximates the objective function by constructing a surrogate model and uses the acquisition function to guide the selection of candidate points. Its core steps include:
1. **Initializing training data**: Sampling initial points through space - filling design and collecting observations.
2. **Fitting the surrogate model**: Using the existing training data to fit a Gaussian Process (GP) as a surrogate model.
3. **Maximizing the acquisition function**: According to the current surrogate model, select the next most promising candidate point for evaluation.
4. **Updating training data**: Add the newly evaluated point to the training data and repeat the above steps until the termination condition is met.
### Main features of NUBO
- **Transparency**: Through clear and understandable code, accurate references and detailed documentation, ensure that users can understand the entire optimization process.
- **User experience**: Modular and flexible design, simple syntax, and carefully selected Bayesian optimization algorithms ensure that users can easily customize the optimization process.
- **Efficiency**: Although the amount of code is small (only 1,322 lines), its performance is comparable to more complex packages, and even better than them in some cases.
- **Wide applicability**: It is suitable for optimizing expensive black - box functions in various discipline areas, such as engineering, computing, natural sciences, etc.
In short, NUBO helps researchers find the optimal solution to complex problems under limited resources by providing a simple, transparent and efficient Bayesian optimization tool.