Bayesian Optimization that Limits Search Region to Lower Dimensions Utilizing Local GPR

Yasunori Taguchi,Hiro Gangi

2024-03-13

Abstract:Optimization of product and system characteristics is required in many fields, including design and control. Bayesian optimization (BO) is often used when there are high observing costs, because BO theoretically guarantees an upper bound on regret. However, computational costs increase exponentially with the number of parameters to be optimized, decreasing search efficiency. We propose a BO that limits the search region to lower dimensions and utilizes local Gaussian process regression (LGPR) to scale the BO to higher dimensions. LGPR treats the low-dimensional search region as "local," improving prediction accuracies there. The LGPR model is trained on a local subset of data specific to that region. This improves prediction accuracy and search efficiency and reduces the time complexity of matrix inversion in the Gaussian process regression. In evaluations with 20D Ackley and Rosenbrock functions, search efficiencies are equal to or higher than those of the compared methods, improved by about 69% and 40% from the case without LGPR. We apply our method to an automatic design task for a power semiconductor device. We successfully reduce the specific on-resistance to 25% better than a conventional method and 3.4% better than without LGPR.

Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: when performing Bayesian Optimization (BO) in a high - dimensional parameter space, the computational cost increases exponentially with the number of parameters, resulting in a decline in search efficiency. Specifically, traditional Bayesian optimization methods face the following challenges when dealing with high - dimensional problems:

1. **High computational complexity**: As the parameter dimension increases, the time complexity of matrix inversion and determinant calculation increases from \( \mathcal{O}(N^3) \) to a non - negligible level.
2. **Low search efficiency**: Global search in high - dimensional space leads to sparse distribution of sampling points, which reduces the prediction accuracy of surrogate models (such as Gaussian Process Regression, GPR).

To solve these problems, the author proposes a new Bayesian optimization method - BOLDUC (Bayesian Optimization that Limits Dimensions Utilizing Local Gaussian Process Regression). This method improves the search efficiency and computational efficiency of high - dimensional optimization problems by restricting the search area to a low - dimensional subspace and using Local Gaussian Process Regression (LGPR). Specific improvements include:

- **Low - dimensional search space**: By defining a low - dimensional affine subspace as the search area, the dimension of the search space is reduced, thereby simplifying the optimization problem.
- **Local Gaussian Process Regression**: Use a Local Subset of Data (LSoD) within the low - dimensional search area to train the LGPR model to improve prediction accuracy and reduce computational complexity.

Through these improvements, BOLDUC can maintain high search efficiency and low computational cost in high - dimensional optimization problems, thus effectively dealing with high - dimensional black - box optimization problems.