Abstract:Basis Function (BF) expansions are a cornerstone of any engineer's toolbox for computational function approximation which shares connections with both neural networks and Gaussian processes. Even though BF expansions are an intuitive and straightforward model to use, they suffer from quadratic computational complexity in the number of BFs if the predictive variance is to be computed. We develop a method to automatically select the most important BFs for prediction in a sub-domain of the model domain. This significantly reduces the computational complexity of computing predictions while maintaining predictive accuracy. The proposed method is demonstrated using two numerical examples, where reductions up to 50-75% are possible without significantly reducing the predictive accuracy.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: **How to significantly reduce the computational complexity of the Basis Function (BF) expansion model while ensuring the prediction accuracy?** Specifically, when the number of basis functions \(L\) is large, the complexity of calculating the predicted value and its variance will rise sharply with the increase of \(L\), reaching a quadratic complexity of \(O(L^{2})\). This makes the consumption of computing resources too large in practical applications, especially in application scenarios such as multi - agent navigation and magnetic field mapping that require real - time processing of a large amount of data, and it is difficult to meet the real - time requirements.
### Core of the problem
1. **High computational complexity**: When calculating the predicted value, especially the predicted variance, in the traditional basis function expansion model, the complexity is \(O(L^{2})\), which is unacceptable for large - scale data sets or real - time application scenarios.
2. **Maintaining prediction accuracy**: Although the computational complexity can be reduced by reducing the number of basis functions, this usually leads to a decline in prediction accuracy. Therefore, how to maintain the prediction accuracy as much as possible while reducing the number of basis functions is a key issue.
### Solution
The paper proposes an **adaptive basis function selection method**, which can automatically select the most important basis functions for a specific test input during the prediction stage, thereby significantly reducing the computational complexity while maintaining the prediction accuracy. The specific steps are as follows:
1. **Determine important basis functions**: Select a subset \(J\) of basis functions that can best represent the original model through an optimization method, so that the prediction model \(\hat{f}(x)=\sum_{j\in J}\phi_{j}(x)\theta_{j}\) can approximate the original model \(f(x)=\sum_{i = 1}^{L}\phi_{i}(x)\theta_{i}\) within a given error range.
The optimization problem can be formalized as:
\[
\min_J\|f(x)-\hat{f}(x)\|_2^2\quad\text{s.t.}\quad|J|=n_J < L
\]
2. **Adaptive selection**: This method can dynamically adjust the basis functions used according to the specific test input, ensuring that only the basis functions important for the current input are used during prediction, thereby reducing unnecessary calculations.
3. **No additional data required**: Unlike traditional feature selection methods, this method does not require additional training data for basis function selection, but selects based on the learned weights and the information of the basis functions themselves.
4. **Extension to sparse Gaussian processes**: This method is not only applicable to general basis function expansion models, but can also be extended to the commonly used sparse Gaussian process (GP) approximation model, further improving its applicability and efficiency.
### Application scenarios
This method is particularly suitable for the following scenarios:
- **Multi - agent navigation**: When performing multi - agent navigation and motion planning in large - scale nonlinear fields, the communication complexity increases with the increase of the number of basis functions, and this method can effectively reduce the computational complexity.
- **Magnetic field mapping**: Used in indoor navigation or other applications that require real - time processing of a large amount of magnetic field data to reduce the computational burden.
- **Tire friction modeling**: In self - driving vehicles, improve the prediction accuracy while maintaining the computational budget.
In conclusion, this paper solves the problem of excessively high computational complexity in the basis function expansion model in large - scale data sets or real - time application scenarios by proposing an adaptive basis function selection method, while maintaining the prediction accuracy.