Mapping-to-Parameter Nonlinear Functional Regression with Novel B-spline Free Knot Placement Algorithm

Chengdong Shi,Ching-Hsun Tseng,Wei Zhao,Xiao-Jun Zeng

2024-01-27

Abstract:We propose a novel approach to nonlinear functional regression, called the Mapping-to-Parameter function model, which addresses complex and nonlinear functional regression problems in parameter space by employing any supervised learning technique. Central to this model is the mapping of function data from an infinite-dimensional function space to a finite-dimensional parameter space. This is accomplished by concurrently approximating multiple functions with a common set of B-spline basis functions by any chosen order, with their knot distribution determined by the Iterative Local Placement Algorithm, a newly proposed free knot placement algorithm. In contrast to the conventional equidistant knot placement strategy that uniformly distributes knot locations based on a predefined number of knots, our proposed algorithms determine knot location according to the local complexity of the input or output functions. The performance of our knot placement algorithms is shown to be robust in both single-function approximation and multiple-function approximation contexts. Furthermore, the effectiveness and advantage of the proposed prediction model in handling both function-on-scalar regression and function-on-function regression problems are demonstrated through several real data applications, in comparison with four groups of state-of-the-art methods.

Machine Learning

What problem does this paper attempt to address?

The main problems this paper attempts to address are:

1. **Nonlinear Function Regression Problem**: Specifically, for the Function-on-Function (FOF) regression problem, existing methods mainly focus on linear models, while research on nonlinear models is relatively scarce. The paper proposes a new nonlinear FOF regression model, called the Mapping-to-Parameter (M2P) function model.

2. **Free Knot Placement Problem of B-Spline Basis Functions**: Traditional B-spline basis functions usually adopt an equidistant knot placement strategy, which is ineffective when dealing with complex functions. The paper proposes a new Iterative Local Placement (ILP) algorithm to determine the optimal common knot distribution to adapt to the local complexity of different functions.

Specifically, the main contributions of the paper include:

1. **Functional Representation and New Knot Placement Algorithm**: The ILP algorithm is proposed to determine the optimal common B-spline basis function set for input or output functions based on the characteristics of the data itself. This functional representation method is the first to apply free knot placement technology to functional regression.

2. **Modeling Nonlinear Relationships in Parameter Space**: By mapping input and output functions to a finite-dimensional parameter space and using conventional supervised learning techniques in this space to learn nonlinear function relationships. This method overcomes the obstacles of existing methods in revealing unknown nonlinear relationships in complex function spaces.

3. **Performance Evaluation of the ILP Algorithm**: Through in-depth comparative studies under single-function and multi-function approximation settings, the approximation performance of the ILP algorithm is evaluated, demonstrating its effectiveness.

4. **Validation of the Predictive Performance of the M2P Model**: Through various practical examples, the robustness and effectiveness of the M2P model in handling scalar output prediction and function output prediction tasks are verified. It is benchmarked against several widely used prediction models, providing empirical evidence supporting its superior performance.

In summary, this paper aims to address key challenges in the nonlinear FOF regression problem by introducing the new M2P model and ILP algorithm, particularly in terms of efficiency and accuracy when dealing with complex function data.