Functional Adaptive Huber Linear Regression

Ling Peng,Xiaohui Liu,Heng Lian

2024-09-17

Abstract:Robust estimation has played an important role in statistical and machine learning. However, its applications to functional linear regression are still under-developed. In this paper, we focus on Huber's loss with a diverging robustness parameter which was previously used in parametric models. Compared to other robust methods such as median regression, the distinction is that the proposed method aims to estimate the conditional mean robustly, instead of estimating the conditional median. We only require $(1+\kappa)$-th moment assumption ($\kappa>0$) on the noise distribution, and the established error bounds match the optimal rate in the least-squares case as soon as $\kappa\ge 1$. We establish convergence rate in probability when the functional predictor has a finite 4-th moment, and finite-sample bound with exponential tail when the functional predictor is Gaussian, in terms of both prediction error and $L^2$ error. The results also extend to the case of functional estimation in a reproducing kernel Hilbert space (RKHS).

Statistics Theory

What problem does this paper attempt to address?

The paper attempts to address the issue of enhancing robustness in Functional Linear Regression (FLR). Specifically, it focuses on improving the robustness of estimates in the presence of outliers or heavy-tailed errors. Traditional linear regression methods assume that errors follow a normal distribution, which can lead to unreliable parameter estimates when dealing with non-Gaussian or heavy-tailed data. To solve this problem, the paper proposes a new method by introducing an adaptive Huber loss function to improve the robustness of the FLR model.

The main contributions of the paper are as follows:
1. **Adaptive Huber Loss Function**: By adjusting the robustness parameter γ in the Huber loss function, it allows for more robust estimation of the conditional mean, rather than the conditional median, within the FLR framework.
2. **Theoretical Results**: The paper establishes the convergence rates of adaptive Huber regression under different scenarios and proves that the method can achieve optimal convergence rates when the noise satisfies certain moment conditions.
3. **Robustness and Efficiency**: By appropriately adjusting γ, it is possible to achieve the best balance between bias and robustness when estimating the conditional mean.
4. **Finite Sample Bound**: The paper also provides exponential tail bounds in the finite sample case, further demonstrating the robustness of the method.

In summary, the paper aims to enhance the robustness and estimation accuracy of functional linear regression models when dealing with complex data structures by introducing an adaptive Huber loss function.