Abstract:Regularization is a long-standing challenge for ill-posed linear inverse problems, and a prototype is the Fredholm integral equation of the first kind with additive Gaussian measurement noise. We introduce a new RKHS regularization adaptive to measurement data and the underlying linear operator. This RKHS arises naturally in a variational approach, and its closure is the function space in which we can identify the true solution. Also, we introduce a small noise analysis to compare regularization norms by sharp convergence rates in the small noise limit. Our analysis shows that the RKHS- and $L^2$-regularizers yield the same convergence rate when their optimal hyper-parameters are selected using the true solution, and the RKHS-regularizer has a smaller multiplicative constant. However, in computational practice, the RKHS regularizer significantly outperforms the $L^2$-and $l^2$-regularizers in producing consistently converging estimators when the noise level decays or the observation mesh refines.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to propose a new adaptive regularization method based on Reproducing Kernel Hilbert Space (RKHS) for the Fredholm integral equation of the first kind, which is a typical ill - posed linear inverse problem. Specifically: 1. **Introducing a new regularization method**: The author proposes a new adaptive RKHS regularization method, which can be adjusted according to the measurement data and the underlying linear operator. This RKHS naturally appears in the variational method, and its closure is the function space that can identify the true solution. 2. **Theoretical analysis and comparison**: By introducing small - noise analysis, the author makes a theoretical comparison of different regularization norms, especially RKHS regularization and $L^2$ regularization. The analysis shows that, in the case of optimal hyper - parameter selection, these two regularization methods have the same convergence rate, but the multiplicative constant of RKHS regularization is smaller. 3. **Numerical experiment verification**: Through numerical experiments, the author shows that when the noise level decreases or the observation grid is refined, RKHS regularization is significantly superior to $L^2$ and $l^2$ regularizations and can produce more consistent and convergent estimators. ### Main contributions 1. **Proposing a new adaptive RKHS regularization**: A new adaptive RKHS for Tikhonov regularization is introduced. The reproducing kernel of this RKHS is determined by the integral kernel and data, and its closure is the linear subspace where the inverse problem has a unique solution. Therefore, this regularization method is adaptive and performs better numerically. 2. **Introducing a small - noise analysis framework**: A small - noise analysis method is proposed to compare different regularization norms in the small - noise limit. The analysis shows that when the optimal hyper - parameter selection minimizes the expected error, the two regularization methods have the same convergence rate, but RKHS regularization has a smaller multiplicative constant. ### Summary of mathematical formulas - **Loss function**: \[ E(\phi) := \|L_K\phi - y\|^2_{L^2_\mu(T)} \] - **RKHS regularization**: \[ \phi_\lambda = \arg \min_\phi E_\lambda(\phi) \quad \text{with} \quad E_\lambda(\phi) := E(\phi) + \lambda \|\phi\|^2_{\mathcal{H}_G} \] where $\mathcal{H}_G$ is the RKHS defined based on the operator $L_G$. - **Mean - square error (MSE) in the small - noise limit**: \[ e_{\mathcal{H}_G}(\lambda) = \sigma^2 A(\lambda) + \lambda^2 B(\lambda) \] \[ e_{L^2_\rho}(\lambda) = \sigma^2 \tilde{A}(\lambda) + \lambda^2 \tilde{B}(\lambda) \] ### Conclusion This research is the first to use adaptive RKHS to regularize Fredholm - type inverse problems, and proves its superiority through strict theoretical analysis and numerical experiments. This provides a new method for automatically selecting reproducing kernels in general linear machine learning and inverse problems, and also provides a theoretical framework for the selection of regularization norms.

An adaptive RKHS regularization for Fredholm integral equations

Small noise analysis for Tikhonov and RKHS regularizations

Scalable iterative data-adaptive RKHS regularization

An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Analysis of Regularized Least-Squares in Reproducing Kernel Kreĭn Spaces

Adaptive optimal regularization of the linear ill posed integral equations

Regularized Regression Problem in hyper-RKHS for Learning Kernels

Regularization of linear inverse problems with irregular noise using embedding operators

Generalization Properties of Hyper-Rkhs and Its Applications

Regularization of linear ill-posed problems involving multiplication operators

Linear Convergence of Inexact Descent Method and Inexact Proximal Gradient Algorithms for Lower-Order Regularization Problems

Convergence Conditions of Online Regularized Statistical Learning in Reproducing Kernel Hilbert Space With Non-Stationary Data

On the Asymptotical Regularization for Linear Inverse Problems in Presence of White Noise

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

Learning the optimal Tikhonov regularizer for inverse problems

On regularized Radon-Nikodym differentiation

Regularization in kernel learning

Exponential relaxation data analysis by parametrized regularization of severely ill-posed Fredholm integral equations of the first kind

An asymptotical regularization with convex constraints for inverse problems

Hanke-Raus heuristic rule for iteratively regularized stochastic gradient descent