Abstract:Random smoothing data augmentation is a unique form of regularization that can prevent overfitting by introducing noise to the input data, encouraging the model to learn more generalized features. Despite its success in various applications, there has been a lack of systematic study on the regularization ability of random smoothing. In this paper, we aim to bridge this gap by presenting a framework for random smoothing regularization that can adaptively and effectively learn a wide range of ground truth functions belonging to the classical Sobolev spaces. Specifically, we investigate two underlying function spaces: the Sobolev space of low intrinsic dimension, which includes the Sobolev space in $D$-dimensional Euclidean space or low-dimensional sub-manifolds as special cases, and the mixed smooth Sobolev space with a tensor structure. By using random smoothing regularization as novel convolution-based smoothing kernels, we can attain optimal convergence rates in these cases using a kernel gradient descent algorithm, either with early stopping or weight decay. It is noteworthy that our estimator can adapt to the structural assumptions of the underlying data and avoid the curse of dimensionality. This is achieved through various choices of injected noise distributions such as Gaussian, Laplace, or general polynomial noises, allowing for broad adaptation to the aforementioned structural assumptions of the underlying data. The convergence rate depends only on the effective dimension, which may be significantly smaller than the actual data dimension. We conduct numerical experiments on simulated data to validate our theoretical results.

A Big Data Processing Technique Based on Tikhonov Regularization

A Tikhonov Regularization Based Algorithm for Scattered Data with Random Noise

Statistical Information of Low-Dimensional Processes Hidden in Random State Data

A Numerical Differentiation Method and Its Application to Reconstruction of Discontinuity

An Analysis of Tikhonov Regularization for Nonlinear Ill-Posed Problems under a General Smoothness Assumption

Tikhonov-type regularization for nonlinear inverse problems with multiple repeated measurement data

First and Second Order Numerical Differentiation with Tikhonov Regularization

Extract the Information from Big Data with Randomly Distributed Noise

Piecewise Tikhonov Regularization Scheme To Reconstruct Discontinuous Density In Computerized Tomography

Numerical Differentiation by a Tikhonov Regularization Method Based on the Discrete Cosine Transform

Reconstruction of High Order Derivatives from Input Data

Tikhonov Regularization for Stochastic Non-Smooth Convex Optimization in Hilbert Spaces

Random Smoothing Regularization in Kernel Gradient Descent Learning

Convergent Data-Driven Regularizations for CT Reconstruction

Image Super-Resolutton Reconstruction Based On Adaptive Interpolation Norm Regularization

Convergence rates for critical point regularization

On the Generalized Discrepancy Principle for Tikhonov Regularization in Hilbert Scales

Regularization of linear inverse problems with irregular noise using embedding operators

Small noise analysis for Tikhonov and RKHS regularizations

A PRIORI STRATEGY IN DISCRETIZATION OF THE TIKHONOV REGULARIZATION