Abstract:Random smoothing data augmentation is a unique form of regularization that can prevent overfitting by introducing noise to the input data, encouraging the model to learn more generalized features. Despite its success in various applications, there has been a lack of systematic study on the regularization ability of random smoothing. In this paper, we aim to bridge this gap by presenting a framework for random smoothing regularization that can adaptively and effectively learn a wide range of ground truth functions belonging to the classical Sobolev spaces. Specifically, we investigate two underlying function spaces: the Sobolev space of low intrinsic dimension, which includes the Sobolev space in $D$-dimensional Euclidean space or low-dimensional sub-manifolds as special cases, and the mixed smooth Sobolev space with a tensor structure. By using random smoothing regularization as novel convolution-based smoothing kernels, we can attain optimal convergence rates in these cases using a kernel gradient descent algorithm, either with early stopping or weight decay. It is noteworthy that our estimator can adapt to the structural assumptions of the underlying data and avoid the curse of dimensionality. This is achieved through various choices of injected noise distributions such as Gaussian, Laplace, or general polynomial noises, allowing for broad adaptation to the aforementioned structural assumptions of the underlying data. The convergence rate depends only on the effective dimension, which may be significantly smaller than the actual data dimension. We conduct numerical experiments on simulated data to validate our theoretical results.

Approximation and Smoothing of Functions Based on Godunov Regularization

Approximation, regularization and smoothing of trigonometric splines

Smoothing the Nonsmoothness

Kernel Based Approximation in Sobolev Spaces with Radial Basis Functions

Smale's α-theory for inexact Newton methods under the γ-condition

Smoothing $\mathcal{L}^2$ gradients in iterative regularization

Error Bounds for a Kernel-Based Constrained Optimal Smoothing Approximation

Regularisation for the approximation of functions by mollified discretisation methods

Bridging Smoothness and Approximation: Theoretical Insights into Over-Smoothing in Graph Neural Networks

A Regularization Method For The Function Reconstruction From Approximate Average Fluxes

Coordinate descent methods beyond smoothness and separability

Gaussian smoothing gradient descent for minimizing functions (GSmoothGD)

Optimal error estimates for non-conforming approximations of linear parabolic problems with minimal regularity

Smoothed Moreau-Yosida Tensor Train Approximation of State-constrained Optimization Problems under Uncertainty

Random Smoothing Regularization in Kernel Gradient Descent Learning

Numerical approximation of variational problems with orthotropic growth

Constrained Global Optimization by Smoothing

Approximation of shape optimization problems with non-smooth PDE constraints

Approximation of optimization problems with constraints through kernel Sum-Of-Squares

On the Approximation of Kernel functions