Abstract:Inspired by the structure of spherical harmonics, we propose the truncated kernel stochastic gradient descent (T-kernel SGD) algorithm with a least-square loss function for spherical data fitting. T-kernel SGD employs a "truncation" operation, enabling the application of series-based kernels function in stochastic gradient descent, thereby avoiding the difficulties of finding suitable closed-form kernel functions in high-dimensional spaces. In contrast to traditional kernel SGD, T-kernel SGD is more effective in balancing bias and variance by dynamically adjusting the hypothesis space during iterations. The most significant advantage of the proposed algorithm is that it can achieve theoretically optimal convergence rates using a constant step size (independent of the sample size) while overcoming the inherent saturation problem of kernel SGD. Additionally, we leverage the structure of spherical polynomials to derive an equivalent T-kernel SGD, significantly reducing storage and computational costs compared to kernel SGD. Typically, T-kernel SGD requires only $\mathcal{O}(n^{1+\frac{d}{d-1}\epsilon})$ computational complexity and $\mathcal{O}(n^{\frac{d}{d-1}\epsilon})$ storage to achieve optimal rates for the d-dimensional sphere, where $0<\epsilon<\frac{1}{2}$ can be arbitrarily small if the optimal fitting or the underlying space possesses sufficient regularity. This regularity is determined by the smoothness parameter of the objective function and the decaying rate of the eigenvalues of the integral operator associated with the kernel function, both of which reflect the difficulty of the estimation problem. Our main results quantitatively characterize how this prior information influences the convergence of T-kernel SGD. The numerical experiments further validate the theoretical findings presented in this paper.

SLERP-TVDRK (STVDRK) Methods for Ordinary Differential Equations on Spheres

SLERP-TVDRK (STVDRK) Methods for Ordinary Differential Equations on Spheres

Runge-Kutta Discontinuous Local Evolution Galerkin Methods for the Shallow Water Equations on the Cubed-Sphere Grid

Spectral Volume from a DG perspective: Oscillation Elimination, Stability, and Optimal Error Estimates

Fourth-order time-stepping for stiff PDEs on the sphere

Analysis of any order Runge-Kutta Spectral Volume Schemes for 1D Hyperbolic Equations

Adaptive rational Krylov methods for exponential Runge--Kutta integrators

A New Framework for Solving Partial Differential Equations Using Semi-Analytical Explicit RK(N)-type Integrators

An Efficient Fourth-Order Three-Point Scheme for Solving Some Nonlinear Dispersive Wave Equations

Compact finite-difference scheme for some Sobolev type equations with Dirichlet boundary conditions

Truncated Kernel Stochastic Gradient Descent on Spheres

Construction of High-Order Accuracy Implicit Residual Smoothing Schemes

Total Variation Error Bounds for the Accelerated Exponential Euler Scheme Approximation of Parabolic Semilinear SPDEs

A New Class of Structure-Preserving Stochastic Exponential Runge-Kutta Integrators for Stochastic Differential Equations

High Order Numerical Algorithms Based on Biquadratic Spline Collocation for Two-Dimensional Parabolic Partial Differential Equations

An Unconditionally Stable Space-Time FE Method for the Korteweg-de Vries Equation

The Runge--Kutta discontinuous Galerkin method with stage-dependent polynomial spaces for hyperbolic conservation laws

Solving parabolic equations on the unit sphere via Laplace transforms and radial basis functions

$L^2$-Wasserstein contraction of modified Euler schemes for SDEs with high diffusivity and applications

A new numerical algorithm for the solutions of hyperbolic partial differential equations in $(2+1)$-dimensional space

Stochastic C-stability and B-consistency of explicit and implicit Euler-type schemes