Abstract:Inspired by the structure of spherical harmonics, we propose the truncated kernel stochastic gradient descent (T-kernel SGD) algorithm with a least-square loss function for spherical data fitting. T-kernel SGD employs a "truncation" operation, enabling the application of series-based kernels function in stochastic gradient descent, thereby avoiding the difficulties of finding suitable closed-form kernel functions in high-dimensional spaces. In contrast to traditional kernel SGD, T-kernel SGD is more effective in balancing bias and variance by dynamically adjusting the hypothesis space during iterations. The most significant advantage of the proposed algorithm is that it can achieve theoretically optimal convergence rates using a constant step size (independent of the sample size) while overcoming the inherent saturation problem of kernel SGD. Additionally, we leverage the structure of spherical polynomials to derive an equivalent T-kernel SGD, significantly reducing storage and computational costs compared to kernel SGD. Typically, T-kernel SGD requires only $\mathcal{O}(n^{1+\frac{d}{d-1}\epsilon})$ computational complexity and $\mathcal{O}(n^{\frac{d}{d-1}\epsilon})$ storage to achieve optimal rates for the d-dimensional sphere, where $0<\epsilon<\frac{1}{2}$ can be arbitrarily small if the optimal fitting or the underlying space possesses sufficient regularity. This regularity is determined by the smoothness parameter of the objective function and the decaying rate of the eigenvalues of the integral operator associated with the kernel function, both of which reflect the difficulty of the estimation problem. Our main results quantitatively characterize how this prior information influences the convergence of T-kernel SGD. The numerical experiments further validate the theoretical findings presented in this paper.

Probability-Generating Function Kernels for Spherical Data

Generating Probability Distributions on Intervals and Spheres: Convex Decomposition

Generating Probability Distributions on Intervals and Spheres with Application to Finite Element Method.

Kernel density estimation with polyspherical data and its applications

Truncated Kernel Stochastic Gradient Descent on Spheres

Probability Density Function Estimation Using Gamma Kernels

The GeometricKernels Package: Heat and Matérn Kernels for Geometric Learning on Manifolds, Meshes, and Graphs

Poisson Kernel-Based Clustering on the Sphere: Convergence Properties, Identifiability, and a Method of Sampling

-Kernel Coresets for Stochastic Points

Adaptive indefinite kernels in hyperbolic spaces

Spherical random projection

Subdifferential characterization of probability functions under Gaussian distribution

Positive semidefinite kernels that are axially symmetric on the sphere and stationary in time: spectral and semi-spectral theory, and constructive approaches

Uniform approximation of common Gaussian process kernels using equispaced Fourier grids

Weighted Bergman kernels for nearly holomorphic functions on bounded symmetric domains

Polyharmonic and Related Kernels on Manifolds: Interpolation and Approximation

Compactly-supported nonstationary kernels for computing exact Gaussian processes on big data

Distributed Uncertainty Quantification of Kernel Interpolation on Spheres

A unified approach to goodness-of-fit testing for spherical and hyperspherical data