Abstract:The existing research on spectral algorithms, applied within a Reproducing Kernel Hilbert Space (RKHS), has primarily focused on general kernel functions, often neglecting the inherent structure of the input feature space. Our paper introduces a new perspective, asserting that input data are situated within a low-dimensional manifold embedded in a higher-dimensional Euclidean space. We study the convergence performance of spectral algorithms in the RKHSs, specifically those generated by the heat kernels, known as diffusion spaces. Incorporating the manifold structure of the input, we employ integral operator techniques to derive tight convergence upper bounds concerning generalized norms, which indicates that the estimators converge to the target function in strong sense, entailing the simultaneous convergence of the function itself and its derivatives. These bounds offer two significant advantages: firstly, they are exclusively contingent on the intrinsic dimension of the input manifolds, thereby providing a more focused analysis. Secondly, they enable the efficient derivation of convergence rates for derivatives of any k-th order, all of which can be accomplished within the ambit of the same spectral algorithms. Furthermore, we establish minimax lower bounds to demonstrate the asymptotic optimality of these conclusions in specific contexts. Our study confirms that the spectral algorithms are practically significant in the broader context of high-dimensional approximation.

Learning Theory of Distributed Spectral Algorithms

Distributed Spectral Pairwise Ranking Algorithms

Distributed Learning with Discretely Observed Functional Data

Random Inverse Problems Over Graphs: Decentralized Online Learning

Decentralized Online Learning for Random Inverse Problems over Graphs

Spectral algorithms for functional linear regression

A Distributed Block Chebyshev-Davidson Algorithm for Parallel Spectral Clustering

An Information Theoretic Framework for Distributed Learning Algorithms

Greedy Sparsity-Promoting Algorithms for Distributed Learning

Automated Spectral Kernel Learning

Spectral Algorithms on Manifolds through Diffusion

Distributed Learning with Convex SUM-of -Non-convex Objective

Learning over inherently distributed data

Spectral Inference Methods on Sparse Graphs: Theory and Applications

Distributed Sparse Total Least-Squares over Networks

Learning Theory of Distributed Regression with Bias Corrected Regularization Kernel Network

Distributed Sparse Recursive Least-Squares over Networks

The D-Subspace Algorithm for Online Learning over Distributed Networks

Distributed Learning with Indefinite Kernels

Distributed Learning With Dependent Samples