Generalization Guarantees for a Binary Classification Framework for Two-Stage Multiple Kernel Learning

Purushottam Kar

DOI: https://doi.org/10.48550/arXiv.1302.0406

IF: 5.414

2013-02-02

Machine Learning

Abstract:We present generalization bounds for the TS-MKL framework for two stage multiple kernel learning. We also present bounds for sparse kernel learning formulations within the TS-MKL framework.

What problem does this paper attempt to address?

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Lin Luo,Hongye Su,Baofen Zheng,Junfeng Zhang

DOI: https://doi.org/10.3233/ifs-141149

2014-01-01

Journal of Intelligent & Fuzzy Systems

Abstract:In many applications of kernel method, such as support vector machine (SVM), the performance greatly depends on the choice of kernel. It is not always clear what is the most suitable kernel for one specified task. Multiple kernel learning (MKL) allows to optimize over linear combinations of kernels. A two-step approach, which alternately optimizes the standard SVM and kernel weights, is proposed to solve the non-convex MKL problem. The generalized convexity of MKL is studied which guarantees the strong duality of the corresponding optimization problem. A further contribution is the utilization of a computationally efficient inexact-projection-based method to optimize the standard SVM. In addition, a nonmonotone gradient method is proposed to optimize the kernel weights and the Hessian matrix is approximated with a variation of Limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) approach. The global convergence of the proposed two-step approach is analyzed. The utility of the proposed scheme is demonstrated by empirical study with several datasets, and its performance is compared with state-of-the-art methods in terms of accuracy and scalability. The results show that the proposed method performs as well as existing methods in accuracy, but takes much less time to converge to the stationary point.
Multi-Task Classification Hypothesis Space with Improved Generalization Bounds

Cong Li,Michael Georgiopoulos,Georgios C. Anagnostopoulos

DOI: https://doi.org/10.48550/arXiv.1312.2606

2013-12-10

Abstract:This paper presents a RKHS, in general, of vector-valued functions intended to be used as hypothesis space for multi-task classification. It extends similar hypothesis spaces that have previously considered in the literature. Assuming this space, an improved Empirical Rademacher Complexity-based generalization bound is derived. The analysis is itself extended to an MKL setting. The connection between the proposed hypothesis space and a Group-Lasso type regularizer is discussed. Finally, experimental results, with some SVM-based Multi-Task Learning problems, underline the quality of the derived bounds and validate the paper's analysis.

Machine Learning
A General Kernelization Framework for Learning Algorithms Based on Kernel PCA.

Changshui Zhang,Feiping Nie,Shiming Xiang

DOI: https://doi.org/10.1016/j.neucom.2009.08.014

IF: 6

2010-01-01

Neurocomputing

Abstract:In this paper, a general kernelization framework for learning algorithms is proposed via a two-stage procedure, i.e., transforming data by kernel principal component analysis (KPCA), and then directly performing the learning algorithm with the transformed data. It is worth noting that although a very few learning algorithms were also kernelized by this procedure before, why and under what condition this procedure is feasible have not been further studied. In this paper, we explicitly present this kernelization framework, and give a rigorous justification to reveal that under some mild conditions, the kernelization under this framework is equivalent to traditional kernel method. We show that these mild conditions are usually satisfied in most of learning algorithms. Therefore, most of learning algorithms can be kernelized under this framework without having to reformulate it into inner product form, which is a common yet vital step in traditional kernel methods. Enlightened by this framework, we also propose a novel kernel method based on the low-rank KPCA, which could be used to remove the noise in the feature space, speed up the kernel algorithm and improve the numerical stability for the kernel algorithm. Experiments are presented to verify the validity and effectiveness of the proposed methods.
Generalization in Kernel Regression Under Realistic Assumptions

Daniel Barzilai,Ohad Shamir

2023-12-26

Abstract:It is by now well-established that modern over-parameterized models seem to elude the bias-variance tradeoff and generalize well despite overfitting noise. Many recent works attempt to analyze this phenomenon in the relatively tractable setting of kernel regression. However, as we argue in detail, most past works on this topic either make unrealistic assumptions, or focus on a narrow problem setup. This work aims to provide a unified theory to upper bound the excess risk of kernel regression for nearly all common and realistic settings. Specifically, we provide rigorous bounds that hold for common kernels and for any amount of regularization, noise, any input dimension, and any number of samples. Furthermore, we provide relative perturbation bounds for the eigenvalues of kernel matrices, which may be of independent interest. These reveal a self-regularization phenomenon, whereby a heavy tail in the eigendecomposition of the kernel provides it with an implicit form of regularization, enabling good generalization. When applied to common kernels, our results imply benign overfitting in high input dimensions, nearly tempered overfitting in fixed dimensions, and explicit convergence rates for regularized regression. As a by-product, we obtain time-dependent bounds for neural networks trained in the kernel regime.

Machine Learning,Artificial Intelligence
Online Kernel Learning with a Near Optimal Sparsity Bound

Lijun Zhang,Jinfeng Yi,Rong Jin,Ming Lin,Xiaofei He

2013-01-01

Abstract:In this work, we focus on Online Sparse Kernel Learning that aims to online learn a kernel classifier with a bounded number of support vectors. Although many online learning algorithms have been proposed to learn a sparse kernel classifier, most of them fail to bound the number of support vectors used by the final solution which is the average of the intermediate kernel classifiers generated by online algorithms. The key idea of the proposed algorithm is to measure the difficulty in correctly classifying a training example by the derivative of a smooth loss function, and give a more chance to a difficult example to be a support vector than an easy one via a sampling scheme. Our analysis shows that when the loss function is smooth, the proposed algorithm yields similar performance guarantee as the standard online learning algorithm but with a near optimal number of support vectors (up to a poly(ln T) factor). Our empirical study shows promising performance of the proposed algorithm compared to the state-of-the-art algorithms for online sparse kernel learning.
Generalization Properties of Hyper-Rkhs and Its Applications

Fanghui Liu,Lei Shi,Xiaolin Huang,Jie Yang,Johan A. K. Suykens

2021-01-01

Journal of Machine Learning Research

Abstract:This paper generalizes regularized regression problems in a hyper-reproducing kernel Hilbert space (hyper-RKHS), illustrates its utility for kernel learning and out-of-sample extensions, and proves asymptotic convergence results for the introduced regression models in an approximation theory view. Algorithmically, we consider two regularized regression models with bivariate forms in this space, including kernel ridge regression (KRR) and support vector regression (SVR) endowed with hyper-RKHS, and further combine divide-and-conquer with Nystrum approximation for scalability in large sample cases. This framework is general: the underlying kernel is learned from a broad class, and can be positive definite or not, which adapts to various requirements in kernel learning. Theoretically, we study the convergence behavior of regularized regression algorithms in hyper-RKHS and derive the learning rates, which goes beyond the classical analysis on RKHS due to the non-trivial independence of pairwise samples and the characterisation of hyper-RKHS. Experimentally, results on several benchmarks suggest that the employed framework is able to learn a general kernel function form an arbitrary similarity matrix, and thus achieves a satisfactory performance on classification tasks.
Kernel machines with two layers and multiple kernel learning

Francesco Dinuzzo

DOI: https://doi.org/10.48550/arXiv.1001.2709

2010-01-15

Abstract:In this paper, the framework of kernel machines with two layers is introduced, generalizing classical kernel methods. The new learning methodology provide a formal connection between computational architectures with multiple layers and the theme of kernel learning in standard regularization methods. First, a representer theorem for two-layer networks is presented, showing that finite linear combinations of kernels on each layer are optimal architectures whenever the corresponding functions solve suitable variational problems in reproducing kernel Hilbert spaces (RKHS). The input-output map expressed by these architectures turns out to be equivalent to a suitable single-layer kernel machines in which the kernel function is also learned from the data. Recently, the so-called multiple kernel learning methods have attracted considerable attention in the machine learning literature. In this paper, multiple kernel learning methods are shown to be specific cases of kernel machines with two layers in which the second layer is linear. Finally, a simple and effective multiple kernel learning method called RLS2 (regularized least squares with two layers) is introduced, and his performances on several learning problems are extensively analyzed. An open source MATLAB toolbox to train and validate RLS2 models with a Graphic User Interface is available.

Machine Learning,Artificial Intelligence
Information-theoretic generalization bounds for black-box learning algorithms

Hrayr Harutyunyan,Maxim Raginsky,Greg Ver Steeg,Aram Galstyan

DOI: https://doi.org/10.48550/arXiv.2110.01584

2021-10-05

Abstract:We derive information-theoretic generalization bounds for supervised learning algorithms based on the information contained in predictions rather than in the output of the training algorithm. These bounds improve over the existing information-theoretic bounds, are applicable to a wider range of algorithms, and solve two key challenges: (a) they give meaningful results for deterministic algorithms and (b) they are significantly easier to estimate. We show experimentally that the proposed bounds closely follow the generalization gap in practical scenarios for deep learning.

Machine Learning
Two-stage multiple kernel learning with multiclass kernel polarization.

Tinghua Wang,Dongyan Zhao,Yansong Feng

DOI: https://doi.org/10.1016/j.knosys.2013.04.006

IF: 8.139

2013-01-01

Knowledge-Based Systems

Abstract:The success of kernel methods is very much dependent on the choice of kernels. Multiple kernel learning (MKL) aims at learning a combination of different kernels in order to better match the underlying problem instead of using a single fixed kernel. In this paper, we propose a simple but effective multiclass MKL method by a two-stage strategy, in which the first stage finds the kernel weights to combine the kernels, and the second stage trains a standard multiclass support vector machine (SVM). Specifically, we first present an evaluation criterion named multiclass kernel polarization (MKP) to assess the quality of a kernel in the multiclass classification scenario, and then develop a heuristic rule to directly assign a weight to each kernel based on the quality of the individual kernel. MKP is a multiclass extension of the kernel polarization, which is a universal kernel evaluation criterion for kernel design and learning. Comprehensive experiments are conducted on several UCI benchmark examples and the results well demonstrate the effectiveness and efficiency of our approach.
Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models

David X. Wu,Anant Sahai

2023-12-06

Abstract:We study the asymptotic generalization of an overparameterized linear model for multiclass classification under the Gaussian covariates bi-level model introduced in Subramanian et al.~'22, where the number of data points, features, and classes all grow together. We fully resolve the conjecture posed in Subramanian et al.~'22, matching the predicted regimes for generalization. Furthermore, our new lower bounds are akin to an information-theoretic strong converse: they establish that the misclassification rate goes to 0 or 1 asymptotically. One surprising consequence of our tight results is that the min-norm interpolating classifier can be asymptotically suboptimal relative to noninterpolating classifiers in the regime where the min-norm interpolating regressor is known to be optimal. The key to our tight analysis is a new variant of the Hanson-Wright inequality which is broadly useful for multiclass problems with sparse labels. As an application, we show that the same type of analysis can be used to analyze the related multilabel classification problem under the same bi-level ensemble.

Machine Learning
Regularized Regression Problem in hyper-RKHS for Learning Kernels

Fanghui Liu,Lei Shi,Xiaolin Huang,Jie Yang,Johan A. K. Suykens

2018-01-01

Abstract:This paper generalizes the two-stage kernel learning framework, illustrates its utility for kernel learning and out-of-sample extensions, and proves {asymptotic} convergence results for the introduced kernel learning model. Algorithmically, we extend target alignment by hyper-kernels in the two-stage kernel learning framework. The associated kernel learning task is formulated as a regression problem in a hyper-reproducing kernel Hilbert space (hyper-RKHS), i.e., learning on the space of kernels itself. To solve this problem, we present two regression models with bivariate forms in this space, including kernel ridge regression (KRR) and support vector regression (SVR) in the hyper-RKHS. By doing so, it provides significant model flexibility for kernel learning with outstanding performance in real-world applications. Specifically, our kernel learning framework is general, that is, the learned underlying kernel can be positive definite or indefinite, which adapts to various requirements in kernel learning. Theoretically, we study the convergence behavior of these learning algorithms in the hyper-RKHS and derive the learning rates. Different from the traditional approximation analysis in RKHS, our analyses need to consider the non-trivial independence of pairwise samples and the characterisation of hyper-RKHS. To the best of our knowledge, this is the first work in learning theory to study the approximation performance of regularized regression problem in hyper-RKHS.
On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models

Peizhong Ju,Xiaojun Lin,Ness B. Shroff

DOI: https://doi.org/10.48550/arXiv.2103.05243

2023-03-08

Abstract:In this paper, we study the generalization performance of min $\ell_2$-norm overfitting solutions for the neural tangent kernel (NTK) model of a two-layer neural network with ReLU activation that has no bias term. We show that, depending on the ground-truth function, the test error of overfitted NTK models exhibits characteristics that are different from the "double-descent" of other overparameterized linear models with simple Fourier or Gaussian features. Specifically, for a class of learnable functions, we provide a new upper bound of the generalization error that approaches a small limiting value, even when the number of neurons $p$ approaches infinity. This limiting value further decreases with the number of training samples $n$. For functions outside of this class, we provide a lower bound on the generalization error that does not diminish to zero even when $n$ and $p$ are both large.

Machine Learning,Statistics Theory
Two-stage sparse multi-kernel optimization classifier method for more accurate and explainable prediction

Zhiwang Zhang,Hongliang Sun,Shuqing Li,Jing He,Jie Cao,Guanghai Cui,Gang Wang

DOI: https://doi.org/10.1016/j.eswa.2023.120635

IF: 8.5

2023-06-11

Expert Systems with Applications

Abstract:Since many classifier methods cannot identify and remove redundant observations and unrelated attributes from data, they usually give more inconsistent classification between actual and predicted outputs. Introducing single- or multi-kernel functions to classifier models helps to solve non-linearly separable problems, but it reduces the predictive interpretability. In this paper, we put forward a novel two-stage sparse multi-kernel optimization classifier (TSMOC) method under the framework of combining support vector classifier (SVC) and multiple kernel learning (MKL), aiming to solve the above issues. With our defined row and column multi-kernel matrices, the proposed method employs iterative updates to compute the l0- norm approximations of coefficients and weights, which extract important observations and attributes besides prediction. Based on the experimental results on thirteen real-world datasets, TSMOC generally outperforms the other seven classifiers of SVC, l1- norm SVC, least-squares SVC, LASSO classifier, SimpleMKL, EasyMKL, and DeepMKL. Besides obtaining the best classification accuracy, TSMOC extracts the smallest number of observations and attributes important to prediction and it can provide explainable prediction with their contribution percentages.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Efficient sparse generalized multiple kernel learning

Haiqin Yang,Zenglin Xu,Jieping Ye,Irwin King,Michael R Lyu

DOI: https://doi.org/10.1109/TNN.2010.2103571

Abstract:Kernel methods have been successfully applied in various applications. To succeed in these applications, it is crucial to learn a good kernel representation, whose objective is to reveal the data similarity precisely. In this paper, we address the problem of multiple kernel learning (MKL), searching for the optimal kernel combination weights through maximizing a generalized performance measure. Most MKL methods employ the L(1)-norm simplex constraints on the kernel combination weights, which therefore involve a sparse but non-smooth solution for the kernel weights. Despite the success of their efficiency, they tend to discard informative complementary or orthogonal base kernels and yield degenerated generalization performance. Alternatively, imposing the L(p)-norm (p > 1) constraint on the kernel weights will keep all the information in the base kernels. This leads to non-sparse solutions and brings the risk of being sensitive to noise and incorporating redundant information. To tackle these problems, we propose a generalized MKL (GMKL) model by introducing an elastic-net-type constraint on the kernel weights. More specifically, it is an MKL model with a constraint on a linear combination of the L(1)-norm and the squared L(2)-norm on the kernel weights to seek the optimal kernel combination weights. Therefore, previous MKL problems based on the L(1)-norm or the L(2)-norm constraints can be regarded as special cases. Furthermore, our GMKL enjoys the favorable sparsity property on the solution and also facilitates the grouping effect. Moreover, the optimization of our GMKL is a convex optimization problem, where a local solution is the global optimal solution. We further derive a level method to efficiently solve the optimization problem. A series of experiments on both synthetic and real-world datasets have been conducted to show the effectiveness and efficiency of our GMKL.
Neural Generalization of Multiple Kernel Learning

Ahmad Navid Ghanizadeh,Kamaledin Ghiasi-Shirazi,Reza Monsefi,Mohammadreza Qaraei

DOI: https://doi.org/10.1007/s11063-024-11516-0

IF: 2.565

2024-02-08

Neural Processing Letters

Abstract:Multiple Kernel Learning (MKL) is a conventional way to learn the kernel function in kernel-based methods. MKL algorithms enhance the performance of kernel methods. However, these methods have a lower complexity compared to deep models and are inferior to them regarding recognition accuracy. Deep learning models can learn complex functions by applying nonlinear transformations to data through several layers. In this paper, we show that a typical MKL algorithm can be interpreted as a one-layer neural network with linear activation functions. By this interpretation, we propose a Neural Generalization of Multiple Kernel Learning (NGMKL), which extends the conventional MKL framework to a multi-layer neural network with nonlinear activation functions. Our experiments show that the proposed method, which has a higher complexity than traditional MKL methods, leads to higher recognition accuracy on several benchmarks.

computer science, artificial intelligence
Generalization error of spectral algorithms

Maksim Velikanov,Maxim Panov,Dmitry Yarotsky

2024-03-18

Abstract:The asymptotically precise estimation of the generalization of kernel methods has recently received attention due to the parallels between neural networks and their associated kernels. However, prior works derive such estimates for training by kernel ridge regression (KRR), whereas neural networks are typically trained with gradient descent (GD). In the present work, we consider the training of kernels with a family of $\textit{spectral algorithms}$ specified by profile $h(\lambda)$, and including KRR and GD as special cases. Then, we derive the generalization error as a functional of learning profile $h(\lambda)$ for two data models: high-dimensional Gaussian and low-dimensional translation-invariant model. Under power-law assumptions on the spectrum of the kernel and target, we use our framework to (i) give full loss asymptotics for both noisy and noiseless observations (ii) show that the loss localizes on certain spectral scales, giving a new perspective on the KRR saturation phenomenon (iii) conjecture, and demonstrate for the considered data models, the universality of the loss w.r.t. non-spectral details of the problem, but only in case of noisy observation.

Machine Learning
An Information-Theoretic Approach to Generalization Theory

Borja Rodríguez-Gálvez,Ragnar Thobaben,Mikael Skoglund

2024-08-20

Abstract:We investigate the in-distribution generalization of machine learning algorithms. We depart from traditional complexity-based approaches by analyzing information-theoretic bounds that quantify the dependence between a learning algorithm and the training data. We consider two categories of generalization guarantees: 1) Guarantees in expectation: These bounds measure performance in the average case. Here, the dependence between the algorithm and the data is often captured by information measures. While these measures offer an intuitive interpretation, they overlook the geometry of the algorithm's hypothesis class. Here, we introduce bounds using the Wasserstein distance to incorporate geometry, and a structured, systematic method to derive bounds capturing the dependence between the algorithm and an individual datum, and between the algorithm and subsets of the training data. 2) PAC-Bayesian guarantees: These bounds measure the performance level with high probability. Here, the dependence between the algorithm and the data is often measured by the relative entropy. We establish connections between the Seeger--Langford and Catoni's bounds, revealing that the former is optimized by the Gibbs posterior. We introduce novel, tighter bounds for various types of loss functions. To achieve this, we introduce a new technique to optimize parameters in probabilistic statements. To study the limitations of these approaches, we present a counter-example where most of the information-theoretic bounds fail while traditional approaches do not. Finally, we explore the relationship between privacy and generalization. We show that algorithms with a bounded maximal leakage generalize. For discrete data, we derive new bounds for differentially private algorithms that guarantee generalization even with a constant privacy parameter, which is in contrast to previous bounds in the literature.

Machine Learning
Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective

Yuxin Dong,Tieliang Gong,Hong Chen,Chen Li

2023-05-02

Abstract:Recently, information theoretic analysis has become a popular framework for understanding the generalization behavior of deep neural networks. It allows a direct analysis for stochastic gradient/Langevin descent (SGD/SGLD) learning algorithms without strong assumptions such as Lipschitz or convexity conditions. However, the current generalization error bounds within this framework are still far from optimal, while substantial improvements on these bounds are quite challenging due to the intractability of high-dimensional information quantities. To address this issue, we first propose a novel information theoretical measure: kernelized Renyi's entropy, by utilizing operator representation in Hilbert space. It inherits the properties of Shannon's entropy and can be effectively calculated via simple random sampling, while remaining independent of the input dimension. We then establish the generalization error bounds for SGD/SGLD under kernelized Renyi's entropy, where the mutual information quantities can be directly calculated, enabling evaluation of the tightness of each intermediate step. We show that our information-theoretical bounds depend on the statistics of the stochastic gradients evaluated along with the iterates, and are rigorously tighter than the current state-of-the-art (SOTA) results. The theoretical findings are also supported by large-scale empirical studies1.

Machine Learning
Generalization Error of Generalized Linear Models in High Dimensions

Melikasadat Emami,Mojtaba Sahraee-Ardakan,Parthe Pandit,Sundeep Rangan,Alyson K. Fletcher

DOI: https://doi.org/10.48550/arXiv.2005.00180

2020-05-01

Abstract:At the heart of machine learning lies the question of generalizability of learned rules over previously unseen data. While over-parameterized models based on neural networks are now ubiquitous in machine learning applications, our understanding of their generalization capabilities is incomplete. This task is made harder by the non-convexity of the underlying learning problems. We provide a general framework to characterize the asymptotic generalization error for single-layer neural networks (i.e., generalized linear models) with arbitrary non-linearities, making it applicable to regression as well as classification problems. This framework enables analyzing the effect of (i) over-parameterization and non-linearity during modeling; and (ii) choices of loss function, initialization, and regularizer during learning. Our model also captures mismatch between training and test distributions. As examples, we analyze a few special cases, namely linear regression and logistic regression. We are also able to rigorously and analytically explain the \emph{double descent} phenomenon in generalized linear models.

Machine Learning
Incomplete-view oriented kernel learning method with generalization error bound

Yingjie Tian,Saiji Fu,Jingjing Tang

DOI: https://doi.org/10.1016/j.ins.2021.10.011

IF: 8.1

2021-12-01

Information Sciences

Abstract:Multi-view learning (MVL) is a promising direction and most MVL methods work under the assumption that data are complete in all views. However, this assumption is often violated in practice due to the difficulties of high cost, equipment failure, and so on. Besides, the amount of data with missing views is particularly huge. Therefore, it is challenging yet valuable to fully leverage the large-scale incomplete-view data. Enlightened by the reduced support vector machine (RSVM) and multi-view privileged support vector machine (PSVM-2V), this paper proposes an efficient kernel method called reduced PSVM-2V (RPSVM-2V). It can not only provide a novel solution to process incomplete-view data, but also can be adjusted to address large-scale complete-view leaning problem efficiently. In addition, the idea of replacing the full kernel with a smaller rectangular reduced kernel is extended to develop another two MVL methods, i.e., reduced SVM-2K (RSVM-2K) and reduced multiple kernel learning method (RMKL). Furthermore, we analyze the generalization error bounds of RPSVM-2V and two extensions by using Rademacher complexity. The comprehensive experiments demonstrate that our proposed models can achieve comparable performance with less time and memory cost. The spectral analysis has further verified the effectiveness of the reduced kernel used in our models.

computer science, information systems

Generalization Guarantees for a Binary Classification Framework for Two-Stage Multiple Kernel Learning

Generalized Convexity-Based Inexact Projection Method for Multiple Kernel Learning

Multi-Task Classification Hypothesis Space with Improved Generalization Bounds

A General Kernelization Framework for Learning Algorithms Based on Kernel PCA.

Generalization in Kernel Regression Under Realistic Assumptions

Online Kernel Learning with a Near Optimal Sparsity Bound

Generalization Properties of Hyper-Rkhs and Its Applications

Kernel machines with two layers and multiple kernel learning

Information-theoretic generalization bounds for black-box learning algorithms

Two-stage multiple kernel learning with multiclass kernel polarization.

Precise Asymptotic Generalization for Multiclass Classification with Overparameterized Linear Models

Regularized Regression Problem in hyper-RKHS for Learning Kernels

On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models

Two-stage sparse multi-kernel optimization classifier method for more accurate and explainable prediction

Efficient sparse generalized multiple kernel learning

Neural Generalization of Multiple Kernel Learning

Generalization error of spectral algorithms

An Information-Theoretic Approach to Generalization Theory

Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective

Generalization Error of Generalized Linear Models in High Dimensions

Incomplete-view oriented kernel learning method with generalization error bound