Abstract:Recent advances have clarified theoretical learning accuracy in Bayesian inference, revealing that the asymptotic behavior of metrics such as generalization loss and free energy, assessing predictive accuracy, is dictated by a rational number unique to each statistical model, termed the learning coefficient (real log canonical threshold) . For models meeting regularity conditions, their learning coefficients are known. However, for singular models not meeting these conditions, exact values of learning coefficients are provided for specific models like reduced-rank regression, but a broadly applicable calculation method for these learning coefficients in singular models remains elusive. The problem of determining learning coefficients relates to finding normal crossings of Kullback-Leibler divergence in algebraic geometry. In this context, it is crucial to perform appropriate coordinate transformations and blow-ups. This paper introduces an approach that utilizes properties of the log-likelihood ratio function for constructing specific variable transformations and blow-ups to uniformly calculate the real log canonical threshold. It was found that linear independence in the differential structure of the log-likelihood ratio function significantly influences the real log canonical threshold. This approach has not been considered in previous research. In this approach, the paper presents cases and methods for calculating the exact values of learning coefficients in statistical models that satisfy a simple condition next to the regularity conditions (semi-regular models), offering examples of learning coefficients for two-parameter semi-regular models and mixture distribution models with a constant mixing ratio.

Learning Theory Estimates for Coefficient-Based Regularized Regression

Concentration Estimates for Learning with ℓ1-Regularizer and Data Dependent Hypothesis Spaces

Coefficient-based regularized distribution regression

Optimal Rates for Coefficient-Based Regularized Regression

Learning with coefficient-based regularization and ℓ 1 −penalty

Regularized Regression Problem in hyper-RKHS for Learning Kernels

Sparse Kernel Regression with Coefficient-based $\Ell_q-$Regularization.

Concentration Estimates for Learning with Unbounded Sampling

Kernel Regression with Coefficient-based l(q)-regularization

Learning Rates of Least-Square Regularized Regression with Polynomial Kernels

Learning Theory Viewpoint of Approximation by Positive Linear Operators

Analysis of Least Squares Regularized Regression in Reproducing Kernel Krein Spaces

Robust Regularized Kernel Regression.

Learning Analysis of Kernel Ridgeless Regression with Asymmetric Kernel Learning

Regularization in kernel learning

Analysis of Regularized Least-Squares in Reproducing Kernel Kreĭn Spaces

Error Analysis on Regularized Regression Based on the Maximum Correntropy Criterion

Optimal Rates for Regularized Conditional Mean Embedding Learning

Learning Coefficients in Semi-Regular Models

A Statistical Theory of Regularization-Based Continual Learning

Analysis of Regularized Learning for Linear-functional Data in Banach Spaces