Abstract:Recent advances have clarified theoretical learning accuracy in Bayesian inference, revealing that the asymptotic behavior of metrics such as generalization loss and free energy, assessing predictive accuracy, is dictated by a rational number unique to each statistical model, termed the learning coefficient (real log canonical threshold) . For models meeting regularity conditions, their learning coefficients are known. However, for singular models not meeting these conditions, exact values of learning coefficients are provided for specific models like reduced-rank regression, but a broadly applicable calculation method for these learning coefficients in singular models remains elusive. The problem of determining learning coefficients relates to finding normal crossings of Kullback-Leibler divergence in algebraic geometry. In this context, it is crucial to perform appropriate coordinate transformations and blow-ups. This paper introduces an approach that utilizes properties of the log-likelihood ratio function for constructing specific variable transformations and blow-ups to uniformly calculate the real log canonical threshold. It was found that linear independence in the differential structure of the log-likelihood ratio function significantly influences the real log canonical threshold. This approach has not been considered in previous research. In this approach, the paper presents cases and methods for calculating the exact values of learning coefficients in statistical models that satisfy a simple condition next to the regularity conditions (semi-regular models), offering examples of learning coefficients for two-parameter semi-regular models and mixture distribution models with a constant mixing ratio.

Real Log Canonical Thresholds at Non-singular Points

Learning Coefficients in Semi-Regular Models

Upper Bound of Real Log Canonical Threshold of Tensor Decomposition and its Application to Bayesian Inference

Classification of real hyperplane singularities by real log canonical thresholds

Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory

CANONICAL THRESHOLDING FOR NON-SPARSE HIGH-DIMENSIONAL LINEAR REGRESSION

Information theoretic limits of learning a sparse rule

Consideration on the learning efficiency of multiple-layered neural networks with linear units

On the maximum likelihood estimation in general log-linear models

Blow-up Algorithm for Sum-of-Products Polynomials and Real Log Canonical Thresholds

Maximum likelihood thresholds of generic linear concentration models

Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression

Estimating the Local Learning Coefficient at Scale

Information-Theoretic Bayes Risk Lower Bounds for Realizable Models

Information-theoretic Analysis of the Gibbs Algorithm: An Individual Sample Approach

On the Tightness of Information-Theoretic Bounds on Generalization Error of Learning Algorithms.

Infinitesimal structure of log canonical thresholds

Theoretical characterization of uncertainty in high-dimensional linear classification

Statistical Learning Theory of Quasi-Regular Cases

On the accuracy of self-normalized log-linear models

General bounds on the quality of Bayesian coresets