The Relation Between Intrinsic Complexity and Generalization of a Model and the Geometric Curvature

Zi-Ang L,LUO Si-Wei,YANG Jian,LIU Yun-Hui,ZOU Qi
DOI: https://doi.org/10.3321/j.issn:0254-4164.2007.07.006
2007-01-01
Chinese Journal of Computers
Abstract:The paper uses the conception of curvature from the point of view of differential geom- etry to explore the intrinsic model complexity that is free of reparametrization;and then through theoretical analysis,shows that the Gauss-Kroneker curvature can describe the whole properties of the statistical manifold,thus gives the relation between curvature and the volume of the mani- fold.An algorithm is proposed based on study of the solution locus in the neighborhood of the ex- pectation of parameters to calculate the curvature of the model.This paper proves that the future residual that is qualified to measure the generalizability can be expressed by using the intrinsic curvature array of model,from which a new model selection criterion GKCIC is given.It not only considers the factors such as the number of parameters,sample size and functional form,but also with very elear and intuitive geometric understanding of model selection.The geometrical method of the statistieal manifold is compared with the statistical learning theory,in particular,the VC dimension versus the Gauss-Kroneker curvature.By running the algorithm on synthetie and real datasets,the author argue that the GKCIC work efficiently.
What problem does this paper attempt to address?