Sharpening Occam's Razor

Ming Li,John Tromp,Paul Vitanyi
DOI: https://doi.org/10.48550/arXiv.cs/0201005
2002-10-11
Abstract:We provide a new representation-independent formulation of Occam's razor theorem, based on Kolmogorov complexity. This new formulation allows us to: (i) Obtain better sample complexity than both length-based and VC-based versions of Occam's razor theorem, in many applications. (ii) Achieve a sharper reverse of Occam's razor theorem than previous work. Specifically, we weaken the assumptions made in an earlier publication, and extend the reverse to superpolynomial running times.
Machine Learning,Disordered Systems and Neural Networks,Artificial Intelligence,Computational Complexity,Probability,Data Analysis, Statistics and Probability
What problem does this paper attempt to address?