Abstract:With this follow-up paper, we continue developing a mathematical framework based on information geometry for representing physical objects. The long-term goal is to lay down informational foundations for physics, especially quantum physics. We assume that we can now model information sources as univariate normal probability distributions N(μ,σ0), as before, but with a constant σ0 not necessarily equal to 1. Then, we also relaxed the independence condition when modeling m sources of information. Now, we model m sources with a multivariate normal probability distribution Nm(μ,Σ0) with a constant variance–covariance matrix Σ0 not necessarily diagonal, i.e., with covariance values different to 0, which leads to the concept of modes rather than sources. Invoking Schrödinger's equation, we can still break the information into m quantum harmonic oscillators, one for each mode, and with energy levels independent of the values of σ0, altogether leading to the concept of "intrinsic". Similarly, as in our previous work with the estimator's variance, we found that the expectation of the quadratic Mahalanobis distance to the sample mean equals the energy levels of the quantum harmonic oscillator, being the minimum quadratic Mahalanobis distance at the minimum energy level of the oscillator and reaching the "intrinsic" Cramér–Rao lower bound at the lowest energy level. Also, we demonstrate that the global probability density function of the collective mode of a set of m quantum harmonic oscillators at the lowest energy level still equals the posterior probability distribution calculated using Bayes' theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. While these new assumptions certainly add complexity to the mathematical framework, the results proven are invariant under transformations, leading to the concept of "intrinsic" information-theoretic models, which are essential for developing physics.

Nonextensive Information Theoretical Machine

Entropy and mutual information in models of deep neural networks

Information-Theoretic Local Minima Characterization and Regularization

Flexible model weighting for one-dependence estimators based on point-wise independence analysis

Information-Theoretic Foundations for Machine Learning

On Statistical Efficiency in Learning

A New Information Theory of Certainty for Machine Learning

Model Information As an Analysis Tool in Deep Learning

Tsallis eXtropy

Discriminability-Transferability Trade-Off: An Information-Theoretic Perspective

Information Theory and its Relation to Machine Learning

An Information-Theoretic Approach to Generalization Theory

A Novel Dual of Shannon Information and Weighting Scheme

Intrinsic Information-Theoretic Models

A Differential Entropy Estimator for Training Neural Networks

Transductive Minimax Probability Machine.

Slicing Mutual Information Generalization Bounds for Neural Networks

Understanding the Generalization Ability of Deep Learning Algorithms: A Kernelized Renyi's Entropy Perspective

An Information-Theoretic Regularizer for Lossy Neural Image Compression

Information-Theoretic Generalization Bounds for Deep Neural Networks

Atr Applications Of Minimax Entropy Models Of Texture And Shape