Abstract:With this follow-up paper, we continue developing a mathematical framework based on information geometry for representing physical objects. The long-term goal is to lay down informational foundations for physics, especially quantum physics. We assume that we can now model information sources as univariate normal probability distributions N(μ,σ0), as before, but with a constant σ0 not necessarily equal to 1. Then, we also relaxed the independence condition when modeling m sources of information. Now, we model m sources with a multivariate normal probability distribution Nm(μ,Σ0) with a constant variance–covariance matrix Σ0 not necessarily diagonal, i.e., with covariance values different to 0, which leads to the concept of modes rather than sources. Invoking Schrödinger's equation, we can still break the information into m quantum harmonic oscillators, one for each mode, and with energy levels independent of the values of σ0, altogether leading to the concept of "intrinsic". Similarly, as in our previous work with the estimator's variance, we found that the expectation of the quadratic Mahalanobis distance to the sample mean equals the energy levels of the quantum harmonic oscillator, being the minimum quadratic Mahalanobis distance at the minimum energy level of the oscillator and reaching the "intrinsic" Cramér–Rao lower bound at the lowest energy level. Also, we demonstrate that the global probability density function of the collective mode of a set of m quantum harmonic oscillators at the lowest energy level still equals the posterior probability distribution calculated using Bayes' theorem from the sources of information for all data values, taking as a prior the Riemannian volume of the informative metric. While these new assumptions certainly add complexity to the mathematical framework, the results proven are invariant under transformations, leading to the concept of "intrinsic" information-theoretic models, which are essential for developing physics.

New information-estimation results for poisson, binomial and negative binomial models

Information and Estimation over Binomial and Negative Binomial Models

The Interplay Between Information and Estimation Measures

Relative Entropy and Score Function: New Information-Estimation Relationships Through Arbitrary Additive Perturbation

Estimators of Entropy and Information via Inference in Probabilistic Models

Mutual Information and Conditional Mean Estimation in Poisson Channels

Information measures and geometry of the hyperbolic exponential families of Poincaré and hyperboloid distributions

Statistical inference of entropy functions of generalized inverse exponential model under progressive type-II censoring test

Estimation and Confidence Intervals for Mutual Information: Issues in Convergence for Non-Normal Distributions

Information divergences and likelihood ratios of Poisson processes and point patterns

Intrinsic Information-Theoretic Models

Additive non-Gaussian noise channels: mutual information and conditional mean estimation

Convergence of Statistical Estimators via Mutual Information Bounds

Synergy, suppression and immorality: forward differences of the entropy function

Conditional Deng Entropy, Joint Deng Entropy and Generalized Mutual Information

Beyond the I-MMSE relation: derivatives of mutual information in Gaussian channels

Sibson's $α$-Mutual Information and its Variational Representations

On Exact and $\infty$-Rényi Common Informations

Mutual Information Estimation via $f$-Divergence and Data Derangements

Optimum estimation via gradients of partition functions and information measures: a statistical-mechanical perspective

Kullback-Leibler Divergence and Akaike Information Criterion in General Hidden Markov Models