Links between the Logarithmic Sobolev Inequality and the convolution inequalities for Entropy and Fisher Information

Thomas A. Courtade
DOI: https://doi.org/10.48550/arXiv.1608.05431
2016-08-19
Abstract:Relative to the Gaussian measure on $\mathbb{R}^d$, entropy and Fisher information are famously related via Gross' logarithmic Sobolev inequality (LSI). These same functionals also separately satisfy convolution inequalities, as proved by Stam. We establish a dimension-free inequality that interpolates among these relations. Several interesting corollaries follow: (i) the deficit in the LSI satisfies a convolution inequality itself; (ii) the deficit in the LSI controls convergence in the entropic and Fisher information central limit theorems; and (iii) the LSI is stable with respect to HWI jumps (i.e., a jump in any of the convolution inequalities associated to the HWI functionals). Another consequence is that the convolution inequalities for Fisher information and entropy powers are reversible in general, up to a factor depending on the Stam defect. An improved form of Nelson's hypercontractivity estimate also follows. Finally, we speculate on the possibility of an analogous reverse Brunn-Minkowski inequality and a related upper bound on surface area associated to Minkowski sums.
Information Theory
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to establish the connections among the Logarithmic Sobolev Inequality (LSI), the Entropy Power Inequality (EPI), and the Fisher Information Inequality (FII), and to explore the applications of these inequalities in information theory. Specifically, the main objectives of the paper include: 1. **Establish an interpolation inequality**: The author attempts to establish a general inequality that can connect LSI, EPI, and FII. This inequality can not only unify these three important inequalities mathematically but also provide a new perspective to understand the relationships among them. 2. **Explore the convolution inequality of LSI**: Through the above - mentioned interpolation inequality, the author proves that LSI itself also satisfies a convolution inequality. This result is of great significance for understanding the stability and convergence of LSI in information theory. 3. **Apply to the Central Limit Theorem**: The paper explores the applications of these inequalities in the Central Limit Theorems (CLTs) in information theory, especially the CLTs of entropy and Fisher information. The author provides estimates of the convergence rates of these theorems, especially the lower - bound estimates in terms of short - term behavior. 4. **Stability analysis**: The paper also studies the stability of LSI under HWI jumps (i.e., the jumps of relative entropy, Wasserstein distance, and Fisher information under convolution). The author proves that if a certain random variable has an HWI jump during the convolution process, then a lower - bound estimate of the LSI defect can be given. 5. **Improve Nelson's hypercontractive estimate**: Based on the above - mentioned interpolation inequality, the author improves Nelson's hypercontractive estimate for the Ornstein - Uhlenbeck semigroup and obtains a more refined result. In general, by establishing a unified interpolation inequality, this paper not only deepens the understanding of the relationships among LSI, EPI, and FII but also provides new tools and methods for the applications of these inequalities in information theory.