Abstract:The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $\Pi_{\mathcal{A}}$, on small enough neighborhoods of $\Pi_{\mathcal{A}}$, using an identity due to Topsøe and the Pinsker's inequality, assuming that the input set of the channel is finite and the constraint set $\mathcal{A}$ is polyhedral, i.e., can be described by (possibly multiple but) finitely many linear constraints. Counterexamples demonstrating nonexistence of such a quadratic bound are provided for the case of infinitely many linear constraints and the case of infinite input sets. Using Taylor's theorem with the remainder term, rather than the Pinsker's inequality and invoking Moreau's decomposition theorem the exact characterization of the slowest decrease of the mutual information with the distance to $\Pi_{\mathcal{A}}$ is determined on small neighborhoods of $\Pi_{\mathcal{A}}$. Corresponding results for classical-quantum channels are established under separable output Hilbert space assumption for the quadratic bound and under finite-dimensional output Hilbert space assumption for the exact characterization. Implications of these observations for the channel coding problem and applications of the proof techniques to related problems are discussed.

Derivative of Mutual Information at Zero SNR: the Gaussian-Noise Case

Additive non-Gaussian noise channels: mutual information and conditional mean estimation

Mutual Information and Minimum Mean-square Error in Gaussian Channels

Asymptotically Optimal One-Bit Quantizer Design for Weak-Signal Detection in Generalized Gaussian Noise and Lossy Binary Communication Channel.

Beyond the I-MMSE relation: derivatives of mutual information in Gaussian channels

Estimation in Gaussian Noise: Properties of the Minimum Mean-Square Error

Estimation of Non-Gaussian Random Variables in Gaussian Noise: Properties of the MMSE.

Optimal input signal distribution and per-sample mutual information for nondispersive nonlinear optical fiber channel at large SNR

A Practical Consideration on Convex Mutual Information

Path-integral approach to mutual information calculation for nonlinear communication channel with small second dispersion at large signal-to-noise power ratio

Submodularity of Mutual Information for Multivariate Gaussian Sources with Additive Noise

The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions

Gray--Wyner and Mutual Information Regions for Doubly Symmetric Binary Sources and Gaussian Sources

Mutual Information Rate of Gaussian and Truncated Gaussian Inputs on Intensity-Driven Signal Transduction Channels

Signal-to-noise ratio aware minimaxity and higher-order asymptotics

Mutual Information Bounded by Fisher Information

The Double-Sided Information Bottleneck Function

Mutual Information and Conditional Mean Estimation in Poisson Channels

New information-estimation results for poisson, binomial and negative binomial models

Breakdown of a concavity property of mutual information for non-Gaussian channels