Abstract:Networks with stochastic variables described by heavy tailed lognormal distribution are ubiquitous in nature, and hence they deserve an exact information-theoretic characterization. We derive analytical formulas for mutual information between elements of different networks with correlated lognormally distributed activities. In a special case, we find an explicit expression for mutual information between neurons when neural activities and synaptic weights are lognormally distributed, as suggested by experimental data. Comparison of this expression with the case when these two variables have short tails, reveals that mutual information with heavy tails for neurons and synapses is generally larger and can diverge for some finite variances in presynaptic firing rates and synaptic weights. This result suggests that evolution might prefer brains with heterogeneous dynamics to optimize information processing.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to provide an accurate information - theoretic description by analyzing the mutual information (MI) between different elements in a network with log - normal - distributed activities. Specifically, the paper focuses on how to derive the analytical expression of mutual information in different networks with correlated log - normal - distributed activities, especially when neuronal activities and synaptic weights are log - normally distributed. This research is particularly important for understanding the information - processing mechanisms in the brain, because experimental data suggest that neuronal activities and synaptic weights may follow a log - normal distribution. The key points of the paper include: 1. **Analytical formula for mutual information**: The author derives a general formula for the mutual information between two random vectors that have log - normal distributions. This formula depends only on the covariance matrices of the corresponding Gaussian random vectors, including the joint distribution and the marginal distribution. 2. **Application in specific networks**: The paper considers several networks with simple topological structures and gives the explicit formula for mutual information in each case. These networks are special cases of a more complex two - part network, which frequently appears in biology and sociology. 3. **Application in neuroscience**: In particular, the paper applies the above mutual information formula to neuroscience and derives the mutual information between the presynaptic neuronal activity vector and the postsynaptic neuronal activity. This helps to understand the information - transfer mechanism between neurons in the cerebral cortex. Through these studies, the paper reveals that log - normally distributed neuronal activities and synaptic weights usually have greater mutual information, and under certain limited presynaptic firing rates and synaptic weight variances, the mutual information can diverge. This result implies that evolution may favor brains with heterogeneous dynamics to optimize information - processing capabilities.

Explicit mutual information for simple networks and neurons with lognormal activities

Explicit mutual information for simple networks and neurons with lognormal activities

Exact minimax entropy models of large-scale neuronal activity

Quantifying information stored in synaptic connections rather than in firing patterns of neural networks

Mutual information for the sparse stochastic block model

Applications of information geometry to spiking neural network activity

Alternating Shrinking Higher-order Interactions for Sparse Neural Population Activity

Identifying Effective Connectivity between Stochastic Neurons with Variable-Length Memory Using a Transfer Entropy Rate Estimator

A Graph Theoretic Interpretation of Neural Complexity

Rich dynamics caused by known biological brain network features resulting in stateful networks

Probabilistic models, compressible interactions, and neural coding

Mutual information in time-varying biochemical systems

How Far can Neural Correlations Reduce Uncertainty? Comparison of Information Transmission Rates for Markov and Bernoulli Processes

Path mutual information for a class of biochemical reaction networks

Entropy and mutual information in models of deep neural networks

Efficiency characterization of a large neuronal network: a causal information approach

Emergence of Irregular Activity in Networks of Strongly Coupled Conductance-Based Neurons

Coexistence of positive and negative information in information-epidemic dynamics on multiplex networks

Approximations of Shannon Mutual Information for Discrete Variables with Applications to Neural Population Coding

How do correlations shape the landscape of information?

The determining role of covariances in large networks of stochastic neurons