Abstract:Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the "curse of dimensionality" in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding.

Neuronal Computation Using High-Order Statistics

Correlated Neuronal Computation

Moment Neuronal Networks: Stochastic Computation in Neuronal Systems

Impact of spatially correlated noise on neuronal firing.

Stochastic neurodynamical computation of brain functions

Correlated Firing and the Information Represented by Neurons in Short Epochs

A Dynamical State Underlying the Second Order Maximum Entropy Principle in Neuronal Networks

Co-operativity in neurons and the role of noise in brain

Exploiting Noise As a Resource for Computation and Learning in Spiking Neural Networks

Correlations and the Encoding of Information in the Nervous System

An Information Theoretic Approach to the Contributions of the Firing Rates and the Correlations Between the Firing of Neurons.

Interplay between Graph Topology and Correlations of Third Order in Spiking Neuronal Networks

Entropy of Neuronal Spike Patterns

Learning from higher-order statistics, efficiently: hypothesis tests, random features, and neural networks

Suprathreshold Stochastic Resonance in Neural Processing Tuned by Correlation.

Joint Statistics of Strongly Correlated Neurons via Dimensional Reduction

Characterizing variability in nonlinear recurrent neuronal networks

Theory of population coupling and applications to describe high order correlations in large populations of interacting neurons

Identifying Effective Connectivity between Stochastic Neurons with Variable-Length Memory Using a Transfer Entropy Rate Estimator

Moment neural network and an efficient numerical method for modeling irregular spiking activity

The determining role of covariances in large networks of stochastic neurons