Abstract:Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the "curse of dimensionality" in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding.

Exact computation of the Maximum Entropy Potential of spiking neural networks models

Maximum Caliber Infers Effective Coupling and Response from Spiking Networks

Maximum entropy models reveal the excitatory and inhibitory correlation structures in cortical neuronal activity

Ising-like model replicating time-averaged spiking behaviour of in vitro neuronal networks

Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains

Exact minimax entropy models of large-scale neuronal activity

Stimulus-dependent maximum entropy models of neural population codes

Exactly solvable statistical physics models for large neuronal populations

A Dynamical State Underlying the Second Order Maximum Entropy Principle in Neuronal Networks

Common-input models for multiple neural spike-train data

Thermodynamic analog of integrate-and-fire neuronal networks by maximum entropy modelling

Maximum Entropy Principle Analysis in Network Systems with Short-Time Recordings

Ensemble Inhibition and Excitation in the Human Cortex: an Ising Model Analysis with Uncertainties

Decoding Spikes in a Spiking Neuronal Network

A Simple Probabilistic Spiking Neuron Model with Hebbian Learning Rules.

Efficient Inference on a Network of Spiking Neurons using Deep Learning

Applications of information geometry to spiking neural network activity

Entropy of Neuronal Spike Patterns

Approximating Nonlinear Functions With Latent Boundaries in Low-Rank Excitatory-Inhibitory Spiking Networks

Maximum Likelihood Decoding of Neuronal Inputs from an Interspike Interval Distribution