The dynamics of neural codes in biological and artificial neural networks

Guillermo B. Morales

2024-02-20

Abstract:Advancing our knowledge of how the brain processes information remains a key challenge in neuroscience. This thesis combines three different approaches to the study of the dynamics of neural networks and their encoding representations: a computational approach, that builds upon basic biological features of neurons and their networks to construct effective models that can simulate their structure and dynamics; a machine-learning approach, which draws a parallel with the functional capabilities of brain networks, allowing us to infer the dynamical and encoding properties required to solve certain input-processing tasks; and a final, theoretical treatment, which will take us into the fascinating hypothesis of the "critical" brain as the mathematical foundation that can explain the emergent collective properties arising from the interactions of millions of neurons. Hand in hand with physics, we venture into the realm of neuroscience to explain the existence of quasi-universal scaling properties across brain regions, setting out to quantify the distance of their dynamics from a critical point. Next, we move into the grounds of artificial intelligence, where the very same theory of critical phenomena will prove very useful for explaining the effects of biologically-inspired plasticity rules in the forecasting ability of Reservoir Computers. Halfway into our journey, we explore the concept of neural representations of external stimuli, unveiling a surprising link between the dynamical regime of neural networks and the optimal topological properties of such representation manifolds. The thesis ends with the singular problem of representational drift in the process of odor encoding carried out by the olfactory cortex, uncovering the potential synaptic plasticity mechanisms that could explain this recently observed phenomenon.

Neurons and Cognition,Disordered Systems and Neural Networks,Statistical Mechanics

What problem does this paper attempt to address?

The main problem this paper attempts to address is understanding how the brain processes information. Specifically, the authors combine three different approaches to study the dynamic properties of neural networks and their encoded representations:

1. **Computational Methods**: Constructing effective models to simulate the structure and dynamic behavior of neurons and their networks based on their fundamental biological characteristics.
2. **Machine Learning Methods**: Inferring the dynamic and encoding properties required to solve certain input processing tasks by drawing analogies to the functional capabilities of brain networks.
3. **Theoretical Methods**: Exploring the hypothesis of the "critical state of the brain" as a mathematical foundation to explain the emergent collective properties resulting from the interactions of millions of neurons.

Using methods from physics, the paper attempts to explain the quasi-universal scaling properties present in different brain regions and quantify the extent to which these dynamic properties are close to the critical point. Additionally, the authors apply the same critical phenomena theory to the field of artificial intelligence to explain the predictive capabilities of biologically inspired shaping rules in Reservoir Computing.

The paper also explores the concept of neural representation of external stimuli, revealing the connection between the dynamic states of neural networks and the optimal topological properties of these representations. Finally, the paper concludes with the issue of representational drift observed in the olfactory cortex during odor encoding, analyzing possible synaptic plasticity mechanisms to explain this newly observed phenomenon. Throughout the study, the authors intersperse historical backgrounds of scientific breakthroughs, brain-inspired literary references, and philosophical debates about existential reality.