Abstract:Many experiments have suggested that the brain operates close to a critical state, based on signatures of criticality such as power-law distributed neuronal avalanches. In neural network models, criticality is a dynamical state that maximizes information processing capacities, e.g. sensitivity to input, dynamical range and storage capacity, which makes it a favorable candidate state for brain function. Although models that self-organize towards a critical state have been proposed, the relation between criticality signatures and learning is still unclear. Here, we investigate signatures of criticality in a self-organizing recurrent neural network (SORN). Investigating criticality in the SORN is of particular interest because it has not been developed to show criticality. Instead, the SORN has been shown to exhibit spatio-temporal pattern learning through a combination of neural plasticity mechanisms and it reproduces a number of biological findings on neural variability and the statistics and fluctuations of synaptic efficacies. We show that, after a transient, the SORN spontaneously self-organizes into a dynamical state that shows criticality signatures comparable to those found in experiments. The plasticity mechanisms are necessary to attain that dynamical state, but not to maintain it. Furthermore, onset of external input transiently changes the slope of the avalanche distributions - matching recent experimental findings. Interestingly, the membrane noise level necessary for the occurrence of the criticality signatures reduces the model's performance in simple learning tasks. Overall, our work shows that the biologically inspired plasticity and homeostasis mechanisms responsible for the SORN's spatio-temporal learning abilities can give rise to criticality signatures in its activity when driven by random input, but these break down under the structured input of short repeating sequences.

Critically Slow Learning in Flashcard Learning Models

Unbounded Human Learning: Optimal Scheduling for Spaced Repetition

Learning fast changing slow in spiking neural networks

Understanding Short-Range Memory Effects in Deep Neural Networks

How to Leverage Predictive Uncertainty Estimates for Reducing Catastrophic Forgetting in Online Continual Learning

Thinking fast and slow and science education

MgSvF: Multi-Grained Slow vs. Fast Framework for Few-Shot Class-Incremental Learning

Criterial Learning and Feedback Delay: Insights from Computational Models and Behavioral Experiments

Online Fast Adaptation and Knowledge Accumulation: a New Approach to Continual Learning

MgSvF: Multi-Grained Slow Versus Fast Framework for Few-Shot Class-Incremental Learning.

Strategy Execution in Cognitive Skill Learning: An Item-Level Test of Candidate Models.

Reducing Catastrophic Forgetting in Self Organizing Maps with Internally-Induced Generative Replay

Nonlinear slow-timescale mechanisms in synaptic plasticity

Interactive Continual Learning: Fast and Slow Thinking

Optimality-based Analysis of XCSF Compaction in Discrete Reinforcement Learning

Continual learning, deep reinforcement learning, and microcircuits: a novel method for clever game playing

Episodic Memory for Learning Subjective-Timescale Models

Adaptive Memory Replay for Continual Learning

Learning from Snapshots of Discrete and Continuous Data Streams

Criticality meets learning: Criticality signatures in a self-organizing recurrent neural network

Learn the Time to Learn: Replay Scheduling in Continual Learning