Abstract:The representation of the natural-density, heterogeneous connectivity of neuronal network models at relevant spatial scales remains a challenge for Computational Neuroscience and Neuromorphic Computing. In particular, the memory demands imposed by the vast number of synapses in brain-scale network simulations constitutes a major obstacle. Limiting the number resolution of synaptic weights appears to be a natural strategy to reduce memory and compute load. In this study, we investigate the effects of a limited synaptic-weight resolution on the dynamics of recurrent spiking neuronal networks resembling local cortical circuits, and develop strategies for minimizing deviations from the dynamics of networks with high-resolution synaptic weights. We mimic the effect of a limited synaptic weight resolution by replacing normally distributed synaptic weights by weights drawn from a discrete distribution, and compare the resulting statistics characterizing firing rates, spike-train irregularity, and correlation coefficients with the reference solution. We show that a naive discretization of synaptic weights generally leads to a distortion of the spike-train statistics. Only if the weights are discretized such that the mean and the variance of the total synaptic input currents are preserved, the firing statistics remains unaffected for the types of networks considered in this study. For networks with sufficiently heterogeneous in-degrees, the firing statistics can be preserved even if all synaptic weights are replaced by the mean of the weight distribution. We conclude that even for simple networks with non-plastic neurons and synapses, a discretization of synaptic weights can lead to substantial deviations in the firing statistics, unless the discretization is performed with care and guided by a rigorous validation process.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to reduce the memory requirements by decreasing the resolution of synaptic weights in neural network simulations while keeping the network dynamic characteristics not significantly affected. Specifically, the research focuses on recurrent spiking neuronal networks, which simulate local cortical circuits. The researchers explored the impact of limited synaptic weight resolution on network dynamics and developed strategies to minimize the dynamic deviation from high - resolution synaptic weight networks. ### Main problems 1. **Memory requirement problem**: Large - scale neural network simulations require a large amount of memory, especially when simulating brain - scale networks. Reducing the resolution of synaptic weights can significantly reduce memory requirements, but it may lead to changes in network dynamics. 2. **Dynamics preservation problem**: How to maintain the dynamic characteristics of the network while reducing the synaptic weight resolution, especially statistical characteristics such as firing rate, spike - train irregularity and correlation coefficient. ### Research methods - **Discretization method**: The researchers simulated the limited synaptic weight resolution by replacing the continuously distributed synaptic weights with discretely distributed weights. They compared two discretization methods: - **Naive discretization**: Directly discretize the weight distribution within a predefined interval. - **Optimized discretization**: Adjust the boundaries of the discretization interval to ensure that the mean and variance of the discretized weight distribution are the same as those of the original continuous distribution. - **Verification method**: Use statistical indicators (such as firing rate, coefficient of variation and correlation coefficient) to evaluate the differences in network dynamics before and after discretization. ### Main findings - **Impact of naive discretization**: Naive discretization usually leads to distortion of spike - train statistical characteristics. - **Effect of optimized discretization**: If the mean and variance of the total synaptic input current are kept unchanged during the discretization process, the firing statistical characteristics of the network can remain unchanged. - **Role of network heterogeneity**: For in - degree networks with sufficient heterogeneity, even if all synaptic weights are replaced with the mean of the weight distribution, the firing statistical characteristics can remain unchanged. ### Conclusion The research results show that even in simple non - plastic neurons and synaptic networks, the discretization of synaptic weights may lead to significant deviations in firing statistical characteristics, unless the discretization process is carefully designed and strictly verified. For the network model used in this study, synaptic weights can be replaced with low - resolution weights without affecting its macroscopic dynamic characteristics, thus saving a large amount of memory.

Prominent characteristics of recurrent neuronal networks are robust against low synaptic weight resolution

Is a 4-bit synaptic weight resolution enough? - Constraints on enabling spike-timing dependent plasticity in neuromorphic hardware

Network algorithmics and the emergence of the cortical synaptic-weight distribution

Reconstruction of recurrent synaptic connectivity of thousands of neurons from simulated spiking activity

Memory-efficient neurons and synapses for spike-timing-dependent-plasticity in large-scale spiking networks

The interplay between homeostatic synaptic scaling and homeostatic structural plasticity maintains the robust firing rate of neural networks

Synchronization of the Small-World Neuronal Network with Unreliable Synapses.

A framework for studying synaptic plasticity with neural spike train data

Synaptic Reorganization in Scaled Networks of Controlled Size.

Structural Insights Into the Dynamic Evolution of Neuronal Networks as Synaptic Density Decreases

Inference of network connectivity from temporally binned spike trains

Extending dynamic memory of spiking neuron networks

How synaptic plasticity influences spike synchronization and its transitions in complex neuronal network.

Non-exchangeable networks of integrate-and-fire neurons: spatially-extended mean-field limit of the empirical measure

Memory-Efficient Synaptic Connectivity for Spike-Timing- Dependent Plasticity

Local dendritic balance enables learning of efficient representations in networks of spiking neurons

Constrained plasticity reserve as a natural way to control frequency and weights in spiking neural networks

Adaptive Synaptic Scaling in Spiking Networks for Continual Learning and Enhanced Robustness

Sub-realtime simulation of a neuronal network of natural density

Large-Scale Algorithmic Search Identifies Stiff and Sloppy Dimensions in Synaptic Architectures Consistent With Murine Neocortical Wiring

Self-organization of microcircuits in networks of neurons with plastic synapses