Abstract:Various animals, including humans, have been suggested to perform Bayesian inferences to handle noisy, time-varying external information. In performing Bayesian inference by the brain, the prior distribution must be acquired and represented by sampling noisy external inputs. However, the mechanism by which neural activities represent such distributions has not yet been elucidated. Our findings reveal that networks with modular structures, composed of fast and slow modules, are adept at representing this prior distribution, enabling more accurate Bayesian inferences. Specifically, the modular network that consists of a main module connected with input and output layers and a sub-module with slower neural activity connected only with the main module outperformed networks with uniform time scales. Prior information was represented specifically by the slow sub-module, which could integrate observed signals over an appropriate period and represent input means and variances. Accordingly, the neural network could effectively predict the time-varying inputs. Furthermore, by training the time scales of neurons starting from networks with uniform time scales and without modular structure, the above slow-fast modular network structure and the division of roles in which prior knowledge is selectively represented in the slow sub-modules spontaneously emerged. These results explain how the prior distribution for Bayesian inference is represented in the brain, provide insight into the relevance of modular structure with time scale hierarchy to information processing, and elucidate the significance of brain areas with slower time scales. Bayesian inference is essential for predicting noisy inputs in the environment and is suggested to be common in various animals, including humans. For the brain, to perform Bayesian inference, the prior distribution of the signal must be acquired and represented in the neural networks by sampling noisy inputs to estimate the posterior distribution of signals. By training recurrent neural networks to predict time-varying inputs, we demonstrated that those with modular structures, characterized by the main module exhibiting faster neural activity and the sub-module exhibiting slower neural activity, achieved highly accurate Bayesian inference to perform the required task. In this network, the prior distribution was specifically represented by the slower sub-module, which effectively integrated the earlier inputs. Furthermore, this modular structure with different time scales and division of representing roles emerged spontaneously through the learning process of Bayesian inference. Our results demonstrate a general mechanism for encoding prior distributions and highlight the importance of the brain's modular structure with time scale differentiation for Bayesian information processing.

Implementation of Bayesian Inference in Distributed Neural Networks

Distributed Sampling-based Bayesian Inference in Coupled Neural Circuits

Hierarchical Bayesian Inference and Learning in Spiking Neural Networks.

Efficient Inference on a Network of Spiking Neurons using Deep Learning

Inferring Inference

Practical Bayesian Inference in Neuroscience: Or How I Learned To Stop Worrying and Embrace the Distribution

Accelerated Physical Emulation of Bayesian Inference in Spiking Neural Networks

Sampling-based Bayesian Inference in Recurrent Circuits of Stochastic Spiking Neurons

Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal Circuits

Bayesian Spike Train Inference via Non-Local Priors

Bayesian Inference Accelerator for Spiking Neural Networks

Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation

Bayesian inference is facilitated by modular neural networks with different time scales

A Full Bayesian Approach to Sparse Network Inference Using Heterogeneous Datasets

Inference on the Macroscopic Dynamics of Spiking Neurons

Distilling Multi-Scale Neural Mechanisms from Diverse Unlabeled Experimental Data Using Deep Domain-Adaptive Inference Framework

Models and simulation of 3D neuronal dendritic trees using Bayesian networks

Unification of MAP Estimation and Marginal Inference in Recurrent Neural Networks

An FPGA implementation of Bayesian inference with spiking neural networks

Neural Network Implementation of Inference on Binary Markov Random Fields with Probability Coding

Efficient Bayes Inference in Neural Networks through Adaptive Importance Sampling