Abstract:Patterns of microcircuitry suggest that the brain has an array of repeated canonical computational units. Yet neural representations are distributed, so the relevant computations may only be related indirectly to single-neuron transformations. It thus remains an open challenge how to define canonical distributed computations. We integrate normative and algorithmic theories of neural computation into a mathematical framework for inferring canonical distributed computations from large-scale neural activity patterns. At the normative level, we hypothesize that the brain creates a structured internal model of its environment, positing latent causes that explain its sensory inputs, and uses those sensory inputs to infer the latent causes. At the algorithmic level, we propose that this inference process is a nonlinear message-passing algorithm on a graph-structured model of the world. Given a time series of neural activity during a perceptual inference task, our framework finds (i) the neural representation of relevant latent variables, (ii) interactions between these variables that define the brain's internal model of the world, and (iii) message-functions specifying the inference algorithm. These targeted computational properties are then statistically distinguishable due to the symmetries inherent in any canonical computation, up to a global transformation. As a demonstration, we simulate recordings for a model brain that implicitly implements an approximate inference algorithm on a probabilistic graphical model. Given its external inputs and noisy neural activity, we recover the latent variables, their neural representation and dynamics, and canonical message-functions. We highlight features of experimental design needed to successfully extract canonical computations from neural data. Overall, this framework provides a new tool for discovering interpretable structure in neural recordings.

Winner-Take-All as Basic Probabilistic Inference Unit of Neuronal Circuits

Emergent Inference of Hidden Markov Models in Spiking Neural Networks Through Winner-Take-All.

Mechanisms Of Reward-Modulated Stdp And Winner-Take-All In Bayesian Spiking Decision-Making Circuit

Versatile networks of simulated spiking neurons displaying winner-take-all behavior

A probabilistic model for learning in cortical microcircuit motifs with data-based divisive inhibition

Implementation of Bayesian Inference in Distributed Neural Networks

Probabilistic Inference of Binary Markov Random Fields in Spiking Neural Networks through Mean-field Approximation

Winner-Takes-All Mechanism Realized By Memristive Neural Network

Sampling-based Bayesian Inference in Recurrent Circuits of Stochastic Spiking Neurons

On the Computational Power of Winner-Take-All

Distributed Sampling-based Bayesian Inference in Coupled Neural Circuits

A Model of Neural Circuit with Adaption and Nonlinear Connection

The most probable neural circuit exhibits low-dimensional sustained activity

Hierarchical Bayesian Inference and Learning in Spiking Neural Networks.

Neuromorphic Implementation of Attractor Dynamics in a Two-Variable Winner-Take-All Circuit with NMDARs: A Simulation Study

A Radically New Theory of how the Brain Represents and Computes with Probabilities

A Winner-Take-All Neural Networks of N Linear Threshold Neurons Without Self-Excitatory Connections

Inferring Inference

Toward stochastic neural computing

Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks

Neuronal message passing using Mean-field, Bethe, and Marginal approximations