Abstract:The efficient coding approach proposes that neural systems represent as much sensory information as biological constraints allow. It aims at formalizing encoding as a constrained optimal process. A different approach, that aims at formalizing decoding, proposes that neural systems instantiate a generative model of the sensory world. Here, we put forth a normative framework that characterizes neural systems as jointly optimizing encoding and decoding. It takes the form of a variational autoencoder: sensory stimuli are encoded in the noisy activity of neurons to be interpreted by a flexible decoder; encoding must allow for an accurate stimulus reconstruction from neural activity. Jointly, neural activity is required to represent the statistics of latent features which are mapped by the decoder into distributions over sensory stimuli; decoding correspondingly optimizes the accuracy of the generative model. This framework yields in a family of encoding-decoding models, which result in equally accurate generative models, indexed by a measure of the stimulus-induced deviation of neural activity from the marginal distribution over neural activity. Each member of this family predicts a specific relation between properties of the sensory neurons—such as the arrangement of the tuning curve means (preferred stimuli) and widths (degrees of selectivity) in the population—as a function of the statistics of the sensory world. Our approach thus generalizes the efficient coding approach. Notably, here, the form of the constraint on the optimization derives from the requirement of an accurate generative model, while it is arbitrary in efficient coding models. Moreover, solutions do not require the knowledge of the stimulus distribution, but are learned on the basis of data samples; the constraint further acts as regularizer, allowing the model to generalize beyond the training data. Finally, we characterize the family of models we obtain through alternate measures of performance, such as the error in stimulus reconstruction. We find that a range of models admits comparable performance; in particular, a population of sensory neurons with broad tuning curves as observed experimentally yields both low reconstruction stimulus error and an accurate generative model that generalizes robustly to unseen data.

Population Decoding Based on an Unfaithful Model.

Population Coding with Correlation and an Unfaithful Model

Unfaithful Population Decoding

Population Coding and Decoding in a Neural Field: A Computational Study

Neural Implementation of Bayesian Inference in Population Codes

Population Rate Coding in Recurrent Neuronal Networks with Unreliable Synapses

Neural Population Decoding in Short-Time Windows

Information processing in a neuron ensemble with the multiplicative correlation structure.

Sequential Bayesian Decoding with a Population of Neurons

Dynamics Analysis of a Population Decoding Model

Revealing nonlinear neural decoding by analyzing choices

Efficient Neural Decoding Based on Multimodal Training

Truth or Deceit? A Bayesian Decoding Game Enhances Consistency and Reliability

Stimulus-dependent maximum entropy models of neural population codes

Asymptotic Behaviors of Population Codes

Modeling correlated noise is necessary to decode uncertainty

Rethinking: Deep-learning-based Demodulation and Decoding

Jointly efficient encoding and decoding in neural populations

Exploiting correlations across trials and behavioral sessions to improve neural decoding

Machine Learning-Aided Efficient Decoding of Reed-Muller Subcodes

DeCoRe: Decoding by Contrasting Retrieval Heads to Mitigate Hallucinations