Abstract:The efficient coding approach proposes that neural systems represent as much sensory information as biological constraints allow. It aims at formalizing encoding as a constrained optimal process. A different approach, that aims at formalizing decoding, proposes that neural systems instantiate a generative model of the sensory world. Here, we put forth a normative framework that characterizes neural systems as jointly optimizing encoding and decoding. It takes the form of a variational autoencoder: sensory stimuli are encoded in the noisy activity of neurons to be interpreted by a flexible decoder; encoding must allow for an accurate stimulus reconstruction from neural activity. Jointly, neural activity is required to represent the statistics of latent features which are mapped by the decoder into distributions over sensory stimuli; decoding correspondingly optimizes the accuracy of the generative model. This framework yields in a family of encoding-decoding models, which result in equally accurate generative models, indexed by a measure of the stimulus-induced deviation of neural activity from the marginal distribution over neural activity. Each member of this family predicts a specific relation between properties of the sensory neurons—such as the arrangement of the tuning curve means (preferred stimuli) and widths (degrees of selectivity) in the population—as a function of the statistics of the sensory world. Our approach thus generalizes the efficient coding approach. Notably, here, the form of the constraint on the optimization derives from the requirement of an accurate generative model, while it is arbitrary in efficient coding models. Moreover, solutions do not require the knowledge of the stimulus distribution, but are learned on the basis of data samples; the constraint further acts as regularizer, allowing the model to generalize beyond the training data. Finally, we characterize the family of models we obtain through alternate measures of performance, such as the error in stimulus reconstruction. We find that a range of models admits comparable performance; in particular, a population of sensory neurons with broad tuning curves as observed experimentally yields both low reconstruction stimulus error and an accurate generative model that generalizes robustly to unseen data.

Jointly efficient encoding and decoding in neural populations

Efficiency turns the table on neural encoding, decoding and noise

Optimal Encoding in Stochastic Latent-Variable Models

Stimulus-dependent maximum entropy models of neural population codes

Robust Transcoding Sensory Information with Neural Spikes

Bayesian Efficient Coding

Neural stochastic codes, encoding and decoding

Neural and perceptual signatures of efficient sensory coding

Efficient population coding of sensory stimuli

Revealing nonlinear neural decoding by analyzing choices

Applying the Efficient Coding Principle to Understand Encoding of Multisensory and Multimodality Sensory Signals

Efficient coding in biophysically realistic excitatory-inhibitory spiking networks

Evolving Efficient Genetic Encoding for Deep Spiking Neural Networks

Population Coding and Decoding in a Neural Field: A Computational Study

Encoding and Decoding Models in Cognitive Electrophysiology

A Unified Theory of Response Sparsity and Variability for Energy-Efficient Neural Coding

Adaptive compressed sensing - a new class of self-organizing coding models for neuroscience

Interpreting Encoding and Decoding Models

Efficient Coding As the Provenance of Matched and Opposite Neuronal Feature Preferences for Multisensory and Multi-Modal Inputs

Adaptive coding efficiency in recurrent cortical circuits via gain control

Multiple Timescale Online Learning Rules for Information Maximization with Energetic Constraints.