Abstract:The efficient coding approach proposes that neural systems represent as much sensory information as biological constraints allow. It aims at formalizing encoding as a constrained optimal process. A different approach, that aims at formalizing decoding, proposes that neural systems instantiate a generative model of the sensory world. Here, we put forth a normative framework that characterizes neural systems as jointly optimizing encoding and decoding. It takes the form of a variational autoencoder: sensory stimuli are encoded in the noisy activity of neurons to be interpreted by a flexible decoder; encoding must allow for an accurate stimulus reconstruction from neural activity. Jointly, neural activity is required to represent the statistics of latent features which are mapped by the decoder into distributions over sensory stimuli; decoding correspondingly optimizes the accuracy of the generative model. This framework yields in a family of encoding-decoding models, which result in equally accurate generative models, indexed by a measure of the stimulus-induced deviation of neural activity from the marginal distribution over neural activity. Each member of this family predicts a specific relation between properties of the sensory neurons—such as the arrangement of the tuning curve means (preferred stimuli) and widths (degrees of selectivity) in the population—as a function of the statistics of the sensory world. Our approach thus generalizes the efficient coding approach. Notably, here, the form of the constraint on the optimization derives from the requirement of an accurate generative model, while it is arbitrary in efficient coding models. Moreover, solutions do not require the knowledge of the stimulus distribution, but are learned on the basis of data samples; the constraint further acts as regularizer, allowing the model to generalize beyond the training data. Finally, we characterize the family of models we obtain through alternate measures of performance, such as the error in stimulus reconstruction. We find that a range of models admits comparable performance; in particular, a population of sensory neurons with broad tuning curves as observed experimentally yields both low reconstruction stimulus error and an accurate generative model that generalizes robustly to unseen data.

Efficiency turns the table on neural encoding, decoding and noise

Jointly efficient encoding and decoding in neural populations

How Do Efficient Coding Strategies Depend on Origins of Noise in Neural Circuits?

Neural and perceptual signatures of efficient sensory coding

Efficient coding explains neural response homeostasis and stimulus-specific adaptation

Optimal Encoding in Stochastic Latent-Variable Models

Bayesian Efficient Coding

Efficient population coding of sensory stimuli

Neural stochastic codes, encoding and decoding

Revealing nonlinear neural decoding by analyzing choices

Efficient representation as a design principle for neural coding and computation

Efficient coding in biophysically realistic excitatory-inhibitory spiking networks

Energy‐efficient Neural Information Processing in Individual Neurons and Neuronal Networks

On the condition for fast neural computation

Energy Optimization Induces Predictive-coding Properties in a Multicompartment Spiking Neural Network Model

Stimulus-dependent maximum entropy models of neural population codes

Applying the Efficient Coding Principle to Understand Encoding of Multisensory and Multimodality Sensory Signals

MSE-based analysis of optimal tuning functions predicts phenomena observed in sensory neurons

Evaluating performance of neural codes in model neural communication networks

Population Rate Coding in Recurrent Neuronal Networks with Unreliable Synapses

Turn Down that Noise: Synaptic Encoding of Afferent SNR in a Single Spiking Neuron