Abstract:When inputs from different sources are correlated, coding them efficiently (Barlow 1961) requires new representations (also called bases) in which the signals are decorrelated. For two sources, this implies two bases involving the respective weighted sum and difference of the inputs. Efficient coding explains many neural receptive field properties in early vision. For example, in stereo coding (Li & Atick 1994), some V1 neurons prefer the sum of inputs from the two eyes and other V1 neurons prefer the difference of these inputs. Input correlation also appears deeper in the brain when multisensory inputs or different unisensory cues converge. For example, medial superior temporal (MST) cortical neurons sense heading direction of self-motion based on optic flow and vestibular inputs; middle temporal (MT) cortical neurons sense depth from binocular disparity and motion parallax. Analogous to stereo, efficient coding predicts that the preferred features (heading direction or depth) from different sources should be matched in some neurons and opposite in others, as indeed is found in MST (Gu, Angelaki, DeAngelis 2008) and MT (Nadler et al 2013). Efficient coding thus accounts for the existence of opposite neurons, which appear useless for cue integration, and instead convey information missed by the matched neurons when input sources are only partially redundant. It predicts how the exact forms (i.e., relative weighting of the sources) of, and neural sensitivities to, individual bases, manifested by the matched and opposite neurons, should adapt to the statistical properties of the inputs (e.g., the correlation between the sources and signal to noise ratios). Generalization to more than two sensory modalities, e.g., vision, audition, and touch, and/or multiple unisensory cues is straightforward. For example, coding of triple-source inputs should be analogous to efficient coding of inputs from red, green, and blue cones (Atick, Li, Redlich 1992) to give three decorrelated bases. Meeting abstract presented at VSS 2017

Efficient Stereo Coding in the Multiscale Representation

From Binocular Input Correlation to Disparity Tuning and Ocular Dominance in Cortex

Efficient Large Scale Stereo Matching Based on Cross-Scale.

Effects of image statistics on stereo coding in human vision

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Applying the Efficient Coding Principle to Understand Encoding of Multisensory and Multimodality Sensory Signals

A Psychophysical Experiment to Test the Efficient Stereo Coding Theory

Modeling pre-attentive stereo grouping by intracortical interactions in early visual cortex

Efficient Coding As the Provenance of Matched and Opposite Neuronal Feature Preferences for Multisensory and Multi-Modal Inputs

A Theory of the Visual Motion Coding in the Primary Visual Cortex

Li and Atick’s Theory of Efficient Binocular Coding: A Tutorial and Mini-Review

Pre-attentive segmentation and correspondence in stereo.

A new scheme of layered coding of stereoscopic video

Pre–attentive Segmentation and Correspondence in Stereo

Disparity Channels in Early Vision.

Neural architectures for stereo vision

Fast Stereoscopic Frame Estimation and Interpolation Algorithm in Compressed Stereo Video Streams

Neural circuits for binocular vision: Ocular dominance, interocular matching, and disparity selectivity

Modeling stereopsis via Markov random field

Structure from stereo by associative learning of the constraints

Binocular Disparity: Neural Mechanisms and Perceptual Learning