Efficient Coding As the Provenance of Matched and Opposite Neuronal Feature Preferences for Multisensory and Multi-Modal Inputs
Li Zhaoping
DOI: https://doi.org/10.1167/17.10.599
2017-01-01
Journal of Vision
Abstract:When inputs from different sources are correlated, coding them efficiently (Barlow 1961) requires new representations (also called bases) in which the signals are decorrelated. For two sources, this implies two bases involving the respective weighted sum and difference of the inputs. Efficient coding explains many neural receptive field properties in early vision. For example, in stereo coding (Li & Atick 1994), some V1 neurons prefer the sum of inputs from the two eyes and other V1 neurons prefer the difference of these inputs. Input correlation also appears deeper in the brain when multisensory inputs or different unisensory cues converge. For example, medial superior temporal (MST) cortical neurons sense heading direction of self-motion based on optic flow and vestibular inputs; middle temporal (MT) cortical neurons sense depth from binocular disparity and motion parallax. Analogous to stereo, efficient coding predicts that the preferred features (heading direction or depth) from different sources should be matched in some neurons and opposite in others, as indeed is found in MST (Gu, Angelaki, DeAngelis 2008) and MT (Nadler et al 2013). Efficient coding thus accounts for the existence of opposite neurons, which appear useless for cue integration, and instead convey information missed by the matched neurons when input sources are only partially redundant. It predicts how the exact forms (i.e., relative weighting of the sources) of, and neural sensitivities to, individual bases, manifested by the matched and opposite neurons, should adapt to the statistical properties of the inputs (e.g., the correlation between the sources and signal to noise ratios). Generalization to more than two sensory modalities, e.g., vision, audition, and touch, and/or multiple unisensory cues is straightforward. For example, coding of triple-source inputs should be analogous to efficient coding of inputs from red, green, and blue cones (Atick, Li, Redlich 1992) to give three decorrelated bases. Meeting abstract presented at VSS 2017