Abstract:Two sensory neurons usually display trial-by-trial spike-count correlations given the repeated representations of a stimulus. The effects of such response correlations on population-level sensory coding have been the focal contention in computational neuroscience over the past few years. In the meantime, multivariate pattern analysis (MVPA) has become the leading analysis approach in functional magnetic resonance imaging (fMRI), but the effects of response correlations among voxel populations remain underexplored. Here, instead of conventional MVPA analysis, we calculate linear Fisher information of population responses in human visual cortex (five males, one female) and hypothetically remove response correlations between voxels. We found that voxelwise response correlations generally enhance stimulus information, a result standing in stark contrast to the detrimental effects of response correlations reported in empirical neurophysiological studies. By voxel-encoding modeling, we further show that these two seemingly opposite effects actually can coexist within the primate visual system. Furthermore, we use principal component analysis to decompose stimulus information in population responses onto different principal dimensions in a high-dimensional representational space. Interestingly, response correlations simultaneously reduce and enhance information on higher- and lower-variance principal dimensions, respectively. The relative strength of the two antagonistic effects within the same computational framework produces the apparent discrepancy in the effects of response correlations in neuronal and voxel populations. Our results suggest that multivariate fMRI data contain rich statistical structures that are directly related to sensory information representation, and the general computational framework to analyze neuronal and voxel population responses can be applied in many types of neural measurements. SIGNIFICANCE STATEMENT Despite the vast research interest in the effect of spike-count noise correlations on population codes in neurophysiology, it remains unclear how the response correlations between voxels influence MVPA in human imaging. We used an information-theoretic approach and showed that unlike the detrimental effects of response correlations reported in neurophysiology, voxelwise response correlations generally improve sensory coding. We conducted a series of in-depth analyses and demonstrated that neuronal and voxel response correlations can coexist within the visual system and share some common computational mechanisms. These results shed new light on how the population codes of sensory information can be evaluated via different neural measurements.

Privileged representational axes in biological and artificial neural networks

On Privileged and Convergent Bases in Neural Network Representations

Identifying Interpretable Visual Features in Artificial and Biological Neural Systems

Neural tuning and representational geometry

High-performing neural network models of visual cortex benefit from high latent dimensionality

Rarely categorical, always high-dimensional: how the neural code changes along the cortical hierarchy

Applying the Efficient Coding Principle to Understand Encoding of Multisensory and Multimodality Sensory Signals

Efficient Coding As the Provenance of Matched and Opposite Neuronal Feature Preferences for Multisensory and Multi-Modal Inputs

Revealing nonlinear neural decoding by analyzing choices

Top-down feedback matters: Functional impact of brainlike connectivity motifs on audiovisual integration

Don't Cut Corners: Exact Conditions for Modularity in Biologically Inspired Representations

Transformations of sensory information in the brain suggest changing criteria for optimality

Representational Geometries Reveal Differential Effects of Response Correlations on Population Codes in Neurophysiology and Functional Magnetic Resonance Imaging

Hierarchical Sparse Coding of Objects in Deep Convolutional Neural Networks

Different Spectral Representations in Optimized Artificial Neural Networks and Brains

Bayesian encoding and decoding as distinct perspectives on neural coding

Neural Elements for Predictive Coding

Emergence of Sparse Coding, Balance and Decorrelation from a Biologically-Grounded Spiking Neural Network Model of Learning in the Primary Visual Cortex

Parsimony, exhaustivity and balanced detection in neocortex

Comparison Against Task Driven Artificial Neural Networks Reveals Functional Organization of Mouse Visual Cortex

Subspace Projection Approaches To Classification And Visualization Of Neural Network-Level Encoding Patterns