Abstract:Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles of computational models in neuroscience. Here we examined the geometry of DNN models of visual cortex by quantifying the latent dimensionality of their natural image representations. A popular view holds that optimal DNNs compress their representations onto low-dimensional subspaces to achieve invariance and robustness, which suggests that better models of visual cortex should have lower dimensional geometries. Surprisingly, we found a strong trend in the opposite direction—neural networks with high-dimensional image subspaces tended to have better generalization performance when predicting cortical responses to held-out stimuli in both monkey electrophysiology and human fMRI data. Moreover, we found that high dimensionality was associated with better performance when learning new categories of stimuli, suggesting that higher dimensional representations are better suited to generalize beyond their training domains. These findings suggest a general principle whereby high-dimensional geometry confers computational benefits to DNN models of visual cortex. The effective dimensionality of neural population codes in both brains and artificial neural networks can be far smaller than the number of neurons in the population. In vision, it has been argued that there are crucial benefits of representing images using codes that are as simple and low-dimensional as possible, allowing representations to emphasize key semantic content and attenuate irrelevant perceptual details. However, there are competing benefits of high-dimensional codes, which can capture rich perceptual information to better support an open-ended set of visual behaviors. We quantified the effective dimensionality of neural networks created using a diverse set of training paradigms and compared these networks with image-evoked codes in the visual cortex of both monkeys and humans. Our findings revealed striking benefits of networks with higher effective dimensionality, which were consistently better predictors of cortical activity patterns and were able to readily learn new categories of images outside of their training domains. Together, these findings demonstrate the computational benefits of high-dimensional sensory codes, and they suggest that current neural network models of visual cortex may be better understood in terms of the richness of their representations rather than the details of their training tasks.

Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity

Simultaneous, cortex-wide and cellular-resolution neuronal population dynamics reveal an unbounded scaling of dimensionality with neuron number

A Computational Approach to Interpreting the Embedding Space of Dimension Reduction

Analyzing neural responses to natural signals: Maximally informative dimensions

Identifying Interpretable Visual Features in Artificial and Biological Neural Systems

Revealing the Dynamics of Neural Information Processing with Multivariate Information Decomposition

Inferring Inference

A unified neural representation model for spatial and semantic computations

Subspace Projection Approaches To Classification And Visualization Of Neural Network-Level Encoding Patterns

Spatial embedding promotes a specific form of modularity with low entropy and heterogeneous spectral dynamics

Comparing high-dimensional neural recordings by aligning their low-dimensional latent representations

Dimensionally-reduced visual cortical network model predicts network response and connects system- and cellular-level descriptions

From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction

A Geometric Framework for Understanding Dynamic Information Integration in Context-Dependent Computation

Local vs distributed representations: What is the right basis for interpretability?

High-performing neural network models of visual cortex benefit from high latent dimensionality

Message-Relevant Dimension Reduction of Neural Populations

Information Processing by Neuron Populations in the Central Nervous System: Mathematical Structure of Data and Operations

Interpreting Deep Neural Networks Through Variable Importance

Dimensionality and Ramping: Signatures of Sentence Integration in the Dynamics of Brains and Deep Language Models

Intrinsic Dimension Correlation: uncovering nonlinear connections in multimodal representations