Abstract:Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles of computational models in neuroscience. Here we examined the geometry of DNN models of visual cortex by quantifying the latent dimensionality of their natural image representations. A popular view holds that optimal DNNs compress their representations onto low-dimensional subspaces to achieve invariance and robustness, which suggests that better models of visual cortex should have lower dimensional geometries. Surprisingly, we found a strong trend in the opposite direction—neural networks with high-dimensional image subspaces tended to have better generalization performance when predicting cortical responses to held-out stimuli in both monkey electrophysiology and human fMRI data. Moreover, we found that high dimensionality was associated with better performance when learning new categories of stimuli, suggesting that higher dimensional representations are better suited to generalize beyond their training domains. These findings suggest a general principle whereby high-dimensional geometry confers computational benefits to DNN models of visual cortex. The effective dimensionality of neural population codes in both brains and artificial neural networks can be far smaller than the number of neurons in the population. In vision, it has been argued that there are crucial benefits of representing images using codes that are as simple and low-dimensional as possible, allowing representations to emphasize key semantic content and attenuate irrelevant perceptual details. However, there are competing benefits of high-dimensional codes, which can capture rich perceptual information to better support an open-ended set of visual behaviors. We quantified the effective dimensionality of neural networks created using a diverse set of training paradigms and compared these networks with image-evoked codes in the visual cortex of both monkeys and humans. Our findings revealed striking benefits of networks with higher effective dimensionality, which were consistently better predictors of cortical activity patterns and were able to readily learn new categories of images outside of their training domains. Together, these findings demonstrate the computational benefits of high-dimensional sensory codes, and they suggest that current neural network models of visual cortex may be better understood in terms of the richness of their representations rather than the details of their training tasks.

High--Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

The unreasonable effectiveness of small neural ensembles in high-dimensional brain

Statistical Inference from High Dimensional Data

High-performing neural network models of visual cortex benefit from high latent dimensionality

Digital medicine and the curse of dimensionality

Dimensionality Reduction Methods for Brain Imaging Data Analysis

High dimensionality: The latest challenge to data analysis

Interpreting the Curse of Dimensionality from Distance Concentration and Manifold Effect

The Curse of Instability

Blessing of dimensionality at the edge

Dimensionality Reduction and Data Visualisation

A Theoretical Perspective on Hyperdimensional Computing

More or fewer latent variables in the high-dimensional data space? That is the question

How I learned to stop worrying and love the curse of dimensionality: an appraisal of cluster validation in high-dimensional spaces

Dimensionality's Blessing: Clustering Images by Underlying Distribution.

A Relational Inductive Bias for Dimensional Abstraction in Neural Networks

Feature dimensionality reduction: a review

Breaking the Curse of Dimensionality with Isolation Kernel

High-dimensional learning of narrow neural networks

Linear Dimensionality Reduction: Survey, Insights, and Generalizations

High-dimensional brain. A tool for encoding and rapid learning of memories by single neurons