Abstract:Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles of computational models in neuroscience. Here we examined the geometry of DNN models of visual cortex by quantifying the latent dimensionality of their natural image representations. A popular view holds that optimal DNNs compress their representations onto low-dimensional subspaces to achieve invariance and robustness, which suggests that better models of visual cortex should have lower dimensional geometries. Surprisingly, we found a strong trend in the opposite direction—neural networks with high-dimensional image subspaces tended to have better generalization performance when predicting cortical responses to held-out stimuli in both monkey electrophysiology and human fMRI data. Moreover, we found that high dimensionality was associated with better performance when learning new categories of stimuli, suggesting that higher dimensional representations are better suited to generalize beyond their training domains. These findings suggest a general principle whereby high-dimensional geometry confers computational benefits to DNN models of visual cortex. The effective dimensionality of neural population codes in both brains and artificial neural networks can be far smaller than the number of neurons in the population. In vision, it has been argued that there are crucial benefits of representing images using codes that are as simple and low-dimensional as possible, allowing representations to emphasize key semantic content and attenuate irrelevant perceptual details. However, there are competing benefits of high-dimensional codes, which can capture rich perceptual information to better support an open-ended set of visual behaviors. We quantified the effective dimensionality of neural networks created using a diverse set of training paradigms and compared these networks with image-evoked codes in the visual cortex of both monkeys and humans. Our findings revealed striking benefits of networks with higher effective dimensionality, which were consistently better predictors of cortical activity patterns and were able to readily learn new categories of images outside of their training domains. Together, these findings demonstrate the computational benefits of high-dimensional sensory codes, and they suggest that current neural network models of visual cortex may be better understood in terms of the richness of their representations rather than the details of their training tasks.

A Normative Theory of Adaptive Dimensionality Reduction in Neural Networks

Neural Network Emulation of Spatio-temporal Data Using Linear and Nonlinear Dimensionality Reduction

Adaptive dimensionality reduction for neural network-based online principal component analysis

Advances in adaptive nonlinear manifolds and dimensionality reduction

A biological model of nonlinear dimensionality reduction

Normative framework for deriving neural networks with multi-compartmental neurons and non-Hebbian plasticity

Analyzing neural responses to natural signals: Maximally informative dimensions

Why do similarity matching objectives lead to Hebbian/anti-Hebbian networks?

Biologically Plausible Online Principal Component Analysis Without Recurrent Neural Dynamics

High-performing neural network models of visual cortex benefit from high latent dimensionality

Model Order Reduction in Neuroscience

Locally Adaptive Nonlinear Dimensionality Reduction

More or fewer latent variables in the high-dimensional data space? That is the question

Low-dimensional Intrinsic Dimension Reveals a Phase Transition in Gradient-Based Learning of Deep Neural Networks

Nonlinear Principal Manifolds --- Adaptive Hybrid Learning Approaches

Dimensionality Reduction and Data Visualisation

Neural scaling laws for an uncertain world

Simultaneous Dimensionality Reduction for Extracting Useful Representations of Large Empirical Multimodal Datasets

A simple normative network approximates local non-Hebbian learning in the cortex

Deep Nonlinear Sufficient Dimension Reduction

Dimensionality reduction for evolving neural network