Abstract:Neuroscience research has extensively explored the commonality of neural representations of sensory stimuli across individuals to uncover universal neural mechanisms in the encoding of sensory information. To compare neural representations across different brains, Representational Similarity Analysis (RSA) has been used, which focuses on the similarity structures of neural representations for different stimuli. Despite the broad applicability and utility of RSA, one limitation is that its conventional framework assumes that neural representations of particular stimuli correspond directly to those of the same stimuli in different brains. This assumption excludes the possibility that neural representations correspond differently and limits the exploration of finer structural similarities. To overcome this limitation, we propose to use an unsupervised alignment framework based on Gromov-Wasserstein Optimal Transport (GWOT) to compare similarity structures without presupposing stimulus correspondences. This method allows for the identification of optimal correspondence between neural representations of stimuli based solely on internal neural representation relationships, and thereby provides a more detailed comparison of neural similarity structures across individuals. We applied this unsupervised alignment to investigate the commonality of representational similarity structures of natural scenes, using large datasets of Neuropixels recordings in mice and fMRI recordings in humans. We found that the similarity structure of neural representations in the same visual cortical areas can be well aligned across individuals in an unsupervised manner in both mice and humans. In contrast, we found that the degree of alignment across different brain areas cannot be fully explained by proximity in the visual processing hierarchy alone, but also found some reasonable alignment results, such that the similarity structures of higher-order visual areas can be well aligned with each other but not with lower-order visual areas. We expect that our unsupervised approach will be useful for revealing more detailed structural commonalities or differences that may not be captured by the conventional supervised approach.

Exploring Geometric Representational Alignment through Ollivier-Ricci Curvature and Ricci Flow

Unsupervised alignment reveals structural commonalities and differences in neural representations of natural scenes across individuals and brain areas

Revisiting Over-smoothing and Over-squashing using Ollivier's Ricci Curvature

Generalized Shape Metrics on Neural Representations

Getting aligned on representational alignment

Dimensions underlying the representational alignment of deep neural networks with humans

Computational Analysis of Deformable Manifolds: from Geometric Modelling to Deep Learning

Discriminating image representations with principal distortions

A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding

Curvature-based Methods for Brain Network Analysis

Unveiling the human-like similarities of automatic facial expression recognition: An empirical exploration through explainable ai

Ricci flow-based brain surface covariance descriptors for diagnosing Alzheimer's disease

All Roads Lead to Rome? Exploring Representational Similarities Between Latent Spaces of Generative Image Models

Deep learning as Ricci flow

Geometric deep learning for galaxy-halo connection: a case study for galaxy intrinsic alignments

Beyond Geometry: Comparing the Temporal Structure of Computation in Neural Circuits with Dynamical Similarity Analysis

Human-Like Geometric Abstraction in Large Pre-trained Neural Networks

One Hundred Neural Networks and Brains Watching Videos: Lessons from Alignment