Abstract:Neuroscience research has extensively explored the commonality of neural representations of sensory stimuli across individuals to uncover universal neural mechanisms in the encoding of sensory information. To compare neural representations across different brains, Representational Similarity Analysis (RSA) has been used, which focuses on the similarity structures of neural representations for different stimuli. Despite the broad applicability and utility of RSA, one limitation is that its conventional framework assumes that neural representations of particular stimuli correspond directly to those of the same stimuli in different brains. This assumption excludes the possibility that neural representations correspond differently and limits the exploration of finer structural similarities. To overcome this limitation, we propose to use an unsupervised alignment framework based on Gromov-Wasserstein Optimal Transport (GWOT) to compare similarity structures without presupposing stimulus correspondences. This method allows for the identification of optimal correspondence between neural representations of stimuli based solely on internal neural representation relationships, and thereby provides a more detailed comparison of neural similarity structures across individuals. We applied this unsupervised alignment to investigate the commonality of representational similarity structures of natural scenes, using large datasets of Neuropixels recordings in mice and fMRI recordings in humans. We found that the similarity structure of neural representations in the same visual cortical areas can be well aligned across individuals in an unsupervised manner in both mice and humans. In contrast, we found that the degree of alignment across different brain areas cannot be fully explained by proximity in the visual processing hierarchy alone, but also found some reasonable alignment results, such that the similarity structures of higher-order visual areas can be well aligned with each other but not with lower-order visual areas. We expect that our unsupervised approach will be useful for revealing more detailed structural commonalities or differences that may not be captured by the conventional supervised approach.

Higher-order Comparisons of Sentence Encoder Representations

A Guide to Representational Similarity Analysis for Social Neuroscience.

Turing Representational Similarity Analysis (RSA): A Flexible Method for Measuring Alignment Between Human and Artificial Intelligence

Deep Representational Similarity Learning for analyzing neural signatures in task-based fMRI dataset

Using Single-Trial Representational Similarity Analysis with EEG to track semantic similarity in emotional word processing

Representational similarity analysis in neuroimaging: proxy vehicles and provisional representations

The Topology of Representational Geometry

Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains

Gradient-based Representational Similarity Analysis with Searchlight for Analyzing fMRI Data

Neural coding of cognitive control: The representational similarity analysis approach

Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis

Topological Representational Similarity Analysis in Brains and Beyond

Unsupervised alignment reveals structural commonalities and differences in neural representations of natural scenes across individuals and brain areas

What Representational Similarity Measures Imply about Decodable Information

Towards a Fully Interpretable and More Scalable RSA Model for Metaphor Understanding

Representational dissimilarity component analysis (ReDisCA)

A Representational Similarity Analysis of the Dynamics of Object Processing Using Single-Trial EEG Classification

The Topology and Geometry of Neural Representations

Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals

Unraveling lexical semantics in the brain: Comparing internal, external, and hybrid language models

Evaluating Representational Similarity Measures from the Lens of Functional Correspondence