Abstract:Neuroscience research has extensively explored the commonality of neural representations of sensory stimuli across individuals to uncover universal neural mechanisms in the encoding of sensory information. To compare neural representations across different brains, Representational Similarity Analysis (RSA) has been used, which focuses on the similarity structures of neural representations for different stimuli. Despite the broad applicability and utility of RSA, one limitation is that its conventional framework assumes that neural representations of particular stimuli correspond directly to those of the same stimuli in different brains. This assumption excludes the possibility that neural representations correspond differently and limits the exploration of finer structural similarities. To overcome this limitation, we propose to use an unsupervised alignment framework based on Gromov-Wasserstein Optimal Transport (GWOT) to compare similarity structures without presupposing stimulus correspondences. This method allows for the identification of optimal correspondence between neural representations of stimuli based solely on internal neural representation relationships, and thereby provides a more detailed comparison of neural similarity structures across individuals. We applied this unsupervised alignment to investigate the commonality of representational similarity structures of natural scenes, using large datasets of Neuropixels recordings in mice and fMRI recordings in humans. We found that the similarity structure of neural representations in the same visual cortical areas can be well aligned across individuals in an unsupervised manner in both mice and humans. In contrast, we found that the degree of alignment across different brain areas cannot be fully explained by proximity in the visual processing hierarchy alone, but also found some reasonable alignment results, such that the similarity structures of higher-order visual areas can be well aligned with each other but not with lower-order visual areas. We expect that our unsupervised approach will be useful for revealing more detailed structural commonalities or differences that may not be captured by the conventional supervised approach.

A Guide to Representational Similarity Analysis for Social Neuroscience.

Representational similarity analysis in neuroimaging: proxy vehicles and provisional representations

The Topology of Representational Geometry

Higher-order Comparisons of Sentence Encoder Representations

Topological Representational Similarity Analysis in Brains and Beyond

Deep Representational Similarity Learning for analyzing neural signatures in task-based fMRI dataset

Gradient-based Representational Similarity Analysis with Searchlight for Analyzing fMRI Data

Neural coding of cognitive control: The representational similarity analysis approach

Using Single-Trial Representational Similarity Analysis with EEG to track semantic similarity in emotional word processing

The Topology and Geometry of Neural Representations

Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis

Establishing the relationship between subjective perception and neural responses: Insights from Correlation Analysis and Representational Similarity Analysis

Blackbox meets blackbox: Representational Similarity and Stability Analysis of Neural Language Models and Brains

Representational dissimilarity component analysis (ReDisCA)

The unreliable influence of multivariate noise normalization on the reliability of neural dissimilarity

A Representational Similarity Analysis of the Dynamics of Object Processing Using Single-Trial EEG Classification

Evaluating Representational Similarity Measures from the Lens of Functional Correspondence

Unsupervised alignment reveals structural commonalities and differences in neural representations of natural scenes across individuals and brain areas

Statistical inference on representational geometries

What Representational Similarity Measures Imply about Decodable Information

Duality of Bures and Shape Distances with Implications for Comparing Neural Representations