Abstract:Modern applications often leverage multiple views of a subject of study. Within neuroscience, there is growing interest in large-scale simultaneous recordings across multiple brain regions. Understanding the relationship between views (e.g., the neural activity in each region recorded) can reveal fundamental principles about the characteristics of each representation and about the system. However, existing methods to characterize such relationships either lack the expressivity required to capture complex nonlinearities, describe only sources of variance that are shared between views, or discard geometric information that is crucial to interpreting the data. Here, we develop a nonlinear neural network-based method that, given paired samples of high-dimensional views, disentangles low-dimensional shared and private latent variables underlying these views while preserving intrinsic data geometry. Across multiple simulated and real datasets, we demonstrate that our method outperforms competing methods. Using simulated populations of lateral geniculate nucleus (LGN) and V1 neurons we demonstrate our model's ability to discover interpretable shared and private structure across different noise conditions. On a dataset of unrotated and corresponding but randomly rotated MNIST digits, we recover private latents for the rotated view that encode rotation angle regardless of digit class, and places the angle representation on a 1-d manifold, while shared latents encode digit class but not rotation angle. Applying our method to simultaneous Neuropixels recordings of hippocampus and prefrontal cortex while mice run on a linear track, we discover a low-dimensional shared latent space that encodes the animal's position. We propose our approach as a general-purpose method for finding succinct and interpretable descriptions of paired data sets in terms of disentangled shared and private latent variables.

Unsupervised discovery of the shared and private geometry in multi-view data

View-Aware Geometry-Structure Joint Learning for Single-View 3D Shape Reconstruction

A Deep Network for Explainable Prediction of Non-Imaging Phenotypes using Anatomical Multi-View Data

Deep Co-Attention Network for Multi-View Subspace Learning

Discovering Common Information in Multi-view Data

Characterization of Hemodynamic Signal by Learning Multi-View Relationships

Interpretable statistical representations of neural population dynamics and geometry

Seeing Unseen: Discover Novel Biomedical Concepts via Geometry-Constrained Probabilistic Modeling

Multi-view data visualisation via manifold learning

Multiview Clustering by Joint Latent Representation and Similarity Learning

Shared Subspace Learning for Latent Representation of Multi-View Data.

Seeking Common Ground While Reserving Differences: Multiple Anatomy Collaborative Framework for Undersampled MRI Reconstruction

Equivariant Multi-View Networks

See without looking: joint visualization of sensitive multi-site datasets

Subspace Projection Approaches To Classification And Visualization Of Neural Network-Level Encoding Patterns

Shared Generative Latent Representation Learning for Multi-View Clustering.

Multi-view anomaly detection via hybrid instance-neighborhood aligning and cross-view reasoning

Infant Brain Development Prediction with Latent Partial Multi-View Representation Learning.

Multi-view embedding learning via robust joint nonnegative matrix factorization

Latent Simplex Position Model: High Dimensional Multi-view Clustering with Uncertainty Quantification