Abstract:Abstract Massively generated single-cell multi-omics datasets are revolutionizing biological studies of heterogenous tissues and organisms, which necessitate powerful computational methods to unleash the full potential of these tremendous data. Here, we present Concerto, stands for self-distillation contrastive learning of cell representations, a self-supervised representation learning framework optimized with asymmetric teacher-student configuration to analyze single-cell multi-omics datasets with scalability up to building 10 million-cell reference within 1.5 hour and querying 10k cells within 8 seconds. Concerto leverages dropout layer as minimal data augmentation to learn meaningful cell representations in a contrastive manner. The teacher module uses attention mechanism to aggregate contextualized gene embeddings within cellular context, while the student module uses simpler dense structure with discreate input. The learned task-agnostic representations can be adapted to a broad range of single-cell computation tasks. 1) Via supervised fine-tuning, Concerto enables automatic cell classification as well as novel cell-type discovery; 2) Attention weights provide model interpretability via automatically extracting specific molecular signatures at single-cell resolution without the needs of clustering; 3) Via source-aware training, Concerto supports efficient data integration by projecting all cells across multiple batches into a joint embedding space. 4) Via batch-aware inference or unsupervised fine-tuning, Concerto enables mapping query cells onto reference and accurately transferring annotations. Concerto can flexibly extend to multi-omics datasets simply through cross-modality summation operation to obtain unified cell embeddings. Using examples from human peripheral blood, human thymus, human pancreas, and mouse tissue atlas, Concerto shows superior performance benchmarking against other top-performing methods. We also demonstrate Concerto recapitulates detailed COVID-19 disease variation through query-to-reference mapping. Concerto can operate on all genes and represents a fully data-driven approach with minimum prior distribution assumptions, eliminating the needs of PCA-like or autoencoder-like dimensionality reduction, which significantly reforms the current best practice. Concerto is a simple, straightforward, robust, and scalable framework, offering a brand new perspective to derive cell representations and can effectively satisfy the emerging paradigm of query-to-reference mapping in the era of atlas-level single-cell multimodal analysis.

Self-distillation Contrastive Learning Enables Clustering-Free Signature Extraction and Mapping to Multimodal Single-Cell Atlas of Multimillion Scale

Contrastive learning enables rapid mapping to multimodal single-cell atlas of multimillion scale

Learning Consistent Subcellular Landmarks to Quantify Changes in Multiplexed Protein Maps.

Learning Consistency and Specificity of Cells from Single-cell Multi-omic Data

CellContrast: Reconstructing spatial relationships in single-cell RNA sequencing data via deep contrastive learning

A sandbox for prediction and integration of DNA, RNA, and proteins in single cells

Integrating T-cell Receptor and Transcriptome for Large-Scale Single-Cell Immune Profiling Analysis

Contrastive Learning for Robust Cell Annotation and Representation from Single-Cell Transcriptomics

CAKE: a flexible self-supervised framework for enhancing cell visualization, clustering and rare cell identification

CellMemory: Hierarchical Interpretation of Out-of-Distribution Cells Using Bottlenecked Transformer

Con-AAE: contrastive cycle adversarial autoencoders for single-cell multi-omics alignment and integration

Ensemble deep learning of embeddings for clustering multimodal single-cell omics data

Multi-batch single-cell comparative atlas construction by deep learning disentanglement

Capturing cell heterogeneity in representations of cell populations for image-based profiling using contrastive learning

Online Single-Cell Data Integration Through Projecting Heterogeneous Datasets into a Common Cell-Embedding Space

Efficient and precise single-cell reference atlas mapping with Symphony

Graph Contrastive Learning as a Versatile Foundation for Advanced scRNA-seq Data Analysis

DEMOC: a deep embedded multi-omics learning approach for clustering single-cell CITE-seq data

ScCCL: Single-Cell Data Clustering Based on Self-Supervised Contrastive Learning

Accurate, Scalable and Cross-platform Cell Identification for High-resolution Spatial Transcriptomics