Abstract:Histology imaging is an important tool in medical diagnosis and research, enabling the examination of tissue structure and composition at the microscopic level. Understanding the underlying molecular mechanisms of tissue architecture is critical in uncovering disease mechanisms and developing effective treatments. Gene expression profiling provides insight into the molecular processes underlying tissue architecture, but the process can be time-consuming and expensive. We present BLEEP (Bi-modaL Embedding for Expression Prediction), a bi-modal embedding framework capable of generating spatially resolved gene expression profiles of whole-slide Hematoxylin and eosin (H&E) stained histology images. BLEEP uses contrastive learning to construct a low-dimensional joint embedding space from a reference dataset using paired image and expression profiles at micrometer resolution. With this approach, the gene expression of any query image patch can be imputed using the expression profiles from the reference dataset. We demonstrate BLEEP's effectiveness in gene expression prediction by benchmarking its performance on a human liver tissue dataset captured using the 10x Visium platform, where it achieves significant improvements over existing methods. Our results demonstrate the potential of BLEEP to provide insights into the molecular mechanisms underlying tissue architecture, with important implications in diagnosis and research of various diseases. The proposed approach can significantly reduce the time and cost associated with gene expression profiling, opening up new avenues for high-throughput analysis of histology images for both research and clinical applications.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to predict spatially resolved gene expression profiles from H&E - stained tissue - section images. Specifically, existing methods face the following challenges in predicting gene expression: 1. **Ill - posedness of the problem**: Although H&E images share some information with paired spatial transcriptomics, image features may not be used to predict the expression of all genes, and vice versa. However, for marker genes (MG) of cell types or subtypes, highly expressed genes (HEG), and highly variable genes (HVG), their expressions should be preferentially predicted because these genes usually have important biological significance in disease diagnosis and drug development. 2. **The curse of dimensionality**: A typical mammalian cell expresses about 5,000 to 15,000 genes, while existing solutions usually predict only a limited number of genes (about 200), and often fail to preserve the variability and heterogeneity in the original dataset, which masks the biological signals in the original data and makes the prediction results ineffective in practical applications. 3. **Experimental artifacts**: Due to the development of spatial transcriptomics methods, existing datasets are easily affected by experimental artifacts within and between samples, which makes it more complicated to train expression prediction models. In addition, the measured gene expression is often inconsistent with the protein profile, which may be reflected during the H&E staining process. To solve these problems, the authors propose BLEEP (Bi - modaL Embedding for Expression Prediction), a dual - modal embedding framework based on contrastive learning. BLEEP accurately infers the gene expression of any query image patch by constructing a low - dimensional joint embedding space to align the paired image and expression features in the reference dataset. The specific methods are as follows: - **Contrastive learning**: Use the contrastive learning method to align the paired image and expression features into a low - dimensional joint embedding space. - **Query - reference imputation**: Infer the gene expression of the query image patch by querying the nearest - neighbor expression profiles in the reference dataset, thereby alleviating the curse of dimensionality problem. - **Robustness**: The expression prediction of BLEEP is robust to experimental artifacts within samples and batch - effect between different samples. Through experimental verification on the human liver tissue dataset, BLEEP significantly outperforms existing methods such as HisToGene and ST - Net in predicting the expressions of marker genes (MG), highly expressed genes (HEG), and highly variable genes (HVG). In addition, BLEEP can also preserve the biological heterogeneity in the predicted expression profiles and recapture and denoise the gene - gene correlations in the original dataset.

Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning

Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer

Multimodal contrastive learning for spatial gene expression prediction using histology images

Gene expression prediction from histology images via hypergraph neural networks

Estrogen Receptor Gene Expression Prediction from H&E Whole Slide Images

HE2Gene: Image-to-RNA Translation Via Multi-Task Learning for Spatial Transcriptomics Data

hist2RNA: An efficient deep learning architecture to predict gene expression from breast cancer histopathology images

HistoSPACE: Histology-Inspired Spatial Transcriptome Prediction And Characterization Engine

Breast Cancer Histopathology Image based Gene Expression Prediction using Spatial Transcriptomics data and Deep Learning

sCellST: a Multiple Instance Learning approach to predict single-cell gene expression from H&E images using spatial transcriptomics

Enhancing Spatial Transcriptomics Analysis by Integrating Image-Aware Deep Learning Methods

Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors

Machine learning enabled prediction of digital biomarkers from whole slide histopathology images

Hist2Cell: Deciphering Fine-grained Cellular Architectures from Histology Images

Contrastive learning-based computational histopathology predict differential expression of cancer driver genes

Inferring single-cell spatial gene expression with tissue morphology via explainable deep learning

Prediction of Tumor-Associated Macrophages and Immunotherapy Benefits Using Weakly Supervised Contrastive Learning in Breast Cancer Pathology Images

PathOmCLIP: Connecting tumor histology with spatial gene expression via locally enhanced contrastive learning of Pathology and Single-cell foundation model

Spatially Resolved Gene Expression Prediction from Histology via Multi-view Graph Contrastive Learning with HSIC-bottleneck Regularization

LEARNING BINARY SEMANTIC EMBEDDING FOR BREAST HISTOLOGY IMAGE CLASSIFICATION AND RETRIEVAL