Abstract:The rise of spatial transcriptomics (ST) is transforming our understanding of tumor heterogeneity by enabling high-resolution, location-specific mapping of gene expression across the tumor microenvironment. However, the translational potential of spatial transcriptomics is still limited by its high cost, hindering the assembly of large patient cohorts needed for robust biomarker discovery. Here we present Path2Space, a deep learning model trained on spatial transcriptomics data to predict spatial gene expression directly from histopathology slides. Studying breast cancer, the model was first trained on one ST breast cancer patients cohort and further independently tested and validated on two other ST cohorts. Path2Space identifies 4,500 genes whose spatial expression is robustly predicted in all three cohorts, markedly more accurate than existing ST predictors. Second, we show that based on the inferred ST, Path2Space accurately infers cell-type abundances in the tumor microenvironment (TME). Thirdly, Applying Path2Space to the TCGA breast cancer cohort we identify three TME spatially grounded subsets of patients that have markedly different survival rates. Fourth, analyzing two large breast cancer patients cohorts treated with trastuzumab, we show that the heterogeneity of the inferred HER2 levels across the tumor microenvironment is predictive of response, concordant with the bystander hypothesis. Finally, we developed a spatially interpretable model for predicting trastuzumab response directly from the H&E-slides. This model's performance surpasses that of existing response prediction models that use measured bulk multi-omics data. Path2Space is the first approach capable of robustly delineating the tumor microenvironment on an unprecedented scale directly from the tumor slides. It heralds the upcoming development of robust spatially grounded biomarkers of patients survival and treatment that promises to transform precision oncology in both the developed and developing worlds.

RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency

ST-Align: A Multimodal Foundation Model for Image-Gene Alignment in Spatial Transcriptomics

Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer

STAIG: Spatial Transcriptomics Analysis via Image-Aided Graph Contrastive Learning for Domain Exploration and Alignment-Free Integration

HE2Gene: Image-to-RNA Translation Via Multi-Task Learning for Spatial Transcriptomics Data

Enhancing Spatial Transcriptomics Analysis by Integrating Image-Aware Deep Learning Methods

High-Resolution Spatial Transcriptomics from Histology Images using HisToSGE

Exploit Spatially Resolved Transcriptomic Data to Infer Cellular Features from Pathology Imaging Data}

Leveraging information in spatial transcriptomics to predict super-resolution gene expression from histology images in tumors

Spatial transcriptomics prediction from histology jointly through Transformer and graph neural networks

Spatially Resolved Gene Expression Prediction from Histology via Multi-view Graph Contrastive Learning with HSIC-bottleneck Regularization

Multimodal contrastive learning for spatial gene expression prediction using histology images

Integrating spatial transcriptomics and bulk RNA-seq: predicting gene expression with enhanced resolution through graph attention networks

Multi-modal Spatial Clustering for Spatial Transcriptomics Utilizing High-resolution Histology Images

Spatial gene expression at single-cell resolution from histology using deep learning with GHIST

STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics

Breast Cancer Histopathology Image based Gene Expression Prediction using Spatial Transcriptomics data and Deep Learning

Pix2Path: Integrating Spatial Transcriptomics and Digital Pathology with Deep Learning to Score Pathological Risk and Link Gene Expression to Disease Mechanisms

Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning

Path2Space: An AI Approach for Cancer Biomarker Discovery Via Histopathology Inferred Spatial Transcriptomics