A deep learning-based multiscale integration of spatial omics with tumor morphology.

Benoit Schmauch,Loïc Herpin,Antoine Olivier,Thomas Duboudin,Rémy Dubois,Lucie Gillet,Jean-Baptiste Schiratti,Valentina Di Proietto,Delphine Le Corre,Alexandre Bourgoin,Julien Taïeb,Jean-François Emile,Wolf Herman Fridman,Elodie Pronier,Pierre Laurent-Puig,Eric Yves Durand
DOI: https://doi.org/10.1101/2024.07.22.604083
2024-07-23
Abstract:Spatial Transcriptomics (spTx) offers unprecedented insights into the spatial arrangement of the tumor microenvironment, tumor initiation/progression and identification of new therapeutic target candidates. However, spTx remains complex and unlikely to be routinely used in the near future. Hematoxylin and eosin (H&E) stained histological slides, on the other hand, are routinely generated for a large fraction of cancer patients. Here, we present a novel deep learning-based approach for multiscale integration of spTx with tumor morphology (MISO). We trained MISO to predict spTx from H&E on a new unpublished dataset of 72 10X Genomics Visium samples, and derived a novel estimate of the upper bound on the achievable performance. We demonstrate that MISO enables near single-cell-resolution, spatially-resolved gene expression prediction from H&E. In addition, MISO provides an effective patient representation framework that enables downstream predictive tasks such as molecular phenotyping or MSI prediction.
Bioinformatics
What problem does this paper attempt to address?
This paper aims to solve the problem of how to use deep - learning techniques to integrate spatial omics and tumor morphology. Specifically, the researchers developed a new deep - learning framework named MISO (Multiscale Integration of Spatial Omics) to achieve the goal of predicting spatial transcriptome data from routine H&E - stained sections. The main contributions of this method are as follows: 1. **Improve resolution**: MISO can predict spatial gene expression close to single - cell resolution from H&E - stained images, thus making up for the deficiency that current spatial transcriptome techniques (such as 10X Genomics Visium) cannot reach single - cell resolution. 2. **Multiscale integration**: MISO models the spatial structure of tissues through the local self - attention mechanism, achieving multiscale integration from the cell to the patient level and improving the representational ability of patient data. 3. **Enhance the performance of downstream tasks**: The study found that MISO not only performs well in predicting spatial gene expression, but also the data representation it generates performs excellently in multiple downstream tasks, including molecular phenotype prediction, microsatellite instability (MSI) prediction, etc. 4. **Extended application of data sets**: The MISO model can predict on new samples only with H&E images, which enables this method to be applied to a large number of existing H&E image data sets without expensive spatial transcriptome sequencing data. In conclusion, this paper proposes an innovative method that combines spatial omics and tumor morphology through deep - learning techniques, providing new tools and perspectives for tumor research.