Abstract:Abstract Computational methods have been proposed to leverage spatially resolved transcriptomic data, pinpointing genes with spatial expression patterns and delineating tissue domains. However, existing approaches fall short in uniformly quantifying spatially variable genes (SVGs). Moreover, from a methodological viewpoint, while SVGs are naturally associated with depicting spatial domains, they are technically dissociated in most methods. Here, we present a framework (PROST) for the quantitative recognition of spatial transcriptomic patterns, consisting of (i) quantitatively characterizing spatial variations in gene expression patterns through the PROST Index; and (ii) unsupervised clustering of spatial domains via a self-attention mechanism. We demonstrate that PROST performs superior SVG identification and domain segmentation with various spatial resolutions, from multicellular to cellular levels. Importantly, PROST Index can be applied to prioritize spatial expression variations, facilitating the exploration of biological insights. Together, our study provides a flexible and robust framework for analyzing diverse spatial transcriptomic data.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the deficiencies of existing methods in quantifying spatially variable genes (SVGs) and detecting spatial domains. Specifically: 1. **Quantifying Spatially Variable Genes (SVGs)**: - Existing methods lack a unified standard when quantifying SVGs, making it difficult to compare different methods. - Although statistical model methods can identify genes with spatial expression patterns, these methods usually rely on statistical significance and it is difficult to interpret the spatial heterogeneity and homogeneity of gene expression. - Although deep - learning methods can identify SVGs, they lack interpretability. 2. **Detecting Spatial Domains**: - When detecting spatial domains with consistent gene expression, due to high - dimensionality and sparsity, traditional non - spatial methods (such as k - means and Louvain algorithms) cannot effectively identify spatially consistent biological domains. - Although some clustering methods specifically designed for spatial transcriptome data (such as stLearn, SpaGCN, etc.) perform well at multi - cell resolution, at cell resolution, especially in complex tissues, there are still challenges, such as dynamic transcriptional heterogeneity and measurement noise. To solve these problems, the paper proposes a framework named PROST (Pattern Recognition Of Spatial Transcriptomics), which includes two modules: PROST Index (PI) and PROST Neural Network (PNN). - **PROST Index (PI)**: - Quantify the spatial pattern of gene expression through a new, assumption - free index (PI score). - The PI score consists of two components: Significance and Separability, which are used to measure the spatial homogeneity and heterogeneity of gene expression respectively. - **PROST Neural Network (PNN)**: - Utilize neighborhood - based graphs and self - attention mechanisms to integrate spatial and transcriptional information and achieve unsupervised tissue segmentation. - Adaptively learn spatial dependencies by optimizing neural network parameters and denoising low - dimensional embeddings, thereby improving the accuracy of tissue segmentation. The paper verifies the superior performance of PROST in SVG identification and spatial domain detection through experiments on multiple spatial transcriptome datasets, especially more prominent at cell resolution.

PROST: quantitative identification of spatially variable genes and domain detection in spatial transcriptomics

Spanve: an Statistical Method to Detect Clustering-friendly Spatially Variable Genes in Large-scale Spatial Transcriptomics Data

ST-SCSR: identifying spatial domains in spatial transcriptomics data via structure correlation and self-representation

DeepST: identifying spatial domains in spatial transcriptomics by deep learning

Deciphering spatial domains from spatially resolved transcriptomics through spatially regularized deep graph networks

Probabilistic cell/domain-type assignment of spatial transcriptomics data with SpatialAnno

SPACE: Spatially variable gene clustering adjusting for cell type effect for improved spatial domain detection

STANCE: a unified statistical model to detect cell-type-specific spatially variable genes in spatial transcriptomics

Vispro improves imaging analysis for Visium spatial transcriptomics

A comprehensive review of approaches for spatial domain recognition of spatial transcriptomes

Evaluating spatially variable gene detection methods for spatial transcriptomics data

Categorization of 33 computational methods to detect spatially variable genes from spatially resolved transcriptomics data

A multi-modality and multi-granularity collaborative learning framework for identifying spatial domains and spatially variable genes

Benchmarking Computational Integration Methods for Spatial Transcriptomics Data

MVST: Identifying spatial domains of spatial transcriptomes from multiple views using multi-view graph convolutional networks

Stmmr: Accurate and Robust Spatial Domain Identification from Spatially Resolved Transcriptomics with Multi-Modal Feature Representation

Generative Self-Supervised Graphs Enhance Integration, Imputation and Domains Identification of Spatial Transcriptomics

Accurately deciphering spatial domains for spatially resolved transcriptomics with stCluster

DESpace : spatially variable gene detection via differential expression testing of spatial clusters

Spatial domains identification in spatial transcriptomics by domain knowledge-aware and subspace-enhanced graph contrastive learning