Abstract:Spatial transcriptomics (ST) has emerged as an advanced technology that provides spatial context to gene expression. Recently, deep learning-based methods have shown the capability to predict gene expression from WSI data using ST data. Existing approaches typically extract features from images and the neighboring regions using pretrained models, and then develop methods to fuse this information to generate the final output. However, these methods often fail to account for the cellular structure similarity, cellular density and the interactions within the microenvironment. In this paper, we propose a framework named BG-TRIPLEX, which leverages boundary information extracted from pathological images as guiding features to enhance gene expression prediction from WSIs. Specifically, our model consists of three branches: the spot, in-context and global branches. In the spot and in-context branches, boundary information, including edge and nuclei characteristics, is extracted using pretrained models. These boundary features guide the learning of cellular morphology and the characteristics of microenvironment through Multi-Head Cross-Attention. Finally, these features are integrated with global features to predict the final output. Extensive experiments were conducted on three public ST datasets. The results demonstrate that our BG-TRIPLEX consistently outperforms existing methods in terms of Pearson Correlation Coefficient (PCC). This method highlights the crucial role of boundary features in understanding the complex interactions between WSI and gene expression, offering a promising direction for future research.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the problem of predicting spatial gene expression from Whole Slide Images (WSI). Specifically, although existing methods can directly predict spatial gene expression from WSI data, they often overlook key factors such as cell - structure similarity, cell density, and interactions within the micro - environment. These problems lead to deficiencies in the prediction accuracy of existing methods. To overcome these challenges, the authors propose a new framework, BG - TRIPLEX, which uses boundary information (such as edge and nuclear features) as guiding features to enhance the accuracy of gene expression prediction. BG - TRIPLEX addresses the limitations of existing methods in the following ways: 1. **Introducing boundary information**: Extract and utilize boundary information (such as edge and nuclear features), which can better capture cell morphology and micro - environment features. 2. **Multi - branch model**: Design a model with three branches (spot branch, in - context branch, and global branch), each responsible for extracting features at different levels. 3. **Multi - Head Cross - Attention (MCA) mechanism**: Integrate boundary information with image features through the MCA mechanism to more accurately capture detailed cell - level features. ### Specific improvements of the model - **Spot branch**: Extract the boundary information (edge and nuclear features) of the target area and guide the learning of image features through the MCA mechanism. - **In - context branch**: Extract the boundary information of the adjacent areas around the target area and also guide feature learning through the MCA mechanism. - **Global branch**: Process all spot patches in the entire WSI to obtain a global view and help the model understand the overall structure and layout of the tissue. Through this multi - branch, multi - level feature extraction and fusion strategy, BG - TRIPLEX significantly improves the accuracy and robustness of spatial gene expression prediction. ### Experimental results The experimental results show that BG - TRIPLEX outperforms existing methods on multiple public datasets, especially showing a significant improvement in the Pearson correlation coefficient (PCC) metric. For example, on the Skin dataset, PCC(M) and PCC(H) reach 0.655 and 0.752 respectively, which is a significant improvement compared to other methods. In addition, the generalization performance of BG - TRIPLEX on the Visium dataset is also excellent, demonstrating its strong ability to handle highly expressed genes. ### Summary This paper effectively improves the accuracy of predicting spatial gene expression from WSI by introducing boundary information and a multi - branch model, providing new directions and ideas for future research.

Boundary-Guided Learning for Gene Expression Prediction in Spatial Transcriptomics

Accurate Spatial Gene Expression Prediction by integrating Multi-resolution features

Enhancing Spatial Transcriptomics Analysis by Integrating Image-Aware Deep Learning Methods

DeepST: identifying spatial domains in spatial transcriptomics by deep learning

Integrating spatial transcriptomics and bulk RNA-seq: predicting gene expression with enhanced resolution through graph attention networks

Spatial transcriptomics prediction from histology jointly through Transformer and graph neural networks

Spatially Resolved Gene Expression Prediction from Histology via Multi-view Graph Contrastive Learning with HSIC-bottleneck Regularization

Generative Self-Supervised Graphs Enhance Integration, Imputation and Domains Identification of Spatial Transcriptomics

ST-SCSR: identifying spatial domains in spatial transcriptomics data via structure correlation and self-representation

stEnTrans: Transformer-based deep learning for spatial transcriptomics enhancement

Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction

Generating single-cell gene expression profiles for high-resolution spatial transcriptomics based on cell boundary images

Edge-relational window-attentional graph neural network for gene expression prediction in spatial transcriptomics analysis

Accurate Spatial Heterogeneity Dissection and Gene Regulation Interpretation for Spatial Transcriptomics using Dual Graph Contrastive Learning

Spatially informed clustering, integration, and deconvolution of spatial transcriptomics with GraphST

DSTG: Deconvoluting Spatial Transcriptomics Data Through Graph-Based Artificial Intelligence

A multi-modality and multi-granularity collaborative learning framework for identifying spatial domains and spatially variable genes

Accurately deciphering spatial domains for spatially resolved transcriptomics with stCluster

STimage-1K4M: A histopathology image-gene expression dataset for spatial transcriptomics

SPACEL: deep learning-based characterization of spatial transcriptome architectures

Enhancing Spatial Domain Identification in Spatially Resolved Transcriptomics Using Graph Convolutional Networks with Adaptively Feature-Spatial Balance and Contrastive Learning