Abstract:Objectives Accurate histological typing plays an important role in diagnosing thymoma or thymic carcinoma (TC) and predicting the corresponding prognosis. In this paper, we develop and validate a deep learning-based thymoma typing method for hematoxylin & eosin (H&E)-stained whole slide images (WSIs), which provides useful histopathology information from patients to assist doctors for better diagnosing thymoma or TC. Methods We propose a multi-path cross-scale vision transformer (MC-ViT), which first uses the cross attentive scale-aware transformer (CAST) to classify the pathological information related to thymoma, and then uses such pathological information priors to assist the WSIs transformer (WT) for thymoma typing. To make full use of the multi-scale (10×, 20×, and 40×) information inherent in a WSI, CAST not only employs parallel multi-path to capture different receptive field features from multi-scale WSI inputs, but also introduces the cross-correlation attention module (CAM) to aggregate multi-scale features to achieve cross-scale spatial information complementarity. After that, WT can effectively convert full-scale WSIs into 1D feature matrices with pathological information labels to improve the efficiency and accuracy of thymoma typing. Results We construct a large-scale thymoma histopathology WSI (THW) dataset and annotate corresponding pathological information and thymoma typing labels. The proposed MC-ViT achieves the Top-1 accuracy of 0.939 and 0.951 in pathological information classification and thymoma typing, respectively. Moreover, the quantitative and statistical experiments on the THW dataset also demonstrate that our pipeline performs favorably against the existing classical convolutional neural networks, vision transformers, and deep learning-based medical image classification methods. Conclusion This paper demonstrates that comprehensively utilizing the pathological information contained in multi-scale WSIs is feasible for thymoma typing and achieves clinically acceptable performance. Specifically, the proposed MC-ViT can well predict pathological information classes as well as thymoma types, which show the application potential to the diagnosis of thymoma and TC and may assist doctors in improving diagnosis efficiency and accuracy.

Histopathology Cross-Modal Retrieval Based on Dual-Transformer Network

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval

A Retrieval System For 3d Multi-Phase Contrast-Enhanced Ct Images Of Focal Liver Lesions Based On Combined Bags Of Visual Words And Texture Words

Texture-specific Bag of Visual Words Model and Spatial Cone Matching-Based Method for the Retrieval of Focal Liver Lesions Using Multiphase Contrast-Enhanced CT Images

RetCCL: Clustering-guided contrastive learning for whole-slide image retrieval

MGCT: Mutual-Guided Cross-Modality Transformer for Survival Outcome Prediction using Integrative Histopathology-Genomic Features

Encoding histopathology whole slide images with location-aware graphs for diagnostically relevant regions retrieval

Dual-branch hybrid encoding embedded network for histopathology image classification.

A self-supervised framework for cross-modal search in histopathology archives using scale harmonization

A Computer-Aided Diagnosis System for Breast Pathology: A Deep Learning Approach with Model Interpretability from Pathological Perspective

Lifelong Histopathology Whole Slide Image Retrieval via Distance Consistency Rehearsal

Shared-Specific Feature Learning with Bottleneck Fusion Transformer for Multi-Modal Whole Slide Image Analysis.

DSCA: A dual-stream network with cross-attention on whole-slide image pyramids for cancer prognosis

HistGen: Histopathology Report Generation via Local-Global Feature Encoding and Cross-modal Context Interaction

Histopathological Image Deep Feature Representation for CBIR in Smart PACS

Dual-Attention Multiple Instance Learning Framework for Pathology Whole-Slide Image Classification

Multimodal Cross-Task Interaction for Survival Analysis in Whole Slide Pathological Images

MC-ViT: Multi-path cross-scale vision transformer for thymoma histopathology whole slide image typing

High-Order Correlation-Guided Slide-Level Histology Retrieval with Self-Supervised Hashing.

Multimodal Co-Attention Transformer for Survival Prediction in Gigapixel Whole Slide Images

Hierarchical Transformer for Survival Prediction Using Multimodality Whole Slide Images and Genomics