Abstract:Background: The advances in deep learning-based pathological image analysis have invoked tremendous insights into cancer prognostication. Still, lack of interpretability remains a significant barrier to clinical application. Methods: We established an integrative prognostic neural network for intrahepatic cholangiocarcinoma (iCCA), towards a comprehensive evaluation of both architectural and fine-grained information from whole-slide images. Then, leveraging on multi-modal data, we conducted extensive interrogative approaches to the models, to extract and visualize the morphological features that most correlated with clinical outcome and underlying molecular alterations. Results: The models were developed and optimized on 373 iCCA patients from our center and demonstrated consistent accuracy and robustness on both internal (n = 213) and external (n = 168) cohorts. The occlusion sensitivity map revealed that the distribution of tertiary lymphoid structures, the geometric traits of the invasive margin, the relative composition of tumor parenchyma and stroma, the extent of necrosis, the presence of the disseminated foci, and the tumor-adjacent micro-vessels were the determining architectural features that impacted on prognosis. Quantifiable morphological vector extracted by CellProfiler demonstrated that tumor nuclei from high-risk patients exhibited significant larger size, more distorted shape, with less prominent nuclear envelope and textural contrast. The multi-omics data (n = 187) further revealed key molecular alterations left morphological imprints that could be attended by the network, including glycolysis, hypoxia, apical junction, mTORC1 signaling, and immune infiltration. Conclusions: We proposed an interpretable deep-learning framework to gain insights into the biological behavior of iCCA. Most of the significant morphological prognosticators perceived by the network are comprehensible to human minds.

An uncertainty-based interpretable deep learning framework for predicting breast cancer outcome

Information bottleneck-based interpretable multitask network for breast cancer classification and segmentation

A domain knowledge-based interpretable deep learning system for improving clinical breast ultrasound diagnosis

Autosurv: interpretable deep learning framework for cancer survival analysis incorporating clinical and multi-omics data

A new xAI framework with feature explainability for tumors decision-making in Ultrasound data: comparing with Grad-CAM

GD‐Net: An Integrated Multimodal Information Model Based on Deep Learning for Cancer Outcome Prediction and Informative Feature Selection

Interpretability methods of machine learning algorithms with applications in breast cancer diagnosis

Intelligent medical diagnosis and treatment for diabetes with deep convolutional fuzzy neural networks

Mining the interpretable prognostic features from pathological image of intrahepatic cholangiocarcinoma using multi-modal deep learning

Uncertainty Quantification in Computer-Aided Diagnosis: Make Your Model say "I don't know" for Ambiguous Cases

An Interpretable Deep Hierarchical Semantic Convolutional Neural Network for Lung Nodule Malignancy Classification

Residual attention based uncertainty-guided mean teacher model for semi-supervised breast masses segmentation in 2D ultrasonography

An Interactive Interpretability System for Breast Cancer Screening with Deep Learning

Deep neural network uncertainty estimation for early oral cancer diagnosis

ISeeU: Visually interpretable deep learning for mortality prediction inside the ICU

UGGNet: Bridging U-Net and VGG for Advanced Breast Cancer Diagnosis

An interpretable integration model improving disease-free survival prediction for gastric cancer based on CT images and clinical parameters

Assessing breast disease with deep learning model using bimodal bi-view ultrasound images and clinical information

Hierarchical Uncertainty Estimation for Medical Image Segmentation Networks

Uncertainty-informed deep learning models enable high-confidence predictions for digital histopathology

Fully Interpretable Deep Learning Model Using IR Thermal Images for Possible Breast Cancer Cases