Abstract:BackgroundThere is growing interest in utilizing artificial intelligence, and particularly deep learning, for computer vision in histopathology. While accumulating studies highlight expert-level performance of convolutional neural networks (CNNs) on focused classification tasks, most studies rely on probability distribution scores with empirically defined cutoff values based on post-hoc analysis. More generalizable tools that allow humans to visualize histology-based deep learning inferences and decision making are scarce.ResultsHere, we leverage t-distributed Stochastic Neighbor Embedding (t-SNE) to reduce dimensionality and depict how CNNs organize histomorphologic information. Unique to our workflow, we develop a quantitative and transparent approach to visualizing classification decisions prior to softmax compression. By discretizing the relationships between classes on the t-SNE plot, we show we can super-impose randomly sampled regions of test images and use their distribution to render statistically-driven classifications. Therefore, in addition to providing intuitive outputs for human review, this visual approach can carry out automated and objective multi-class classifications similar to more traditional and less-transparent categorical probability distribution scores. Importantly, this novel classification approach is driven by a priori statistically defined cutoffs. It therefore serves as a generalizable classification and anomaly detection tool less reliant on post-hoc tuning.ConclusionRoutine incorporation of this convenient approach for quantitative visualization and error reduction in histopathology aims to accelerate early adoption of CNNs into generalized real-world applications where unanticipated and previously untrained classes are often encountered.

The strong convergence of visual classification method and its applications in disease diagnosis

The strong convergence of visual classification method and its applications

Disease prediction with edge-variational graph convolutional networks

An Automatic Multi-View Disease Detection System Via Collective Deep Region-based Feature Representation

A better approach to diagnose retinal diseases: Combining our Segmentation-based Vascular Enhancement with deep learning features

A New Approach for Classification: Visual Simulation Point of View

A Health Diagnosis System based on Transfer Learning and Multi-scale Dilated Convolution from Binocular Fundus Image Pairs

Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction

Identifying novel disease categories through divergence optimization: An approach to prevent misdiagnosis in medical imaging

Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review

Feature Selection and Feature Extraction-Aided Classification Approaches for Disease Diagnosis

Popular deep learning algorithms for disease prediction: a review

Exploring Web Images to Enhance Skin Disease Analysis under A Computer Vision Framework

Opportunities and Challenges: Classification of Skin Disease Based on Deep Learning

A supervised learning algorithm based on the quasi-Bayesian method for the probability density functions and application for medical data

Convolutional Neural Networks for Medical Image Diagnosis and Prognosis

Application of deep-learning based computer vision in medical image analysis

Exploring Local Features and the Bag-of-Visual-Words Approach for Bioimage Classification.

Discriminative Pattern Mining for Breast Cancer Histopathology Image Classification via Fully Convolutional Autoencoder

A multi-class fundus disease classification system based on an adaptive scale discriminator and hybrid loss

Disease classification method based on fundus images