Abstract:Problem: The past decades have yielded an explosion of research using artificial intelligence for cancer detection and diagnosis in the field of computational pathology. Yet, an often unspoken assumption of this research is that a glass microscopy slide faithfully represents the underlying disease. Here we show systematic failure modes may dominate the slides digitized from a given medical center, such that neither the whole slide images nor the glass slides are suitable for rendering a diagnosis. Methods: We quantitatively define high quality data as a set of whole slide images where the type of surgery the patient received may be accurately predicted by an automated system such as ours, called "iQC". We find iQC accurately distinguished biopsies from nonbiopsies, e.g. prostatectomies or transurethral resections (TURPs, a.k.a. prostate chips), only when the data qualitatively appeared to be high quality, e.g. vibrant histopathology stains and minimal artifacts. Crucially, prostate needle biopsies appear as thin strands of tissue, whereas prostatectomies and TURPs appear as larger rectangular blocks of tissue. Therefore, when the data are of high quality, iQC (i) accurately classifies pixels as tissue, (ii) accurately generates statistics that describe the distribution of tissue in a slide, and (iii) accurately predicts surgical procedure. Results: While we do not control any medical center's protocols for making or storing slides, we developed the iQC tool to hold all medical centers and datasets to the same objective standard of quality. We validate this standard across five Veterans Affairs Medical Centers (VAMCs) and the Automated Gleason Grading Challenge (AGGC) 2022 public dataset. For our surgical procedure prediction task, we report an Area Under Receiver Operating Characteristic (AUROC) of 0.9966-1.000 at the VAMCs that consistently produce high quality data and AUROC of 0.9824 for the AGGC dataset. In contrast, we report an AUROC of 0.7115 at the VAMC that consistently produced poor quality data. An attending pathologist determined poor data quality was likely driven by faded histopathology stains and protocol differences among VAMCs. Specifically, this VAMC produced slides entirely by hand, whereas all other VAMCs leveraged automated methods to produce slides. Conclusion: Our surgical procedure prediction AUROC may be a quantitative indicator positively associated with high data quality at a medical center or for a specific dataset. To produce high quality data, we recommend producing slides using robotics or other forms of automation whenever possible. We recommend scanning slides digitally before the glass slide has time to develop signs of age, e.g faded stains and acrylamide bubbles. To our knowledge, iQC is the first automated system in computational pathology that validates data quality against objective evidence, e.g. surgical procedure data available in the EHR or LIMS, which requires zero efforts or annotations from anatomic pathologists.

Artificial Intelligence-Based Quality Assessment of Histopathology Whole-Slide Images within a Clinical Workflow: Assessment of 'PathProfiler' in a Diagnostic Pathology Setting

Artificial intelligence in digital histopathology for predicting patient prognosis and treatment efficacy in breast cancer

Validation of Whole-Slide Imaging for Histolopathogical Diagnosis: Current State

Independent real‐world application of a clinical‐grade automated prostate cancer detection system

Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy

Artificial intelligence in diagnostic pathology

Clinical Validation of Artificial Intelligence-Augmented Pathology Diagnosis Demonstrates Significant Gains in Diagnostic Accuracy in Prostate Cancer Detection.

An Integrated Digital Pathology Platform for Tumors Using Artificial Intelligence Analysis

Artificial Intelligence for Digital and Computational Pathology

An Artificial Intelligent System for Prostate Cancer Diagnosis in Whole Slide Images

iQC: machine-learning-driven prediction of surgical procedure uncovers systematic confounds of cancer whole slide images in specific medical centers

Accurate diagnosis achieved via super-resolution whole slide images by pathologists and artificial intelligence

Preparing Data for Artificial Intelligence in Pathology with Clinical-Grade Performance

Digital Pathology for Better Clinical Practice

Artificial intelligence in histopathological image analysis of central nervous system tumours: A systematic review

Critical evaluation of artificial intelligence as a digital twin of pathologists for prostate cancer pathology

Using artificial intelligence to prioritize pathology samples: report of a test drive

The Effect of Quality Control on Accuracy of Digital Pathology Image Analysis

Artificial intelligence in digital pathology: a diagnostic test accuracy systematic review and meta-analysis

Study Protocol: Development and Retrospective Validation of an Artificial Intelligence System for Diagnostic Assessment of Prostate Biopsies