Abstract:Our manuscript describes the limitations of deep learning in histopathological image analysis and introduces a standardised approach to explore which Deep Neural Networks (DNNs), are trustworthy or deceptive on histopathological tasks. After replicating findings that DNNs can predict whole‐slide molecular panels, we discovered they cannot accurately predict spatially resolved genomic‐data. Aims Deep learning holds immense potential for histopathology, automating tasks that are simple for expert pathologists and revealing novel biology for tasks that were previously considered difficult or impossible to solve by eye alone. However, the extent to which the visual strategies learned by deep learning models in histopathological analysis are trustworthy or not has yet to be systematically analysed. Here, we systematically evaluate deep neural networks (DNNs) trained for histopathological analysis in order to understand if their learned strategies are trustworthy or deceptive. Methods and results We trained a variety of DNNs on a novel data set of 221 whole‐slide images (WSIs) from lung adenocarcinoma patients, and evaluated their effectiveness at (1) molecular profiling of KRAS versus EGFR mutations, (2) determining the primary tissue of a tumour and (3) tumour detection. While DNNs achieved above‐chance performance on molecular profiling, they did so by exploiting correlations between histological subtypes and mutations, and failed to generalise to a challenging test set obtained through laser capture microdissection (LCM). In contrast, DNNs learned robust and trustworthy strategies for determining the primary tissue of a tumour as well as detecting and localising tumours in tissue. Conclusions Our work demonstrates that DNNs hold immense promise for aiding pathologists in analysing tissue. However, they are also capable of achieving seemingly strong performance by learning deceptive strategies that leverage spurious correlations, and are ultimately unsuitable for research or clinical work. The framework we propose for model evaluation and interpretation is an important step towards developing reliable automated systems for histopathological analysis.

Explaining Deep Learning Models Applied in Histopathology: Current Developments and the Path to Sustainability

Deep Learning Models for Digital Pathology

Deep neural network models for computational histopathology: A survey

Histopathological Images Analysis and Predictive Modeling Implemented in Digital Pathology—Current Affairs and Perspectives

Advancing Histopathology with Deep Learning Under Data Scarcity: A Decade in Review

Machine learning in computational histopathology: Challenges and opportunities

Deep learning in histopathology: the path to the clinic

From modern CNNs to vision transformers: Assessing the performance, robustness, and classification strategies of deep learning models in histopathology

Deep Learning in Digital Pathology Analysis

Practical Application of Deep Learning in Diagnostic Neuropathology—Reimagining a Histological Asset in the Era of Precision Medicine

Deep Learning in Digital Pathology Image Analysis: a Survey

Time to Embrace Natural Language Processing (NLP)-based Digital Pathology: Benchmarking NLP- and Convolutional Neural Network-based Deep Learning Pipelines

A Computer-Aided Diagnosis System for Breast Pathology: A Deep Learning Approach with Model Interpretability from Pathological Perspective

Understanding the Mechanism of Deep Learning Framework for Lesion Detection in Pathological Images with Breast Cancer

Scalable deep learning artificial intelligence histopathology slide analysis and validation

Towards the Augmented Pathologist: Challenges of Explainable-AI in Digital Pathology

The Role of Machine Learning in Cardiovascular Pathology

State of machine and deep learning in histopathological applications in digestive diseases

Counterfactual Diffusion Models for Mechanistic Explainability of Artificial Intelligence Models in Pathology

Digital Histopathology with Graph Neural Networks: Concepts and Explanations for Clinicians

Deceptive learning in histopathology