Abstract:Background and objective: Histopathology is the gold standard for diagnosis of many cancers. Recent advances in computer vision, specifically deep learning, have facilitated the analysis of histopathology images for many tasks, including the detection of immune cells and microsatellite instability. However, it remains difficult to identify optimal models and training configurations for different histopathology classification tasks due to the abundance of available architectures and the lack of systematic evaluations. Our objective in this work is to present a software tool that addresses this need and enables robust, systematic evaluation of neural network models for patch classification in histology in a light-weight, easy-to-use package for both algorithm developers and biomedical researchers. Methods: Here we present ChampKit (Comprehensive Histopathology Assessment of Model Predictions toolKit): an extensible, fully reproducible evaluation toolkit that is a one-stop-shop to train and evaluate deep neural networks for patch classification. ChampKit curates a broad range of public datasets. It enables training and evaluation of models supported by timm directly from the command line, without the need for users to write any code. External models are enabled through a straightforward API and minimal coding. As a result, Champkit facilitates the evaluation of existing and new models and deep learning architectures on pathology datasets, making it more accessible to the broader scientific community. To demonstrate the utility of ChampKit, we establish baseline performance for a subset of possible models that could be employed with ChampKit, focusing on several popular deep learning models, namely ResNet18, ResNet50, and R26-ViT, a hybrid vision transformer. In addition, we compare each model trained either from random weight initialization or with transfer learning from ImageNet pretrained models. For ResNet18, we also consider transfer learning from a self-supervised pretrained model. Results: The main result of this paper is the ChampKit software. Using ChampKit, we were able to systemically evaluate multiple neural networks across six datasets. We observed mixed results when evaluating the benefits of pretraining versus random intialization, with no clear benefit except in the low data regime, where transfer learning was found to be beneficial. Surprisingly, we found that transfer learning from self-supervised weights rarely improved performance, which is counter to other areas of computer vision. Conclusions: Choosing the right model for a given digital pathology dataset is nontrivial. ChampKit provides a valuable tool to fill this gap by enabling the evaluation of hundreds of existing (or user-defined) deep learning models across a variety of pathology tasks. Source code and data for the tool are freely accessible at https://github.com/SBU-BMI/champkit.

PatchSorter: A High Throughput Deep Learning Digital Pathology Tool for Object Labeling

PatchSorter: a high throughput deep learning digital pathology tool for object labeling

Efficient Classification of Histopathology Images

HunCRC: annotated pathological slides to enhance deep learning applications in colorectal cancer screening

Augmenting the Pathology Lab: An Intelligent Whole Slide Image Classification System for the Real World

A Generalized Deep Learning Framework for Whole-Slide Image Segmentation and Analysis

EasierPath: An Open-source Tool for Human-in-the-loop Deep Learning of Renal Pathology

Open and reusable deep learning for pathology with WSInfer and QuPath

HEAL: an automated deep learning framework for cancer histopathology image analysis

PathUp: Patch-wise Timestep Tracking for Multi-class Large Pathology Image Synthesising Diffusion Model

Automated curation of large‐scale cancer histopathology image datasets using deep learning

Label-Efficient Breast Cancer Histopathological Image Classification

Openhi: Open Platform For Histopathological Image Annotation

Computational Pathology at Health System Scale -- Self-Supervised Foundation Models from Three Billion Images

ChampKit: A framework for rapid evaluation of deep neural networks for patch-based histopathology classification

PDBL: Improving Histopathological Tissue Classification with Plug-and-Play Pyramidal Deep-Broad Learning

DEPICTER: Deep representation clustering for histology annotation

High Throughput Image Labeling on Chest Computed Tomography by Deep Learning.

FastPathology: An open-source platform for deep learning-based research and decision support in digital pathology

Quick Annotator: an open-source digital pathology based rapid image annotation tool

ComPRePS: An Automated Cloud-based Image Analysis tool to democratize AI in Digital Pathology