Abstract:The use of deep learning algorithms to extract meaningful diagnostic features from biomedical images holds the promise to improve patient care given the expansion of digital pathology. Among these deep learning models, Vision Transformer (ViT) models have been demonstrated to capture long-range spatial relationships with more robust prediction power for image classification tasks than regular convolutional neural network (CNN) models, and also better model interpretability. Model interpretation is important for understanding and elucidating how a deep learning model makes predictions, especially for developing transparent models for digital pathology. However, like other deep learning algorithms, with limited annotated biomedical imaging datasets, ViT models are prone to poor performance due to overfitting, which can lead to false predictions due to random noise. Overfitting affects model interpretation when predictions are made out of random noise. To address this issue, we introduce a novel metric – Training Attention and Validation Attention Consistency (TAVAC) – for evaluating ViT model degree of overfitting on imaging datasets and quantifying the reproducibility of interpretation. Specifically, the model interpretation is performed by comparing the high-attention regions in the image between training and testing. We test the method on four publicly available image classification datasets and two independent breast cancer histological image datasets. All overfitted models exhibited significantly lower TAVAC scores than the good-fit models. The TAVAC score quantitatively measures the level of generalization of model interpretation on a fine-grained level for small groups of cells in each H&E image, which cannot be provided by traditional performance evaluation metrics like prediction accuracy. Furthermore, the application of TAVAC extends beyond medical diagnostic AI models; it enhances the monitoring of model interpretative reproducibility at pixel-resolution in basic research, to reveal critical spatial patterns and cellular structures essential to understanding biological processes and disease mechanisms. TAVAC sets a new standard for evaluating the performance of deep learning model interpretation and provides a method for determining the significance of high-attention regions detected from the attention map of the biomedical images.

Statistical Test for Attention Map in Vision Transformer

How Does Attention Work in Vision Transformers? A Visual Analytics Attempt

Class-Discriminative Attention Maps for Vision Transformers

Fixating on Attention: Integrating Human Eye Tracking into Vision Transformers

Constituent Attention for Vision Transformers

ReViT: Enhancing Vision Transformers Feature Diversity with Attention Residual Connections

You Only Need Less Attention at Each Stage in Vision Transformers

Quantifying Interpretation Reproducibility in Vision Transformer Models with TAVAC

Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights

Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets

Emergence of Human-Like Attention in Self-Supervised Vision Transformers: an eye-tracking study

AttentionViz: A Global View of Transformer Attention

On the Surprising Effectiveness of Attention Transfer for Vision Transformers

VTAMIQ: Transformers for Attention Modulated Image Quality Assessment

Rethinking Attention Mechanisms in Vision Transformers with Graph Structures

RegionViT: Regional-to-Local Attention for Vision Transformers

Masked Attention as a Mechanism for Improving Interpretability of Vision Transformers

DAT++: Spatially Dynamic Vision Transformer with Deformable Attention

Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers