Abstract:The quest for enhancing the interpretability of neural networks has become a prominent focus in recent research endeavors. Prototype-based neural networks have emerged as a promising avenue for imbuing models with interpretability by gauging the similarity between image components and category prototypes to inform decision-making. However, these networks face challenges as they share similarity activations during both the inference and explanation processes, creating a trade-off between accuracy and interpretability. To address this issue and ensure that a network achieves high accuracy and robust interpretability in the classification process, this paper introduces a groundbreaking prototype-based neural network termed the “Decoupling Prototypical Network” (DProtoNet). This novel architecture comprises encoder, inference, and interpretation modules. In the encoder module, we introduce decoupling feature masks to facilitate the generation of feature vectors and prototypes, enhancing the generalization capabilities of the model. The inference module leverages these feature vectors and prototypes to make predictions based on similarity comparisons, thereby preserving an interpretable inference structure. Meanwhile, the interpretation module advances the field by presenting a novel approach: a “multiple dynamic masks decoder” that replaces conventional upsampling similarity activations. This decoder operates by perturbing images with mask vectors of varying sizes and learning saliency maps through consistent activation. This methodology offers a precise and innovative means of interpreting prototype-based networks. DProtoNet effectively separates the inference and explanation components within prototype-based networks. By eliminating the constraints imposed by shared similarity activations during the inference and explanation phases, our approach concurrently elevates accuracy and interpretability. Experimental evaluations on diverse public natural datasets, including CUB-200-2011, Stanford Cars, and medical datasets like RSNA and iChallenge-PM, corroborate the substantial enhancements achieved by our method compared to previous state-of-the-art approaches. Furthermore, ablation studies are conducted to provide additional evidence of the effectiveness of our proposed components.

This actually looks like that: Proto-BagNets for local and global interpretability-by-design

HQProtoPNet: an Evidence-Based Model for Interpretable Image Recognition.

MProtoNet: A Case-Based Interpretable Model for Brain Tumor Classification with 3D Multi-parametric Magnetic Resonance Imaging

Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes

This Looks Like That: Deep Learning for Interpretable Image Recognition

Evaluation and Improvement of Interpretability for Self-Explainable Part-Prototype Networks

This Looks Better than That: Better Interpretable Models with ProtoPNeXt

Interpreting and Correcting Medical Image Classification with PIP-Net

Decoupling Deep Learning for Enhanced Image Recognition Interpretability

Decoupling Deep Learning for Interpretable Image Recognition

Towards Human-Interpretable Prototypes for Visual Assessment of Image Classification Models

ProtoArgNet: Interpretable Image Classification with Super-Prototypes and Argumentation [Technical Report]

These do not Look Like Those: An Interpretable Deep Learning Model for Image Recognition

This Patient Looks Like That Patient: Prototypical Networks for Interpretable Diagnosis Prediction from Clinical Text

MAProtoNet: A Multi-scale Attentive Interpretable Prototypical Part Network for 3D Magnetic Resonance Imaging Brain Tumor Classification

Prototype-based Interpretable Breast Cancer Prediction Models: Analysis and Challenges

Attri-Net: A Globally and Locally Inherently Interpretable Model for Multi-Label Classification Using Class-Specific Counterfactuals

This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks

Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations

INSightR-Net: Interpretable Neural Network for Regression using Similarity-based Comparisons to Prototypical Examples

XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations