Abstract:The quest for enhancing the interpretability of neural networks has become a prominent focus in recent research endeavors. Prototype-based neural networks have emerged as a promising avenue for imbuing models with interpretability by gauging the similarity between image components and category prototypes to inform decision-making. However, these networks face challenges as they share similarity activations during both the inference and explanation processes, creating a trade-off between accuracy and interpretability. To address this issue and ensure that a network achieves high accuracy and robust interpretability in the classification process, this paper introduces a groundbreaking prototype-based neural network termed the “Decoupling Prototypical Network” (DProtoNet). This novel architecture comprises encoder, inference, and interpretation modules. In the encoder module, we introduce decoupling feature masks to facilitate the generation of feature vectors and prototypes, enhancing the generalization capabilities of the model. The inference module leverages these feature vectors and prototypes to make predictions based on similarity comparisons, thereby preserving an interpretable inference structure. Meanwhile, the interpretation module advances the field by presenting a novel approach: a “multiple dynamic masks decoder” that replaces conventional upsampling similarity activations. This decoder operates by perturbing images with mask vectors of varying sizes and learning saliency maps through consistent activation. This methodology offers a precise and innovative means of interpreting prototype-based networks. DProtoNet effectively separates the inference and explanation components within prototype-based networks. By eliminating the constraints imposed by shared similarity activations during the inference and explanation phases, our approach concurrently elevates accuracy and interpretability. Experimental evaluations on diverse public natural datasets, including CUB-200-2011, Stanford Cars, and medical datasets like RSNA and iChallenge-PM, corroborate the substantial enhancements achieved by our method compared to previous state-of-the-art approaches. Furthermore, ablation studies are conducted to provide additional evidence of the effectiveness of our proposed components.

Supplementary Material:Interpretable Neural Networks Decoupling

Interpretable Neural Network Decoupling.

Visualizing and comparing AlexNet and VGG using deconvolutional layers

Functional Network: A Novel Framework for Interpretability of Deep Neural Networks

Novel Interpretable Mechanism of Neural Networks Based on Network Decoupling Method

Deeper Interpretability of Deep Networks

Decoupling Deep Learning for Enhanced Image Recognition Interpretability

Network Decoupling: From Regular to Depthwise Separable Convolutions

PICNN: A Pathway towards Interpretable Convolutional Neural Networks

Retina Gap Junction Networks Facilitate Blind Denoising in the Visual Hierarchy

DRNet: Dissect and Reconstruct the Convolutional Neural Network Via Interpretable Manners

Convolutional neural network models of neuronal responses in macaque V1 reveal limited non-linear processing

Decoupling Deep Learning for Interpretable Image Recognition

Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters

Decoupled Networks ( with Appendix )

Understanding Neural Networks Through Deep Visualization

Interpretable Disentanglement of Neural Networks by Extracting Class-Specific Subnetwork

Interpretable Function Embedding and Module in Convolutional Neural Networks

Supplementary Material for “DualNet: Learn Complementary Features for Image Recognition”

Visualizing Deep Neural Network by Alternately Image Blurring and Deblurring

Identifying Interpretable Visual Features in Artificial and Biological Neural Systems