Abstract:While deep neural networks have achieved remarkable performance, they tend to lack transparency in prediction. The pursuit of greater interpretability in neural networks often results in a degradation of their original performance. Some works strive to improve both interpretability and performance, but they primarily depend on meticulously imposed conditions. In this paper, we propose a simple yet effective framework that acquires more explainable activation heatmaps and simultaneously increase the model performance, without the need for any extra supervision. Specifically, our concise framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. The explanation consistency metric is utilized to measure the similarity between the model's visual explanations of the original samples and those of semantic-preserved adversarial samples, whose background regions are perturbed by using image adversarial attack techniques. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations (i.e., low explanation consistency), for which the current model cannot provide robust interpretations. Comprehensive experimental results on various benchmarks demonstrate the superiority of our framework in multiple aspects, including higher recognition accuracy, greater data debiasing capability, stronger network robustness, and more precise localization ability on both regular networks and interpretable networks. We also provide extensive ablation studies and qualitative analyses to unveil the detailed contribution of each component.

Interpretable Disentanglement of Neural Networks by Extracting Class-Specific Subnetwork

Interpret Neural Networks by Extracting Critical Subnetworks

Functional Network: A Novel Framework for Interpretability of Deep Neural Networks

How to Explain Neural Networks: A perspective of data space division

Unsupervised Learning of Neural Networks to Explain Neural Networks (extended abstract)

Visual Interpretability forDeepLearning

Interpretable Neural Network Decoupling.

Interpretable Function Embedding and Module in Convolutional Neural Networks

Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks

DISCOVER: Making Vision Networks Interpretable via Competition and Dissection

HSDN: A High-Order Structural Semantic Disentangled Neural Network

An Interpretable Neural Network Model Through Piecewise Linear Approximation

CHIP: Channel-wise Disentangled Interpretation of Deep Convolutional Neural Networks

Visual Interpretability for Deep Learning: a Survey

An Interpretability Algorithm of Neural Network Based on Neural Support Decision Tree

InfoDisent: Explainability of Image Classification Models by Information Disentanglement

Improving Network Interpretability via Explanation Consistency Evaluation

Unsupervised Learning of Neural Networks to Explain Neural Networks

Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces

Novel Interpretable Mechanism of Neural Networks Based on Network Decoupling Method

An Interpretable Neuron Embedding for Static Knowledge Distillation