Abstract:Back propagation based visualizations have been proposed to interpret deep neural networks (DNNs), some of which produce interpretations with good visual quality. However, there exist doubts about whether these intuitive visualizations are related to the network decisions. Recent studies have confirmed this suspicion by verifying that almost all these modified back-propagation visualizations are not faithful to the model's decision-making process. Besides, these visualizations produce vague "relative importance scores", among which low values can't guarantee to be independent of the final prediction. Hence, it's highly desirable to develop a novel back-propagation framework that guarantees theoretical faithfulness and produces a quantitative attribution score with a clear understanding. To achieve the goal, we resort to mutual information theory to generate the interpretations, studying how much information of output is encoded in each input neuron. The basic idea is to learn a source signal by back-propagation such that the mutual information between input and output should be as much as possible preserved in the mutual information between input and the source signal. In addition, we propose a Mutual Information Preserving Inverse Network, termed MIP-IN, in which the parameters of each layer are recursively trained to learn how to invert. During the inversion, forward Relu operation is adopted to adapt the general interpretations to the specific input. We then empirically demonstrate that the inverted source signal satisfies completeness and minimality property, which are crucial for a faithful interpretation. Furthermore, the empirical study validates the effectiveness of interpretations generated by MIP-IN.

An Interpretable Regularization Method Based on Minimizing Mutual Information

Towards Interpreting Recurrent Neural Networks Through Probabilistic Abstraction

Information-Theoretic Local Minima Characterization and Regularization

Interpretable Robust Feature Selection via Joint -Norms Minimization.

Mutual Information Preserving Back-propagation: Learn to Invert for Faithful Attribution

How far from automatically interpreting deep learning

Interpreting Deep Learning Model Using Rule-based Method

An Interpretable Neural Network Model Through Piecewise Linear Approximation

Explaining Representation by Mutual Information

Interpretable Deep Convolutional Neural Networks via Meta-learning

Learning Structured and Non-Redundant Representations with Deep Neural Networks

Neighborhood Region Smoothing Regularization for Finding Flat Minima In Deep Neural Networks

Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond

Optimizing for Interpretability in Deep Neural Networks with Tree Regularization

On Learnable Parameters of Optimal and Suboptimal Deep Learning Models

Supposed Maximum Mutual Information for Improving Generalization and Interpretation of Multi-Layered Neural Networks

Interpreting Deep Neural Networks Through Variable Importance

An Information-Theoretic Regularizer for Lossy Neural Image Compression

Neighborhood Region Smoothing Regularization for Finding Flat Minima in Deep Neural Networks

A Survey of the Interpretability Aspect of Deep Learning Models

Large-Margin Regularized Softmax Cross-Entropy Loss