Abstract:Understanding the decision process of neural networks is hard. One vital method for explanation is to attribute its decision to pivotal features. Although many algorithms are proposed, most of them solely improve the faithfulness to the model. However, the real environment contains many random noises, which may leads to great fluctuations in the explanations. More seriously, recent works show that explanation algorithms are vulnerable to adversarial attacks. All of these make the explanation hard to trust in real scenarios. To bridge this gap, we propose a model-agnostic method \emph{Median Test for Feature Attribution} (MeTFA) to quantify the uncertainty and increase the stability of explanation algorithms with theoretical guarantees. MeTFA has the following two functions: (1) examine whether one feature is significantly important or unimportant and generate a MeTFA-significant map to visualize the results; (2) compute the confidence interval of a feature attribution score and generate a MeTFA-smoothed map to increase the stability of the explanation. Experiments show that MeTFA improves the visual quality of explanations and significantly reduces the instability while maintaining the faithfulness. To quantitatively evaluate the faithfulness of an explanation under different noise settings, we further propose several robust faithfulness metrics. Experiment results show that the MeTFA-smoothed explanation can significantly increase the robust faithfulness. In addition, we use two scenarios to show MeTFA's potential in the applications. First, when applied to the SOTA explanation method to locate context bias for semantic segmentation models, MeTFA-significant explanations use far smaller regions to maintain 99\%+ faithfulness. Second, when tested with different explanation-oriented attacks, MeTFA can help defend vanilla, as well as adaptive, adversarial attacks against explanations.

Provable Robust Saliency-based Explanations

Robust Ranking Explanations

Which Neural Network Makes More Explainable Decisions? an Approach Towards Measuring Explainability

Extracting Robust Models with Uncertain Examples

Provably Robust and Plausible Counterfactual Explanations for Neural Networks via Robust Optimisation

Defense Against Explanation Manipulation

On the Trade-offs between Adversarial Robustness and Actionable Explanations

Towards Faithful Explanations for Text Classification with Robustness Improvement and Explanation Guided Training

Adversarial Attack for Explanation Robustness of Rationalization Models

Advancing Certified Robustness of Explanation Via Gradient Quantization

Enhancing adversarial robustness in Natural Language Inference using explanations

Unifying Model Explainability and Robustness for Joint Text Classification and Rationale Extraction

Robust Explainable Recommendation

From Robustness to Explainability and Back Again

Evaluating Robustness of Counterfactual Explanations

Finding Regions of Counterfactual Explanations via Robust Optimization

On the Robustness of Explanations of Deep Neural Network Models: A Survey

Robust Counterfactual Explanations in Machine Learning: A Survey

"Is your explanation stable?": A Robustness Evaluation Framework for Feature Attribution

Robust Explanation for Free or at the Cost of Faithfulness.

On the (In)fidelity and Sensitivity for Explanations