Abstract:By highlighting important features that contribute to model prediction, visual saliency is used as a natural form to interpret the working mechanism of deep neural networks. Numerous methods have been proposed to achieve better saliency results. However, we find that previous visual saliency methods are not reliable enough to provide meaningful interpretation through a simple sanity check: saliency methods are required to explain the output of non-maximum prediction classes, which are usually not ground-truth classes. For example, let the methods interpret an image of "dog" given a wrong class label "fish" as the query. This procedure can test whether these methods reliably interpret model's predictions based on existing features that appear in the data. Our experiments show that previous methods failed to pass the test by generating similar saliency maps or scattered patterns. This false saliency response can be dangerous in certain scenarios, such as medical diagnosis. We find that these failure cases are mainly due to the attribution vanishing and adversarial noise within these methods. In order to learn reliable visual saliency, we propose a simple method that requires the output of the model to be close to the original output while learning an explanatory saliency mask. To enhance the smoothness of the optimized saliency masks, we then propose a simple Hierarchical Attribution Fusion (HAF) technique. In order to fully evaluate the reliability of visual saliency methods, we propose a new task Disturbed Weakly Supervised Object Localization (D-WSOL) to measure whether these methods can correctly attribute the model's output to existing features. Experiments show that previous methods fail to meet this standard, and our approach helps to improve the reliability by suppressing false saliency responses. After observing a significant layout difference in saliency masks between real and adversarial samples. we propose to train a simple CNN on these learned hierarchical attribution masks to distinguish adversarial samples. Experiments show that our method can improve detection performance over other approaches significantly.

Enhancing the confidence of deep learning classifiers via interpretable saliency maps

Improving Deep Neural Network Classification Confidence using Heatmap-based eXplainable AI

TbExplain: A Text-based Explanation Method for Scene Classification Models with the Statistical Prediction Correction

From Heatmaps to Structural Explanations of Image Classifiers

Towards Self-Explainability of Deep Neural Networks with Heatmap Captioning and Large-Language Models

Explanation Strategies for Image Classification in Humans vs. Current Explainable AI

Explainable Image Recognition via Enhanced Slot-attention Based Classifier

Classification Metrics for Image Explanations: Towards Building Reliable XAI-Evaluations

Learning Reliable Visual Saliency for Model Explanations

Human attention guided explainable artificial intelligence for computer vision models

Leveraging saliency priors and explanations for enhanced consistent interpretability

Explainable GeoAI: can saliency maps help interpret artificial intelligence's learning process? An empirical study on natural feature detection

Interpretable Deep Learning Models: Enhancing Transparency and Trustworthiness in Explainable AI

Improving Disease Classification Performance and Explainability of Deep Learning Models in Radiology with Heatmap Generators

Evaluating the quality of visual explanations on chest X-ray images for thorax diseases classification

Building trust in deep learning-based immune response predictors with interpretable explanations

Deep learning for understanding multilabel imbalanced Chest X-ray datasets

Explanation strategies in humans versus current explainable artificial intelligence: Insights from image classification

Explainable Deep Image Classifiers for Skin Lesion Diagnosis

Deep Learning Methods With the Improved Attention for Explainable Image Recognition

Visualizing Color-wise Saliency of Black-Box Image Classification Models