Grad-CAM: Visual Explanations from Deep Networks via Gradient-Basde Localization

RR Selvaruju, M Cogswell, A Das, R Vedantam, D Parikh, D Batra

2016-01-01

Abstract:

What problem does this paper attempt to address?

Grad-CAM: Visual explanations from deep networks via gradient-based localization. arXiv 2016

RR Selvaraju, M Cogswell, A Das, R Vedantam, D Parikh, D Batra

2022-01-01

Abstract:
Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization

R Ramprasaath, D Abhishek, V Ramakrishna, C Michael, P Devi, B Dhruv

2016-01-01

Abstract:
Grad-CAM: visual explanations from deep networks via gradient-based localization

Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra

2020-02-01

Abstract:We propose a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say ‘dog’ in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM …
Statistic-CAM: A Gradient-Free Visual Explanations for Deep Convolutional Network

Yiqin Wang,Lei Xie,Hongye Su

DOI: https://doi.org/10.1109/safeprocess58597.2023.10295912

2023-01-01

Abstract:Algorithms of class activation maps are used to provide visual explanations for CNN models to reduce the black-box nature and are widely used for model evaluation and optimization. However, it is not common to apply such algorithms to model deployment, although real-time explanations for the CNN model is much in need. The reason for that is state-of-the-art CAM algorithms are either gradient-based or complex. It is really difficult to deploy them on edge devices which are power-limited and may not support back-propagation. In response to the criticism of those techniques, we propose a new methodology to generate class activation maps for CNN models in a gradient-free and computation-saving way. Our approach, Statistic-CAM are built upon an existing CAM methodology (called backbone method) and consists of three stages: training stage, saliences fixing stage and inferring stage. By collecting and sorting channel saliences of the last feature maps by classes, the visual explanations can be generated via a simple forward propagation and a look-up in the dictionary. Compared to the backbones, our approach can achieve the performance of less than 2.1% mismatch and save up to 99.95% extra computational cost while maintain the feature of gradient-free. It is no doubt the easiest CAM algorithm to be deployed.
Exclusive Feature Constrained Class Activation Mapping for Better Visual Explanation.

Pengda Wang,Xiangwei Kong,Weikuo Guo,Xunpeng Zhang

DOI: https://doi.org/10.1109/access.2021.3073465

IF: 3.9

2021-01-01

IEEE Access

Abstract:Whereas Deep Neural Network(DNN) shows wonderful performance on large scale data, lacking interpretability limits their usage in scenarios relevant to security. To make visual explanations less noisy and more class-discriminative, in this work, we propose a visual explanation method of DNN, named Exclusive Feature Constrained Class Activation Mapping(EFC-CAM). A new exclusive feature constraint is introduced to optimize the weight calculated from Grad-CAM or initialized from a constant vector. To better measure visual explanation methods, we design an effective evaluation metric which does not need bounding boxes as auxiliary information. Extensive quantitative experiments and visual inspection on ImageNet and Fashion validation set show the effectiveness of the proposed method.
Grad-CAM: Why did you say that?

Ramprasaath R Selvaraju,Abhishek Das,Ramakrishna Vedantam,Michael Cogswell,Devi Parikh,Dhruv Batra

DOI: https://doi.org/10.48550/arXiv.1611.07450

2017-01-26

Abstract:We propose a technique for making Convolutional Neural Network (CNN)-based models more transparent by visualizing input regions that are 'important' for predictions -- or visual explanations. Our approach, called Gradient-weighted Class Activation Mapping (Grad-CAM), uses class-specific gradient information to localize important regions. These localizations are combined with existing pixel-space visualizations to create a novel high-resolution and class-discriminative visualization called Guided Grad-CAM. These methods help better understand CNN-based models, including image captioning and visual question answering (VQA) models. We evaluate our visual explanations by measuring their ability to discriminate between classes, to inspire trust in humans, and their correlation with occlusion maps. Grad-CAM provides a new way to understand CNN-based models. We have released code, an online demo hosted on CloudCV, and a full version of this extended abstract.

Machine Learning,Computer Vision and Pattern Recognition
Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring

Sam Sattarzadeh,Mahesh Sudhakar,Konstantinos N. Plataniotis,Jongseong Jang,Yeonjeong Jeong,Hyunwoo Kim

DOI: https://doi.org/10.48550/arXiv.2102.07805

2021-02-16

Abstract:Visualizing the features captured by Convolutional Neural Networks (CNNs) is one of the conventional approaches to interpret the predictions made by these models in numerous image recognition applications. Grad-CAM is a popular solution that provides such a visualization by combining the activation maps obtained from the model. However, the average gradient-based terms deployed in this method underestimates the contribution of the representations discovered by the model to its predictions. Addressing this problem, we introduce a solution to tackle this issue by computing the path integral of the gradient-based terms in Grad-CAM. We conduct a thorough analysis to demonstrate the improvement achieved by our method in measuring the importance of the extracted representations for the CNN's predictions, which yields to our method's administration in object localization and model interpretation.

Computer Vision and Pattern Recognition
Is Grad-CAM Explainable in Medical Images?

Subhashis Suara,Aayush Jha,Pratik Sinha,Arif Ahmed Sekh

2023-07-20

Abstract:Explainable Deep Learning has gained significant attention in the field of artificial intelligence (AI), particularly in domains such as medical imaging, where accurate and interpretable machine learning models are crucial for effective diagnosis and treatment planning. Grad-CAM is a baseline that highlights the most critical regions of an image used in a deep learning model's decision-making process, increasing interpretability and trust in the results. It is applied in many computer vision (CV) tasks such as classification and explanation. This study explores the principles of Explainable Deep Learning and its relevance to medical imaging, discusses various explainability techniques and their limitations, and examines medical imaging applications of Grad-CAM. The findings highlight the potential of Explainable Deep Learning and Grad-CAM in improving the accuracy and interpretability of deep learning models in medical imaging. The code is available in (will be available).

Image and Video Processing,Computer Vision and Pattern Recognition,Computers and Society
Grad++ScoreCAM: Enhancing Visual Explanations of Deep Convolutional Networks Using Incremented Gradient and Score- Weighted Methods

Shafiullah Soomro,Asim Niaz,Kwang Nam Choi

DOI: https://doi.org/10.1109/access.2024.3392853

IF: 3.9

2024-05-03

IEEE Access

Abstract:We propose a novel method that combines the strengths of two popular class activation mapping techniques, GradCAM++ and ScoreCAM, to improve the interpretability and localization of convolutional neural networks (CNNs). Our proposed method, called "Grad++-ScoreCAM", first utilizes the GradCAM++ algorithm to generate a coarse heatmap of an input image, highlighting the regions of importance for a particular class. Then, we employ the ScoreCAM algorithm to refine the heatmap by incorporating the localization information from the intermediate layers of the network. By combining these two techniques, we can generate more accurate and fine-grained heatmaps that highlight the regions of the input image that are most relevant to the prediction of the CNN. We evaluate our proposed method on a benchmark dataset and demonstrate its superiority over existing methods in terms of accuracy and interpretability. Our method has potential applications in various fields, including medical imaging, object recognition, and natural language processing.

computer science, information systems,telecommunications,engineering, electrical & electronic
Recipro-CAM: Fast gradient-free visual explanations for convolutional neural networks

Won-Jo Lee,Seokhyun Byun

2022-09-28

Abstract:The Convolutional Neural Network (CNN) is a widely used deep learning architecture for computer vision. However, its black box nature makes it difficult to interpret the behavior of the model. To mitigate this issue, AI practitioners have explored explainable AI methods like Class Activation Map (CAM) and Grad-CAM. Although these methods have shown promise, they are limited by architectural constraints or the burden of gradient computing. To overcome this issue, Score-CAM and Ablation-CAM have been proposed as gradient-free methods, but they have longer execution times compared to CAM or Grad-CAM based methods, making them unsuitable for real-world solution though they resolved gradient related issues and enabled inference mode XAI. To address this challenge, we propose a fast gradient-free Reciprocal CAM (Recipro-CAM) method. Our approach involves spatially masking the extracted feature maps to exploit the correlation between activation maps and network predictions for target classes. Our proposed method has yielded promising results, outperforming current state-of-the-art method in the Average Drop-Coherence-Complexity (ADCC) metric by $1.78 \%$ to $3.72 \%$, excluding VGG-16 backbone. Moreover, Recipro-CAM generates saliency maps at a similar rate to Grad-CAM and is approximately $148$ times faster than Score-CAM. The source code for Recipro-CAM is available in our data analysis framework.

Computer Science
Cross-CAM: Focused Visual Explanations for Deep Convolutional Networks Via Training-Set Tracing

Yu Sun,Kailang Ma,Xuanxin Liu,Jian Cui

DOI: https://doi.org/10.1007/978-3-031-10983-6_56

2022-01-01

Abstract:In recent years, the widely used deep learning technologies have always been controversial in terms of reliability and credibility. Class Activation Map (CAM) has been proposed to explain the deep learning models. Existing CAM-based algorithms highlight critical portions of the input image, but they don't go any farther in tracing the neural network's decision-basis. This work proposes Cross-CAM, a visual interpretation method which supports deep traceability for prediction-basis samples and focuses on similar regions of the category based on the input image and the prediction-basis samples. The Cross-CAM extracts deep discriminative feature vectors and screens out the prediction-basis samples from the training set. The similarity-weight and the grad-weight are then combined to form the cross-weight, which highlights similar regions and aids in classification decisions. On the ILSVRC-15 dataset, the proposed Cross-CAM is tested. The new weakly-supervised localization evaluation metric IoS (Intersection over Self) is proposed to effectively evaluate the focusing effect. Using Cross-CAM highlight regions, the top-1 location error for weakly-supervised localization achieves 44.95% on the ILSVRC-15 validation set, which is 16.25% lower than Grad-CAM. In comparison to Grad-CAM, Cross-CAM focuses on the key regions using the similarity between the test image and the prediction-basis samples, according to the visualisation results.
Generalizing GradCAM for Embedding Networks

Mudit Bachhawat

2024-02-05

Abstract:Visualizing CNN is an important part in building trust and explaining model's prediction. Methods like CAM and GradCAM have been really successful in localizing area of the image responsible for the output but are only limited to classification models. In this paper, we present a new method EmbeddingCAM, which generalizes the Grad-CAM for embedding networks. We show that for classification networks, EmbeddingCAM reduces to GradCAM. We show the effectiveness of our method on CUB-200-2011 dataset and also present quantitative and qualitative analysis on the dataset.

Computer Vision and Pattern Recognition
Eigen-CAM: Visual Explanations for Deep Convolutional Neural Networks

Mohammed Bany Muhammad,Mohammed Yeasin

DOI: https://doi.org/10.1007/s42979-021-00449-3

2021-01-20

SN Computer Science

Abstract:The adoption of deep convolutional neural networks (CNN) is growing exponentially in wide varieties of applications due to exceptional performance that equals to or is better than classical machine learning as well as a human. However, such models are difficult to interpret, susceptible to overfit, and hard to decode failure. An increasing body of literature, such as class activation map (CAM), focused on understanding what representations or features a model learned from the data. This paper presents novel Eigen-CAM to enhance explanations of CNN predictions by visualizing principal components of learned representations from convolutional layers. The Eigen-CAM is intuitive, easy to use, computationally efficient, and does not require correct classification by the model. Eigen-CAM can work with all CNN models without the need to modify layers or retrain models. For the task of generating a visual explanation of CNN predictions, compared to state-of-the-art methods, Eigen-CAM is more consistent, class discriminative, and robust against classification errors made by dense layers. Empirical analyses and comparison with the best state-of-the-art methods show up to 12% improvement in weakly-supervised object localization, an average of 13% improvement in weakly-supervised segmentation, and at least 15% improvement in generic object proposal.
Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks

Haofan Wang,Zifan Wang,Mengnan Du,Fan Yang,Zijian Zhang,Sirui Ding,Piotr Mardziel,Xia Hu

DOI: https://doi.org/10.1109/cvprw50498.2020.00020

2020-01-01

Abstract:Recently, increasing attention has been drawn to the internal mechanisms of convolutional neural networks, and the reason why the network makes specific decisions. In this paper, we develop a novel post-hoc visual explanation method called Score-CAM based on class activation mapping. Unlike previous class activation mapping based approaches, Score-CAM gets rid of the dependence on gradients by obtaining the weight of each activation map through its forward passing score on target class, the final result is obtained by a linear combination of weights and activation maps. We demonstrate that Score-CAM achieves better visual performance and fairness for interpreting the decision making process. Our approach outperforms previous methods on both recognition and localization tasks, it also passes the sanity check. We also indicate its application as debugging tools. The implementation is available(1).
Shap-CAM: Visual Explanations for Convolutional Neural Networks Based on Shapley Value.

Quan Zheng,Ziwei Wang,Jie Zhou,Jiwen Lu

DOI: https://doi.org/10.1007/978-3-031-19775-8_27

2022-01-01

Abstract: Explaining deep convolutional neural networks has been recently drawing increasing attention since it helps to understand the networks' internal operations and why they make certain decisions. Saliency maps, which emphasize salient regions largely connected to the network's decision-making, are one of the most common ways for visualizing and analyzing deep networks in the computer vision community. However, saliency maps generated by existing methods cannot represent authentic information in images due to the unproven proposals about the weights of activation maps which lack solid theoretical foundation and fail to consider the relations between each pixel. In this paper, we develop a novel post-hoc visual explanation method called Shap-CAM based on class activation mapping. Unlike previous gradient-based approaches, Shap-CAM gets rid of the dependence on gradients by obtaining the importance of each pixel through Shapley value. We demonstrate that Shap-CAM achieves better visual performance and fairness for interpreting the decision making process. Our approach outperforms previous methods on both recognition and localization tasks.
Generalizing Adversarial Explanations with Grad-CAM

Tanmay Chakraborty,Utkarsh Trehan,Khawla Mallat,Jean-Luc Dugelay

DOI: https://doi.org/10.48550/arXiv.2204.05427

2022-04-12

Abstract:Gradient-weighted Class Activation Mapping (Grad- CAM), is an example-based explanation method that provides a gradient activation heat map as an explanation for Convolution Neural Network (CNN) models. The drawback of this method is that it cannot be used to generalize CNN behaviour. In this paper, we present a novel method that extends Grad-CAM from example-based explanations to a method for explaining global model behaviour. This is achieved by introducing two new metrics, (i) Mean Observed Dissimilarity (MOD) and (ii) Variation in Dissimilarity (VID), for model generalization. These metrics are computed by comparing a Normalized Inverted Structural Similarity Index (NISSIM) metric of the Grad-CAM generated heatmap for samples from the original test set and samples from the adversarial test set. For our experiment, we study adversarial attacks on deep models such as VGG16, ResNet50, and ResNet101, and wide models such as InceptionNetv3 and XceptionNet using Fast Gradient Sign Method (FGSM). We then compute the metrics MOD and VID for the automatic face recognition (AFR) use case with the VGGFace2 dataset. We observe a consistent shift in the region highlighted in the Grad-CAM heatmap, reflecting its participation to the decision making, across all models under adversarial attacks. The proposed method can be used to understand adversarial attacks and explain the behaviour of black box CNN models for image analysis.

Computer Vision and Pattern Recognition,Cryptography and Security
Adapting Grad-CAM for Embedding Networks

Lei Chen,Jianhui Chen,Hossein Hajimirsadeghi,Greg Mori

DOI: https://doi.org/10.48550/arXiv.2001.06538

2020-01-18

Abstract:The gradient-weighted class activation mapping (Grad-CAM) method can faithfully highlight important regions in images for deep model prediction in image classification, image captioning and many other tasks. It uses the gradients in back-propagation as weights (grad-weights) to explain network decisions. However, applying Grad-CAM to embedding networks raises significant challenges because embedding networks are trained by millions of dynamically paired examples (e.g. triplets). To overcome these challenges, we propose an adaptation of the Grad-CAM method for embedding networks. First, we aggregate grad-weights from multiple training examples to improve the stability of Grad-CAM. Then, we develop an efficient weight-transfer method to explain decisions for any image without back-propagation. We extensively validate the method on the standard CUB200 dataset in which our method produces more accurate visual attention than the original Grad-CAM method. We also apply the method to a house price estimation application using images. The method produces convincing qualitative results, showcasing the practicality of our approach.

Computer Vision and Pattern Recognition
Towards Explainable LiDAR Point Cloud Semantic Segmentation via Gradient Based Target Localization

Abhishek Kuriyal,Vaibhav Kumar

2024-02-19

Abstract:Semantic Segmentation (SS) of LiDAR point clouds is essential for many applications, such as urban planning and autonomous driving. While much progress has been made in interpreting SS predictions for images, interpreting point cloud SS predictions remains a challenge. This paper introduces pGS-CAM, a novel gradient-based method for generating saliency maps in neural network activation layers. Inspired by Grad-CAM, which uses gradients to highlight local importance, pGS-CAM is robust and effective on a variety of datasets (SemanticKITTI, Paris-Lille3D, DALES) and 3D deep learning architectures (KPConv, RandLANet). Our experiments show that pGS-CAM effectively accentuates the feature learning in intermediate activations of SS architectures by highlighting the contribution of each point. This allows us to better understand how SS models make their predictions and identify potential areas for improvement. Relevant codes are available at https://github.com/geoai4cities/pGS-CAM.

Artificial Intelligence,Computer Vision and Pattern Recognition

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Basde Localization

Grad-CAM: Visual explanations from deep networks via gradient-based localization. arXiv 2016

Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization

Grad-CAM: visual explanations from deep networks via gradient-based localization

Statistic-CAM: A Gradient-Free Visual Explanations for Deep Convolutional Network

Exclusive Feature Constrained Class Activation Mapping for Better Visual Explanation.

Grad-CAM: Why did you say that?

Integrated Grad-CAM: Sensitivity-Aware Visual Explanation of Deep Convolutional Networks via Integrated Gradient-Based Scoring

Is Grad-CAM Explainable in Medical Images?

Grad++ScoreCAM: Enhancing Visual Explanations of Deep Convolutional Networks Using Incremented Gradient and Score- Weighted Methods

Recipro-CAM: Fast gradient-free visual explanations for convolutional neural networks

Cross-CAM: Focused Visual Explanations for Deep Convolutional Networks Via Training-Set Tracing

Generalizing GradCAM for Embedding Networks

Eigen-CAM: Visual Explanations for Deep Convolutional Neural Networks

Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks

Shap-CAM: Visual Explanations for Convolutional Neural Networks Based on Shapley Value.

Generalizing Adversarial Explanations with Grad-CAM

Adapting Grad-CAM for Embedding Networks

Towards Explainable LiDAR Point Cloud Semantic Segmentation via Gradient Based Target Localization