Abstract:We propose a technique for producing 'visual explanations' for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable. Our approach—Gradient-weighted Class Activation Mapping (Grad-CAM), uses the gradients of any target concept (say 'dog' in a classification network or a sequence of words in captioning network) flowing into the final convolutional layer to produce a coarse localization map highlighting the important regions in the image for predicting the concept. Unlike previous approaches, Grad-CAM is applicable to a wide variety of CNN model-families: (1) CNNs with fully-connected layers (e.g.VGG), (2) CNNs used for structured outputs (e.g.captioning), (3) CNNs used in tasks with multi-modal inputs (e.g.visual question answering) or reinforcement learning, all without architectural changes or re-training. We combine Grad-CAM with existing fine-grained visualizations to create a high-resolution class-discriminative visualization, Guided Grad-CAM, and apply it to image classification, image captioning, and visual question answering (VQA) models, including ResNet-based architectures. In the context of image classification models, our visualizations (a) lend insights into failure modes of these models (showing that seemingly unreasonable predictions have reasonable explanations), (b) outperform previous methods on the ILSVRC-15 weakly-supervised localization task, (c) are robust to adversarial perturbations, (d) are more faithful to the underlying model, and (e) help achieve model generalization by identifying dataset bias. For image captioning and VQA, our visualizations show that even non-attention based models learn to localize discriminative regions of input image. We devise a way to identify important neurons through Grad-CAM and combine it with neuron names (Bau et al. in Computer vision and pattern recognition, 2017) to provide textual explanations for model decisions. Finally, we design and conduct human studies to measure if Grad-CAM explanations help users establish appropriate trust in predictions from deep networks and show that Grad-CAM helps untrained users successfully discern a 'stronger' deep network from a 'weaker' one even when both make identical predictions. Our code is available at <a href="https://github.com/ramprs/grad-cam/">https://github.com/ramprs/grad-cam/</a>, along with a demo on CloudCV (Agrawal et al., in: Mobile cloud visual media computing, pp 265–290. Springer, 2015) (<a href="http://gradcam.cloudcv.org">http://gradcam.cloudcv.org</a>) and a video at <a href="http://youtu.be/COjUB9Izk6E">http://youtu.be/COjUB9Izk6E</a>.

Simulating Realistic MRI variations to Improve Deep Learning model and visual explanations using GradCAM

Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50

Deep neural models for automated multi-task diagnostic scan management—quality enhancement, view classification and report generation

PILOT: Physics-Informed Learned Optimized Trajectories for Accelerated MRI

Is visual explanation with Grad-CAM more reliable for deeper neural networks? a case study with automatic pneumothorax diagnosis

Is Grad-CAM Explainable in Medical Images?

Refining neural network algorithms for accurate brain tumor classification in MRI imagery

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

Synthetic Brain Images: Bridging the Gap in Brain Mapping With Generative Adversarial Model

Respond-CAM: Analyzing Deep Models for 3D Imaging Data by Visualizations

Enhancing EfficientNetv2 with global and efficient channel attention mechanisms for accurate MRI-Based brain tumor classification

MambaRecon: MRI Reconstruction with Structured State Space Models

An XAI-enhanced efficientNetB0 framework for precision brain tumor detection in MRI imaging

Doctor's Dilemma: Evaluating an Explainable Subtractive Spatial Lightweight Convolutional Neural Network for Brain Tumor Diagnosis

MCE: Medical Cognition Embedded in 3D MRI feature extraction for advancing glioma staging

Brain MR image simulation for deep learning based medical image analysis networks

Fast and Robust Detection of Anatomical Landmarks Using Cascaded 3D Convolutional Networks Guided by Linear Square Regression.

Leveraging 2D Deep Learning ImageNet-trained models for Native 3D Medical Image Analysis

Data and Physics Driven Learning Models for Fast MRI -- Fundamentals and Methodologies from CNN, GAN to Attention and Transformers

Learning to Synthesize 7 T MRI from 3 T MRI with Few Data by Deformable Augmentation.

GAN-TL: Generative Adversarial Networks with Transfer Learning for MRI Reconstruction