Feature Activation Map: Visual Explanation of Deep Learning Models for Image Classification

Yi Liao,Yongsheng Gao,Weichuan Zhang

2023-07-11

Abstract:Decisions made by convolutional neural networks(CNN) can be understood and explained by visualizing discriminative regions on images. To this end, Class Activation Map (CAM) based methods were proposed as powerful interpretation tools, making the prediction of deep learning models more explainable, transparent, and trustworthy. However, all the CAM-based methods (e.g., CAM, Grad-CAM, and Relevance-CAM) can only be used for interpreting CNN models with fully-connected (FC) layers as a classifier. It is worth noting that many deep learning models classify images without FC layers, e.g., few-shot learning image classification, contrastive learning image classification, and image retrieval tasks. In this work, a post-hoc interpretation tool named feature activation map (FAM) is proposed, which can interpret deep learning models without FC layers as a classifier. In the proposed FAM algorithm, the channel-wise contribution weights are derived from the similarity scores between two image embeddings. The activation maps are linearly combined with the corresponding normalized contribution weights, forming the explanation map for visualization. The quantitative and qualitative experiments conducted on ten deep learning models for few-shot image classification, contrastive learning image classification and image retrieval tasks demonstrate the effectiveness of the proposed FAM algorithm.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning,Performance

What problem does this paper attempt to address?

### Problems Addressed by the Paper

This paper aims to address the limitations of existing Class Activation Map (CAM)-based methods in explaining deep learning models without fully connected (FC) layers. Specifically:

1. **Limitations of Existing CAM Methods**:
- Existing CAM methods (such as CAM, Grad-CAM, Relevance-CAM, etc.) can only be used to explain convolutional neural network (CNN) models that include fully connected layers as classifiers.
- These methods rely on FC layers to generate explanation maps, thereby limiting their application scope, especially in tasks that do not use FC layers, such as few-shot learning, contrastive learning, and image retrieval tasks.

2. **Proposed New Method**:
- The paper proposes a new post-processing explanation tool called Feature Activation Map (FAM), which can explain deep learning models that do not use FC layers as classifiers.
- The FAM algorithm calculates the contribution weights of each channel in the feature map and linearly combines these weights with the normalized activation map to form a visual explanation map.
- This method is applicable to various deep learning models, including those used in few-shot learning, contrastive learning, and image retrieval tasks.

3. **Experimental Validation**:
- The paper conducts both quantitative and qualitative experiments on multiple datasets to validate the effectiveness of the FAM algorithm.
- Experimental results show that the FAM algorithm can accurately highlight key areas of the target object and has high localization capability and fidelity.

By proposing the FAM algorithm, the paper fills the gap in existing CAM methods for explaining models without FC layers, providing a new solution for the interpretability of deep learning models.