Abstract:In this paper, we present a Neuron Abandoning Attention Flow (NAFlow) method to address the open problem of visually explaining the attention evolution dynamics inside CNNs when making their classification decisions. A novel cascading neuron abandoning back-propagation algorithm is designed to trace neurons in all layers of a CNN that involve in making its prediction to address the problem of significant interference from abandoned neurons. Firstly, a Neuron Abandoning Back-Propagation (NA-BP) module is proposed to generate Back-Propagated Feature Maps (BPFM) by using the inverse function of the intermediate layers of CNN models, on which the neurons not used for decision-making are abandoned. Meanwhile, the cascading NA-BP modules calculate the tensors of importance coefficients which are linearly combined with the tensors of BPFMs to form the NAFlow. Secondly, to be able to visualize attention flow for similarity metric-based CNN models, a new channel contribution weights module is proposed to calculate the importance coefficients via Jacobian Matrix. The effectiveness of the proposed NAFlow is validated on nine widely-used CNN models for various tasks of general image classification, contrastive learning classification, few-shot image classification, and image retrieval.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: how to correctly visualize the evolution process of the internal attention mechanism in convolutional neural networks (CNNs), especially during the classification decision - making process. Existing visual explanation methods can only generate attention maps of the final output layer and cannot effectively explain the attention evolution of intermediate layers, because these methods will include a large amount of irrelevant neuron information that does not participate in decision - making when generating attention maps, resulting in inaccurate explanations. Specifically, this paper proposes a new method - Neuron Abandoning Attention Flow (NAFlow) to solve the following problems: 1. **Limitations of existing methods**: Existing visual explanation methods (such as Grad - CAM, Layer - CAM, etc.) are inaccurate and have significant interference when generating attention maps of intermediate layers because they use all neurons (including "abandoned" neurons that do not participate in decision - making). 2. **Correct generation of intermediate - layer attention maps**: In order to correctly generate attention maps of intermediate layers, a method that can identify and exclude neurons that do not participate in the final classification decision is required. This helps to more clearly show the attention evolution process of CNNs at different levels. 3. **Applicable to CNN models based on similarity metrics**: For CNN models that perform classification based on similarity metrics (such as models for few - shot image classification, contrastive learning classification, and image retrieval), existing explanation methods cannot effectively generate attention maps. Therefore, a new method is needed to solve this problem. To solve the above problems, the paper proposes the following innovations: - **Neuron Abandoning Back - Propagation (NA - BP)**: Through the back - propagation algorithm, back - track layer by layer starting from the final output layer to generate Back - Propagated Feature Maps (BPFMs), and exclude neurons that do not participate in decision - making during this process. - **Importance coefficient calculation**: Calculate the importance coefficient of each layer through Jacobian matrix operations, and linearly combine these coefficients with BPFMs to form the final NAFlow. - **Channel contribution weight method**: For CNN models based on similarity metrics, a new channel contribution weight method is proposed, which can generate attention maps of all internal layers of such models for the first time. Through these innovations, the NAFlow method can more accurately explain the attention evolution of CNNs during the classification decision - making process, thereby improving the interpretability and transparency of the model.

Neuron Abandoning Attention Flow: Visual Explanation of Dynamics inside CNN Models

Attention Backpropagation

IM-LIF: Improved Neuronal Dynamics with Attention Mechanism for Direct Training Deep Spiking Neural Network

Top-Down Neural Attention by Excitation Backprop

Loss-Based Attention for Interpreting Image-Level Prediction of Convolutional Neural Networks.

Neural Consciousness Flow

Saliency Prediction on Omnidirectional Images with Brain-Like Shallow Neural Network.

GCA-Net: A Global Context Aggregation Network for Effective Optical Flow

AM3F-FlowNet: Attention-Based Multi-Scale Multi-Branch Flow Network

Multi-attention Guided Activation Propagation in CNNs.

Attention Flows: Analyzing and Comparing Attention Mechanisms in Language Models

Interpretable local flow attention for multi-step traffic flow prediction

Optical Flow Estimation Using a Non-Local Convolutional Network.

Understanding contributing neurons via attribution visualization

Understanding Neural Network Through Neuron Level Visualization.

Attention Branch Network: Learning of Attention Mechanism for Visual Explanation

Attention Graph: Learning Effective Visual Features for Large-Scale Image Classification

FPCR-Net: Feature pyramidal correlation and residual reconstruction for optical flow estimation

Towards Better Analysis of Deep Convolutional Neural Networks

FlowNAS: Neural Architecture Search for Optical Flow Estimation

ReFlowNet: Revisiting Coarse-to-fine Learning of Optical Flow