Abstract:Recent advancements in deep neural networks have shown promise in aiding disease diagnosis and medical decision-making. However, ensuring transparent decision-making processes of AI models in compliance with regulations requires a comprehensive understanding of the model's internal workings. However, previous methods heavily rely on expensive pixel-wise annotated datasets for interpreting the model, presenting a significant drawback in medical domains. In this paper, we propose a novel medical neuron concept annotation method, named Mask-free Medical Model Interpretation (MAMMI), addresses these challenges. By using a vision-language model, our method relaxes the need for pixel-level masks for neuron concept annotation. MAMMI achieves superior performance compared to other interpretation methods, demonstrating its efficacy in providing rich representations for neurons in medical image analysis. Our experiments on a model trained on NIH chest X-rays validate the effectiveness of MAMMI, showcasing its potential for transparent clinical decision-making in the medical domain. The code is available at <a class="link-external link-https" href="https://github.com/ailab-kyunghee/MAMMI" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the interpretability problem of deep neural networks in medical applications, especially how to explain the internal working mechanism of the model without relying on expensive pixel - level labeled datasets. Specifically, the paper proposes a new method named **Mask - free Medical Model Interpretation (MAMMI)** for explaining neuron concepts in medical image analysis. #### Main problems and challenges: 1. **The need for transparent decision - making**: - Deep neural networks (DNNs) perform well in disease diagnosis and medical decision - making, but in order to ensure compliance with regulatory requirements (such as the General Data Protection Regulation (GDPR) in the European Union), a transparent decision - making process needs to be provided. - This means that the internal working mechanism of the model needs to be understood to ensure that its decisions can be explained and verified. 2. **Limitations of existing methods**: - Previous methods (such as Network Dissection and TSI) rely on expensive pixel - level labeled datasets, which are especially expensive and time - consuming in the medical field. - Data in the medical field usually has the problem of class imbalance, which has a negative impact on the performance of neuron concept annotation methods. 3. **Lack of specific information for the medical field**: - Existing concept sets may not be applicable to the medical field because they are usually composed of general vocabulary and cannot accurately reflect specific concepts in medical images. #### Innovations of the MAMMI method: - **No need for pixel - level labeling**: By using vision - language models (such as CLIP), MAMMI can perform neuron concept annotation without pixel - level labeling, thus reducing the cost of data collection. - **Concept set adapted to the medical field**: Nouns are extracted from medical reports to construct a concept set specifically for the medical field, which improves the relevance and accuracy of model interpretation. - **Adaptive selection of representative images for neurons**: Considering the class imbalance problem in medical data, MAMMI proposes an adaptive method to select representative images, ensuring more accurate concept annotation for each neuron. Through these innovations, MAMMI not only improves the quality of model interpretation but also reduces the dependence on expensive datasets, providing strong support for transparent clinical decision - making in the medical field.

Mask-Free Neuron Concept Annotation for Interpreting Neural Networks in Medical Domain

Deep neural models for automated multi-task diagnostic scan management—quality enhancement, view classification and report generation

Transparent medical image AI via an image–text foundation model grounded in medical literature

Iterative annotation to ease neural network training: Specialized machine learning in medical image analysis

Fostering transparent medical image AI via an image-text foundation model grounded in medical literature

Rethinking masked image modelling for medical image representation

Mask of truth: model sensitivity to unexpected regions of medical images

MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network

Improving Interpretability of Deep Neural Networks in Medical Diagnosis by Investigating the Individual Units

Abstracting Deep Neural Networks into Concept Graphs for Concept Level Interpretability

Brainomaly: Unsupervised Neurologic Disease Detection Utilizing Unannotated T1-weighted Brain MR Images

Understanding Deep Neural Network Predictions for Medical Imaging Applications

Doctor's Dilemma: Evaluating an Explainable Subtractive Spatial Lightweight Convolutional Neural Network for Brain Tumor Diagnosis

Enhancing lung abnormalities diagnosis using hybrid DCNN-ViT-GRU model with explainable AI: A deep learning approach

AIMNet: Adaptive Image-Tag Merging Network For Automatic Medical Report Generation

MEDICAL IMAGING ENHANCEMENT WITH AI MODELS FOR AUTOMATIC DISEASE DETECTION AND CLASSIFICATION BASED ON MEDICAL IMAGES

Optimizing Universal Lesion Segmentation: State Space Model-Guided Hierarchical Networks with Feature Importance Adjustment

RGFSAMNet: An interpretable COVID-19 detection and classification by using the deep residual network with global feature fusion and attention mechanism

Masked Image Modeling Advances 3D Medical Image Analysis

Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation?

MNet-10: A robust shallow convolutional neural network model performing ablation study on medical images assessing the effectiveness of applying optimal data augmentation technique