Local Contrastive Learning for Medical Image Recognition

S. A. Rizvi,R. Tang,X. Jiang,X. Ma,X. Hu

2023-03-25

Abstract:The proliferation of Deep Learning (DL)-based methods for radiographic image analysis has created a great demand for expert-labeled radiology data. Recent self-supervised frameworks have alleviated the need for expert labeling by obtaining supervision from associated radiology reports. These frameworks, however, struggle to distinguish the subtle differences between different pathologies in medical images. Additionally, many of them do not provide interpretation between image regions and text, making it difficult for radiologists to assess model predictions. In this work, we propose Local Region Contrastive Learning (LRCLR), a flexible fine-tuning framework that adds layers for significant image region selection as well as cross-modality interaction. Our results on an external validation set of chest x-rays suggest that LRCLR identifies significant local image regions and provides meaningful interpretation against radiology text while improving zero-shot performance on several chest x-ray medical findings.

Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

This paper attempts to address several key issues in medical image recognition:

1. **Reducing the need for expert-labeled data**: Although existing self-supervised frameworks obtain supervision signals by associating radiology reports, they still struggle to distinguish subtle differences between different pathologies in medical images.
2. **Improving model interpretability**: Many existing frameworks fail to provide explanations between image regions and text, making it difficult for radiologists to evaluate model predictions.
3. **Enhancing local feature selection capability**: Medical observations often appear in small regions of the image, and existing methods often fail to effectively highlight these local features.
4. **Compatibility and flexibility**: Existing frameworks are usually standalone architectures and are not compatible with other pre-trained models.

To address the above issues, the authors propose Local Region Contrastive Learning (LRCLR), a flexible fine-tuning framework that improves zero-shot classification performance and provides meaningful explanations through significant image region selection and cross-modal interaction layers. Experimental results show that LRCLR achieves better zero-shot classification performance on multiple rare chest X-ray findings and demonstrates interpretability results between image regions and radiology report texts.