Topicwise Separable Sentence Retrieval for Medical Report Generation

Junting Zhao,Yang Zhou,Zhihao Chen,Huazhu Fu,Liang Wan

2024-05-07

Abstract:Automated radiology reporting holds immense clinical potential in alleviating the burdensome workload of radiologists and mitigating diagnostic bias. Recently, retrieval-based report generation methods have garnered increasing attention due to their inherent advantages in terms of the quality and consistency of generated reports. However, due to the long-tail distribution of the training data, these models tend to learn frequently occurring sentences and topics, overlooking the rare topics. Regrettably, in many cases, the descriptions of rare topics often indicate critical findings that should be mentioned in the report. To address this problem, we introduce a Topicwise Separable Sentence Retrieval (Teaser) for medical report generation. To ensure comprehensive learning of both common and rare topics, we categorize queries into common and rare types to learn differentiated topics, and then propose Topic Contrastive Loss to effectively align topics and queries in the latent space. Moreover, we integrate an Abstractor module following the extraction of visual features, which aids the topic decoder in gaining a deeper understanding of the visual observational intent. Experiments on the MIMIC-CXR and IU X-ray datasets demonstrate that Teaser surpasses state-of-the-art models, while also validating its capability to effectively represent rare topics and establish more dependable correspondences between queries and topics.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### The Problems This Paper Attempts to Solve

This paper aims to address two key issues in the automatic generation of medical reports:

1. **Long-tail Distribution Problem**: Existing retrieval-based methods tend to retrieve high-frequency sentences from the training set while ignoring low-frequency sentences. However, in many cases, low-frequency sentences often indicate important findings that should be mentioned in the generated report. Therefore, this approach may lead to the omission of critical information.

2. **Unclear Query-to-Topic Correspondence**: Existing methods fail to effectively learn the relationship between queries and different topics, resulting in different queries potentially mapping to the same topic, thus causing repetitive and semantically identical sentences in the report.

To address these issues, the authors propose a method called **Topicwise Separable Sentence Retrieval (Teaser)**. Teaser improves the generation of medical reports through the following approaches:

- **Classifying Queries**: Queries are divided into common and rare queries to retrieve common and rare topics respectively.
- **Contrastive Loss**: Introducing Topic Contrastive Loss (TCL) to ensure that semantically similar queries are closer in the latent space, while dissimilar queries are pushed apart, thereby alleviating the problem of many-to-one or many-to-many mappings.
- **Abstractor Module**: Employing an Abstractor module to compress visual features and reduce noise, enabling better matching of visual and textual findings.

Experimental results show that Teaser outperforms existing state-of-the-art models on the MIMIC-CXR and IU X-ray datasets, validating its ability to effectively represent rare topics.