Abstract:Mapping X-ray images, radiology reports, and other medical data as binary codes in the common space, which can assist clinicians to retrieve pathology-related data from heterogeneous modalities (i.e., hashing-based cross-modal medical data retrieval), provides a new view to promot computeraided diagnosis. Nevertheless, there remains a barrier to boost medical retrieval accuracy: how to reveal the ambiguous semantics of medical data without the distraction of superfluous information. To circumvent this drawback, we propose Deep Supervised Information Bottleneck Hashing (DSIBH), which effectively strengthens the discriminability of hash codes. Specifically, the Deep Deterministic Information Bottleneck (Yu, Yu, and Principe 2021) for single modality is extended to the cross-modal scenario. Benefiting from this, the superfluous information is reduced, which facilitates the discriminability of hash codes. Experimental results demonstrate the superior accuracy of the proposed DSIBH compared with state-of-the-arts in cross-modal medical data retrieval tasks.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is: **How to improve the accuracy of cross - modal medical data retrieval, especially reducing the interference of redundant information on the discriminative ability of hash codes**. Specifically, the author focuses on how to effectively map medical data of different modalities such as X - ray images and radiology reports into binary codes in computer - aided diagnosis (CAD), thus helping clinicians retrieve pathology - related data from heterogeneous modalities. ### Specific description of the problem 1. **Complexity of multi - modal medical data**: With the development of medical technology, medical data includes not only X - ray images but also text data such as radiology reports. These data are large in quantity and diverse in modality. Manual evaluation and diagnosis of diseases are time - consuming and error - prone. 2. **Limitations of existing methods**: - **CAD based on classifiers**: It can only provide diagnosis results and lacks interpretability. - **Content - based image retrieval (CBIR) CAD**: Although it can provide similar images, it is limited to a single modality and cannot fully utilize the advantages of multi - modal data. - **Existing deep supervised hashing (DSH) methods**: Although it can establish semantic relationships, it ignores the influence of redundant information, resulting in limited retrieval accuracy. ### Proposed method To overcome the above problems, the author proposes the **Deep Supervised Information Bottleneck Hashing (DSIBH)** method. This method reduces the interference of redundant information in the hash code learning process by optimizing the information bottleneck principle, thereby enhancing the discriminative ability of hash codes. Specific improvement points include: - **Introducing the information bottleneck principle**: By maximizing the mutual information \( I(G; Y) \) and minimizing the redundant information \( I(G; X) \), ensure that the hash code only retains information related to the semantic label. \[ \max L = I(G; Y)-\beta I(G; X) \] - **Extension to cross - modal scenarios**: Extend the single - modal Deep Deterministic Information Bottleneck (DDIB) to cross - modal scenarios to handle the association between X - ray images and radiology reports. - **Consistency constraint**: Ensure the consistency of hash codes of different modalities through the \( \ell_2 \) loss function. ### Experimental verification The experimental results show that DSIBH performs better than other existing methods on the MIMIC - CXR dataset, especially under hash codes of different bit numbers, the MAP (Mean Average Precision) index is significantly improved. This indicates that DSIBH can more effectively retrieve pathology - related heterogeneous medical data, thereby improving the accuracy of computer - aided diagnosis. ### Conclusion By introducing cross - modal retrieval technology and the information bottleneck principle, DSIBH successfully reduces the interference of redundant information, enhances the discriminative ability of hash codes, and provides a new solution for computer - aided diagnosis of large - scale multi - modal medical data.

Deep Supervised Information Bottleneck Hashing for Cross-modal Retrieval based Computer-aided Diagnosis

Efficient Discrete Supervised Hashing for Large-scale Cross-modal Retrieval

Discrete Similarity Preserving Hashing for Cross-modal Retrieval.

Discrete Cross-Modal Hashing for Efficient Multimedia Retrieval

Nonlinear Discrete Cross-Modal Hashing for Visual-Textual Data

Semantic Consistency Hashing for Cross-Modal Retrieval

Deep Discriminative Hashing for Cross-Modal Hashing Based Computer-Aided Diagnosis

Multi-Manifold Deep Discriminative Cross-Modal Hashing for Medical Image Retrieval

Deep Medical Cross-Modal Attention Hashing.

Deep Adaptively-Enhanced Hashing With Discriminative Similarity Guidance for Unsupervised Cross-Modal Retrieval

Discriminative Supervised Hashing for Cross-Modal similarity Search

Deep Manifold Hashing: A Divide-and-Conquer Approach for Semi-Paired Unsupervised Cross-Modal Retrieval

Order-Sensitive Deep Hashing for Multimorbidity Medical Image Retrieval.

Sequential Discrete Hashing for Scalable Cross-Modality Similarity Retrieval

Pseudo-label driven deep hashing for unsupervised cross-modal retrieval

Deep Binary Reconstruction for Cross-modal Hashing

Deep Self-Supervised Hashing With Fine-Grained Similarity Mining for Cross-Modal Retrieval

Deep Semantic-Alignment Hashing for Unsupervised Cross-Modal Retrieval

Deep Supervised Hashing by Classification for Image Retrieval

Deep noise mitigation and semantic reconstruction hashing for unsupervised cross-modal retrieval

Unsupervised Dual Deep Hashing with Semantic-Index and Content-Code for Cross-Modal Retrieval