Abstract:Background: Medical images have already become an essential tool for the diagnosis of many diseases. Thus a large number of medical images are being generated due to the daily routine inspection. An efficient image-based disease retrieval system will not only make full use of existing data, but also help physicians to prognosis the diseases. Medical image retrieval is represented by the classification and localization of common thorax diseases in x-ray images. Although extensive efforts have been put into this field, there are still many challenges. Purpose: Most of the existing fine-grained image research methods just apply existing deep learning frameworks in extracting the image features. However, these high-level features mainly focus on the global representations of the object, rather than simultaneously considering the local ones. It requires fine-grained details to classify the images with similar lesion areas. Thus, it is necessary to combine the global features and local ones to make the features more discriminative. On the other hand, training CNN models based on current existing strategies have a high time complexity, and is hard to get the discriminative features mentioned above. In addition, the visual retrieval method of fine-grained medical images still has the problem of insufficient sample data with accurate annotation information. Methods: To address above challenges, we introduced a novel fine-grained medical images retrieval method. First, a centralized contrastive loss (CCLoss) is proposed as our metric learning loss function. Parameters are updated by using the center point, which not only improves the distinguishing performance of features, but also effectively reduces the time complexity of the algorithm. In addition, a weakly supervised progressive feature extraction method is proposed to gradually extract the combined features. And the attention mechanism module is applied to screen the target information after the initial positioning for fine refinement, so as to separate the features with a high degree of discrimination. The retrieval of 14 different chest diseases is evaluated on the chest x-ray datasets. Results: Compared with the existing research methods, the proposed method shows a better retrieval result for Recall@8 by 2.26 % ∼ 4.6 % $\%{\sim }4.6\%$ and achieves a very efficient training speed which is 100 times faster than the pair-wise loss-based training strategy. We also assessed the effects of Recall@k (k = 2, 4, 6, 8) for progressive features extracted from different steps to obtain a model with the best retrieval performance. Conclusions: The proposed model is capable of learning discriminative representations from chest x-ray datasets, and it achieves better performance compared with other state-of-the-art methods. Therefore, the developed model would be useful in the diagnosis of common thorax disease or unknown chest disease.

GLAN: Global Local Attention Network for Thoracic Disease Classification to Enhance Heart Failure Diagnosis

CasCRNN-GL-Net: Cascaded Convolutional and Recurrent Neural Networks with Global and Local Pathways for Classification of Focal Liver Lesions in Multi-Phase CT Images

Residual Convolutional Neural Networks With Global And Local Pathways For Classification Of Focal Liver Lesions

Combining Convolutional and Recurrent Neural Networks for Classification of Focal Liver Lesions in Multi-phase CT Images.

Thorax Disease Classification with Attention Guided Convolutional Neural Network.

Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classification

Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographs

Multi-attention Network for Thoracic Disease Classification and Localization

Weighing features of lung and heart regions for thoracic disease classification

Learning group-wise spatial attention and label dependencies for multi-task thoracic disease classification

Identifying cardiomegaly in chest x-rays using dual attention network

Anatomy-XNet: An Anatomy Aware Convolutional Neural Network for Thoracic Disease Classification in Chest X-rays

A Cascade Attention Network for Liver Lesion Classification in Weakly-Labeled Multi-phase CT Images.

SDFN: Segmentation-based deep fusion network for thoracic disease classification in chest X-ray images

Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rays

A Weakly Supervised Adaptive DenseNet for Classifying Thoracic Diseases and Identifying Abnormalities

Part-Aware Mask-Guided Attention for Thorax Disease Classification

Automated thorax disease diagnosis using multi-branch residual attention network

Multi-disease X-ray Image Classification of the Chest Based on Global and Local Fusion Adaptive Networks

Centralized contrastive loss with weakly supervised progressive feature extraction for fine-grained common thorax disease retrieval in chest x-ray

Spatial Non-Local Attention for Thoracic Disease Diagnosis and Visualisation in Weakly Supervised Learning.