Abstract:Automated multi-label chest X-ray (CXR) image classification has recently made significant progress in clinical diagnosis based on the advanced deep learning techniques. However, most existing methods mainly focus on analyzing locality visual cues from a single image but fail to leverage the underlying explicit correlations among different images for precise disease diagnosis. By contrast, an experienced radiologist expertizes in transferring knowledge from previous tasks to diagnose the present radiograph. To enable the machine like a radiologist, this paper proposes a novel Semantic Similarity Graph Embedding (SSGE) framework, which explicitly explores the semantic similarities among images to optimize the visual feature embedding for improving the performance of multi-label CXR images classification. Specifically, the proposed SSGE framework contains three main components: the image feature embedding (IFE) module, similarity graph construction (SGC) module, and semantic similarity learning (SSL) module. To realize interactive teaching and learning between visual and semantic information, the proposed SSGE framework is built on the "Teacher-Student" (semantic-visual) learning mechanism. With the guidance and supervision of the cross-image similarity graph generated by the SGC module, the SSL module leverages Graph Convolutional Network (GCN) to adaptively recalibrate the multi-image feature representations extracted from the IFE module, which guarantees their semantic consistency. Furthermore, we propose a novel re-weighting strategy to learn a more optimal semantic-similarity graph for the information propagation of the GCN layers. Extensive experiments on two benchmark datasets demonstrate the effectiveness of the proposed method in comparison with some state-of-the-art baselines.

Multi-Label Learning With Visual-Semantic Embedded Knowledge Graph for Diagnosis of Radiology Imaging

Multi-Label Chest X-Ray Image Classification via Semantic Similarity Graph Embedding

K-Diag: Knowledge-enhanced Disease Diagnosis in Radiographic Imaging

Label Semantic Improvement with Graph Convolutional Networks for Multi-Label Chest X-Ray Image Classification

Multi-label local awareness and global co-occurrence priori learning improve chest X-ray classification

Modeling Global and Local Label Correlation with Graph Convolutional Networks for Multi-Label Chest X-ray Image Classification

Diagnose Chest Pathology In X-Ray Images By Learning Multi-Attention Convolutional Neural Network

Label Generation Network based on Self-selected Historical Information for Multiple Disease Classification on Chest Radiography.

When Radiology Report Generation Meets Knowledge Graph

Expert Uncertainty and Severity Aware Chest X-Ray Classification by Multi-Relationship Graph Learning

Multi-Label Local to Global Learning: A Novel Learning Paradigm for Chest X-Ray Abnormality Classification.

GCN-RNN: an Unified Framework for Modeling the Multi-Label Diagnosis of Chest X-ray Disease

Vision-knowledge fusion model for multi-domain medical report generation

ImageGCN: Multi-Relational Image Graph Convolutional Networks for Disease Identification with Chest X-rays

Multi-label neural architecture search for chest radiography image classification

Dynamic Multi-Domain Knowledge Networks for Chest X-ray Report Generation

BB-GCN: A Bi-modal Bridged Graph Convolutional Network for Multi-label Chest X-Ray Recognition

Dynamic Graph Correlation Learning for Disease Diagnosis with Incomplete Labels

Knowledge Graph Embeddings for Multi-Lingual Structured Representations of Radiology Reports

Multi-Label Annotation of Chest Abdomen Pelvis Computed Tomography Text Reports Using Deep Learning

Multilabel Classification for Lung Disease Detection: Integrating Deep Learning and Natural Language Processing