Abstract:Chest X-ray (CXR) anatomical abnormality detection aims at localizing and characterising cardiopulmonary radiological findings in the radiographs, which can expedite clinical workflow and reduce observational oversights. Most existing methods attempted this task in either fully supervised settings which demanded costly mass per-abnormality annotations, or weakly supervised settings which still lagged badly behind fully supervised methods in performance. In this work, we propose a co-evolutionary image and report distillation (CEIRD) framework, which approaches semi-supervised abnormality detection in CXR by grounding the visual detection results with text-classified abnormalities from paired radiology reports, and vice versa. Concretely, based on the classical teacher-student pseudo label distillation (TSD) paradigm, we additionally introduce an auxiliary report classification model, whose prediction is used for report-guided pseudo detection label refinement (RPDLR) in the primary vision detection task. Inversely, we also use the prediction of the vision detection model for abnormality-guided pseudo classification label refinement (APCLR) in the auxiliary report classification task, and propose a co-evolution strategy where the vision and report models mutually promote each other with RPDLR and APCLR performed alternatively. To this end, we effectively incorporate the weak supervision by reports into the semi-supervised TSD pipeline. Besides the cross-modal pseudo label refinement, we further propose an intra-image-modal self-adaptive non-maximum suppression, where the pseudo detection labels generated by the teacher vision model are dynamically rectified by high-confidence predictions by the student. Experimental results on the public MIMIC-CXR benchmark demonstrate CEIRD's superior performance to several up-to-date weakly and semi-supervised methods.

Unlocking the Potential of Weakly Labeled Data: A Co-Evolutionary Learning Framework for Abnormality Detection and Report Generation

Point Beyond Class: A Benchmark for Weakly Semi-supervised Abnormality Localization in Chest X-Rays

You've Got Two Teachers: Co-evolutionary Image and Report Distillation for Semi-supervised Anatomical Abnormality Detection in Chest X-ray

A Model-Agnostic Framework for Universal Anomaly Detection of Multi-organ and Multi-modal Images

An Inclusive Task-Aware Framework for Radiology Report Generation

Anatomy-Guided Weakly-Supervised Abnormality Localization in Chest X-rays

Weakly Supervised Anomaly Detection for Chest X-Ray Image

Improving Radiology Report Generation with Multi-Grained Abnormality Prediction

Automated Radiological Report Generation For Chest X-Rays With Weakly-Supervised End-to-End Deep Learning

Anomaly Detection in Chest X-rays Based on Dual-Attention Mechanism and Multi-Scale Feature Fusion

Dual-distribution discrepancy with self-supervised refinement for anomaly detection in medical images

Dual-Distribution Discrepancy for Anomaly Detection in Chest X-Rays

Multi-task weak supervision enables anatomically-resolved abnormality detection in whole-body FDG-PET/CT

Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes

Learning Visual-Semantic Embeddings for Reporting Abnormal Findings on Chest X-rays

A label information fused medical image report generation framework

Unsupervised Disease Tags for Automatic Radiology Report Generation

Supervised and unsupervised language modelling in Chest X-Ray radiological reports

Learning Hierarchical Attention for Weakly-Supervised Chest X-Ray Abnormality Localization and Diagnosis.

Multi-task Learning for Chest X-ray Abnormality Classification on Noisy Labels

Abnormal Chest X-Ray Identification With Generative Adversarial One-Class Classifier