Abstract:This study aimed to assess the performance of a deep learning algorithm in helping radiologist achieve improved efficiency and accuracy in chest radiograph diagnosis. We adopted a deep learning algorithm to concurrently detect the presence of normal findings and 13 different abnormalities in chest radiographs and evaluated its performance in assisting radiologists. Each competing radiologist had to determine the presence or absence of these signs based on the label provided by the AI. The 100 radiographs were randomly divided into two sets for evaluation: one without AI assistance (control group) and one with AI assistance (test group). The accuracy, false-positive rate, false-negative rate, and analysis time of 111 radiologists (29 senior, 32 intermediate, and 50 junior) were evaluated. A radiologist was given an initial score of 14 points for each image read, with 1 point deducted for an incorrect answer and 0 points given for a correct answer. The final score for each doctor was automatically calculated by the backend calculator. We calculated the mean scores of each radiologist in the two groups (the control group and the test group) and calculated the mean scores to evaluate the performance of the radiologists with and without AI assistance. The average score of the 111 radiologists was 597 (587–605) in the control group and 619 (612–626) in the test group ( P < 0.001). The time spent by the 111 radiologists on the control and test groups was 3279 (2972–3941) and 1926 (1710–2432) s, respectively ( P < 0.001). The performance of the 111 radiologists in the two groups was evaluated by the area under the receiver operating characteristic curve (AUC). The radiologists showed better performance on the test group of radiographs in terms of normal findings, pulmonary fibrosis, heart shadow enlargement, mass, pleural effusion, and pulmonary consolidation recognition, with AUCs of 1.0, 0.950, 0.991, 1.0, 0.993, and 0.982, respectively. The radiologists alone showed better performance in aortic calcification (0.993), calcification (0.933), cavity (0.963), nodule (0.923), pleural thickening (0.957), and rib fracture (0.987) recognition. This competition verified the positive effects of deep learning methods in assisting radiologists in interpreting chest X-rays. AI assistance can help to improve both the efficacy and efficiency of radiologists.

A deep learning solution to detect left ventricular structural abnormalities with chest X-rays: towards trustworthy AI in cardiology

Deep learning to detect left ventricular structural abnormalities in chest X-rays

Can Artificial Intelligence Reliably Report Chest X-Rays?: Radiologist Validation of an Algorithm trained on 2.3 Million X-Rays

Can Deep Learning Reliably Recognize Abnormality Patterns on Chest X-rays? A Multi-Reader Study Examining One Month of AI Implementation in Everyday Radiology Clinical Practice

A Deep Learning System for Detecting Cardiomegaly Disease Based on CXR Image

Deep learning improves physician accuracy in the comprehensive detection of abnormalities on chest X-rays

Deep Learning for Chest X-ray Diagnosis: Competition Between Radiologists with or Without Artificial Intelligence Assistance

AI-Driven Thoracic X-ray Diagnostics: Transformative Transfer Learning for Clinical Validation in Pulmonary Radiography

xECGArch: a trustworthy deep learning architecture for interpretable ECG analysis considering short-term and long-term features

Deep Learning-Enabled Assessment of Left Heart Structure and Function Predicts Cardiovascular Outcomes

Deep Chest: an artificial intelligence model for multi-disease diagnosis by chest x-rays

Revealing unforeseen diagnostic image features with deep learning by detecting cardiovascular diseases from apical four-chamber ultrasounds

Enhancing the detection of airway disease by applying deep learning and explainable artificial intelligence

Using Computer Vision Techniques to Automatically Detect Abnormalities in Chest X-rays

A deep learning framework assisted echocardiography with diagnosis, lesion localization, phenogrouping heterogeneous disease, and anomaly detection

AI Models for Chest Radiograph Analysis: Internal Clinical Trial vs. Gold Standards

Deep Learning to Estimate Left Ventricular Ejection Fraction From Routine Coronary Angiographic Images

Detection of subclinical atherosclerosis by image-based deep learning on chest x-ray

Explaining Deep Learning for ECG Analysis: Building Blocks for Auditing and Knowledge Discovery

Evaluation of the Performance of an Artificial Intelligence (AI) Algorithm in Detecting Thoracic Pathologies on Chest Radiographs

Trustworthy deep learning framework for the detection of abnormalities in X-ray shoulder images