Abstract:Abstract Purpose To compare the effects of different annotation strategies on the performance of Radiomics models in identifying COVID-19. Methods A total of 775 CT scans were retrospectively collected from 5 hospitals in China between Jan 19 and Mar 26, 2020, including 310 COVID-19 scans and 465 other community-acquired pneumonia (CAP) scans. Coarse annotation which labels the major lesions on certain CT slides and fine annotation which delineates the contour of lesions on each slide was performed on CT images. Four feature selection methods and four machine learning algorithms were then applied in combinations to develop Radiomics models on different sizes of datasets, including small (56 CT scans) and large (56 + 489 CT scans). Model performance was evaluated by ROC curve, PR curve, and other diagnostic metrics on an external test set. Statistical analyses were performed with Chi-square tests and DeLong Test; P < 0.05 was considered statistically significant. Results Differences between coarse and fine annotated data were quantitatively analyzed by a Dice index of 0.689, an average Hausdorff distance of 3.7%, and an average volume difference of 5.8%. Inaccurate segmentations were observed in coarse annotated images, including relatively smaller ROI and missed delineation of ground-glass opacity. In addition, more abundant features were extracted from fine annotated images in categories of FirstOrder, GLSZM, and GLCM features. With regard to model performance, fine annotation enabled an over better performance of Radiomics models while enlarged dataset size could remedy the influence of coarse annotation. Meanwhile, models trained on large datasets displayed more stable performance on all selection methods and algorithm combinations. Among them, L1-LR-MLP was selected as the optimal combination for modeling. In particular, SDFine, SDRough, LDFine, and LDRough datasets developed L1-LR-MLP models achieved the AUROC of 0.864,0.707, 0.904, and 0.899, and the AUPR of 0.888, 0.714, 0.934 and 0.896, respectively, on the external dataset. Conclusions Fine annotation generally enables a better model performance in the identification of COVID-19 while the efficient coarse annotation strategy could also be applied to achieve the equivalent diagnostic performance by expanding the training dataset, especially in urgent scenarios. L1-LR-MLP displayed great potential to be applied for establishing COVID-19 identification models.

Large-scale screening to distinguish between COVID-19 and community-acquired pneumonia using infection size-aware classification

Large-Scale Screening of COVID-19 from Community Acquired Pneumonia using Infection Size-Aware Classification

Unsupervised Detection of Pulmonary Opacities for Computer-Aided Diagnosis of COVID-19 on CT Images.

Impact of ROIs Delineation Strategies on the Performance of Artificial Intelligence-Aided COVID-19 Screening Algorithms

A deep learning algorithm using CT images to screen for Corona virus disease (COVID-19)

Dual-Sampling Attention Network for Diagnosis of COVID-19 From Community Acquired Pneumonia

Automatic Distinction Between COVID-19 and Common Pneumonia Using Multi-Scale Convolutional Neural Network on Chest CT Scans

Assisting scalable diagnosis automatically via CT images in the combat against COVID-19

A Rapid, Accurate and Machine-Agnostic Segmentation and Quantification Method for CT-Based COVID-19 Diagnosis

Fully automatic pipeline of convolutional neural networks and capsule networks to distinguish COVID-19 from community-acquired pneumonia via CT images

Automatic CT whole-lung segmentation in radiomics discrimination: Methodology and application in pneumonia diagnosis and distinguishment

Lung volume reduction and infection localization revealed in Big data CT imaging of COVID-19

Development and validation of chest CT-based imaging biomarkers for early stage COVID-19 screening

Multi-classifier-based identification of COVID-19 from chest computed tomography using generalizable and interpretable radiomics features

Classification of COVID-19 by Compressed Chest CT Image Through Deep Learning on a Large Patients Cohort

Using Artificial Intelligence to Detect COVID-19 and Community-acquired Pneumonia Based on Pulmonary CT: Evaluation of the Diagnostic Accuracy

Classification of COVID-19 from community-acquired pneumonia: Boosting the performance with capsule network and maximum intensity projection image of CT scans

A radiomics model utilizing CT for the early detection and diagnosis of severe community-acquired pneumonia

Automatically discriminating and localizing COVID-19 from community-acquired pneumonia on chest X-rays

Quantitative CT Imaging Features for COVID-19 Evaluation: the Ability to Differentiate COVID-19 from Non- COVID-19 (highly Suspected) Pneumonia Patients During the Epidemic Period

A Deep Learning System to Screen Novel Coronavirus Disease 2019 Pneumonia