Abstract:Purpose: This study aims to comprehensively evaluate the quality of radiomics research by examining unique papers from reviews using the radiomics quality score (RQS). Methods: A literature search was conducted in PubMed (last search date: April 14, 2024). Systematic or non-systematic reviews using the RQS to evaluate radiomic studies were potentially included. Exclusion was applied at two levels: first, at the review level, and second, at the study level (i.e., for the individual articles previously evaluated within the reviews). Score-wise and item-wise analyses were performed, along with trend, multivariable, and subgroup analyses based on baseline study characteristics and validation methods. Results: A total of 1574 unique papers (published online between 1999 and 2023) from 89 reviews were included in the final analysis. The median RQS percentage was 31% with an IQR of 25% (25th-75th percentiles, 14-39%). A positive correlation between median RQS percentage and publication year (2014-2023) was found, with Kendall's tau coefficient of 0.908 (p < 0.001), suggesting an improvement in quality over time. The quality of radiomics publications significantly varied according to different subfields of radiology (p < 0.001). Around one-third of the publications (32%) lacked a separate validation set. Papers with internal validation (54%) dominated those with external validation (14%). Higher-quality validation practices were significantly associated with better RQS percentage scores, independent of the validation's effect on the final score. Item-wise analysis revealed significant shortcomings in several areas. Conclusion: Radiomics research quality is low but improving according to RQS. Significant variation exists across radiology subfields. Critical areas were identified for targeted improvement. Clinical relevance statement: Our study shows that the quality of radiomics research is generally low but improving over time, with item-wise analysis highlighting critical areas needing improvement. It also reveals that the quality of radiomics research differs across subfields and validation methods. Key points: Overall quality of radiomics research remains low and highly variable, although a significant positive trend suggests an improvement in quality over time. Considerable variations exist in the quality of radiomics publications across different subfields of radiology and validation types. The item-wise analysis highlights several critical areas requiring attention, emphasizing the need for targeted improvements.

Self-reported checklists and quality scoring tools in radiomics: a meta-research

CheckList for EvaluAtion of Radiomics research (CLEAR): a step-by-step reporting guideline for authors and reviewers endorsed by ESR and EuSoMII

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII

Current Status and Quality of Radiomics Studies in Lymphoma: a Systematic Review

Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews

Reproducibility of radiomics quality score: an intra- and inter-rater reliability study

Evaluation of the reliability and usability of CARE‐Radiology: A descriptive‐analytic study

Measuring Quality of Reporting in Systematic Reviews of Diagnostic Test Accuracy Studies in Medical Imaging: Comparison of PRISMA‐DTA and PRISMA

CLEAR guideline for radiomics: Early insights into current reporting practices endorsed by EuSoMII

Evaluating the quality of radiomics-based studies for endometrial cancer using RQS and METRICS tools

Reporting checklists in neuroimaging: promoting transparency, replicability, and reproducibility

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

Radiomics of computed tomography and magnetic resonance imaging in renal cell carcinoma-a systematic review and meta-analysis

Radiomics in medical imaging—“how-to” guide and critical reflection

The Quality and Clinical Translation of Radiomics Studies Based on MRI for Predicting Ki-67 Levels in Patients with Breast Cancer

Quality Assessment of Radiomics Studies on Functional Outcomes After Acute Ischemic Stroke-A Systematic Review

Evaluating progress in automatic chest X-ray radiology report generation

Development of ARCADIA: a tool for assessing the quality of peer-review reports in biomedical research

Diagnostic performance of radiomics in prediction of Ki-67 index status in non-small cell lung cancer: A systematic review and meta-analysis

Cone beam computed tomography (CBCT) and megavoltage computed tomography (MVCT)-based radiomics in head and neck cancers: a systematic review and radiomics quality score assessment

Cardiac CT and MRI radiomics: systematic review of the literature and radiomics quality score assessment