Abstract:Purpose: This study aims to comprehensively evaluate the quality of radiomics research by examining unique papers from reviews using the radiomics quality score (RQS). Methods: A literature search was conducted in PubMed (last search date: April 14, 2024). Systematic or non-systematic reviews using the RQS to evaluate radiomic studies were potentially included. Exclusion was applied at two levels: first, at the review level, and second, at the study level (i.e., for the individual articles previously evaluated within the reviews). Score-wise and item-wise analyses were performed, along with trend, multivariable, and subgroup analyses based on baseline study characteristics and validation methods. Results: A total of 1574 unique papers (published online between 1999 and 2023) from 89 reviews were included in the final analysis. The median RQS percentage was 31% with an IQR of 25% (25th-75th percentiles, 14-39%). A positive correlation between median RQS percentage and publication year (2014-2023) was found, with Kendall's tau coefficient of 0.908 (p < 0.001), suggesting an improvement in quality over time. The quality of radiomics publications significantly varied according to different subfields of radiology (p < 0.001). Around one-third of the publications (32%) lacked a separate validation set. Papers with internal validation (54%) dominated those with external validation (14%). Higher-quality validation practices were significantly associated with better RQS percentage scores, independent of the validation's effect on the final score. Item-wise analysis revealed significant shortcomings in several areas. Conclusion: Radiomics research quality is low but improving according to RQS. Significant variation exists across radiology subfields. Critical areas were identified for targeted improvement. Clinical relevance statement: Our study shows that the quality of radiomics research is generally low but improving over time, with item-wise analysis highlighting critical areas needing improvement. It also reveals that the quality of radiomics research differs across subfields and validation methods. Key points: Overall quality of radiomics research remains low and highly variable, although a significant positive trend suggests an improvement in quality over time. Considerable variations exist in the quality of radiomics publications across different subfields of radiology and validation types. The item-wise analysis highlights several critical areas requiring attention, emphasizing the need for targeted improvements.

Systematic review of the radiomics quality score applications: an EuSoMII Radiomics Auditing Group Initiative

Reproducibility of radiomics quality score: an intra- and inter-rater reliability study

Current Status and Quality of Radiomics Studies in Lymphoma: a Systematic Review

A Systematic Review of the Current Status and Quality of Radiomics for Glioma Differential Diagnosis

METhodological RadiomICs Score (METRICS): a quality scoring tool for radiomics research endorsed by EuSoMII

Evaluating the quality of radiomics-based studies for endometrial cancer using RQS and METRICS tools

Cardiac CT and MRI radiomics: systematic review of the literature and radiomics quality score assessment

Quality of radiomics research: comprehensive analysis of 1574 unique publications from 89 reviews

A Systematic Review of Radiomics in Osteosarcoma: Utilizing Radiomics Quality Score As a Tool Promoting Clinical Translation

The Main Findings, Challenges and Limitations of the Methodologies

Oncologic Imaging and Radiomics: A Walkthrough Review of Methodological Challenges

Systematic Review on the Association of Radiomics with Tumor Biological Endpoints

Radiomics for clinical decision support in radiation oncology

Current status of magnetic resonance imaging radiomics in hepatocellular carcinoma: A quantitative review with Radiomics Quality Score

Decoding Radiomics: A Step-by-Step Guide to Machine Learning Workflow in Hand-Crafted and Deep Learning Radiomics Studies

CLEAR guideline for radiomics: Early insights into current reporting practices endorsed by EuSoMII

Delta radiomics: a systematic review

Delta radiomics: an updated systematic review

Self-reported checklists and quality scoring tools in radiomics: a meta-research

Quality Assessment of Radiomics Studies on Functional Outcomes After Acute Ischemic Stroke-A Systematic Review

Cone beam computed tomography (CBCT) and megavoltage computed tomography (MVCT)-based radiomics in head and neck cancers: a systematic review and radiomics quality score assessment