Abstract:Our study investigates the effects of heterogeneity in image parameters on the reproducibility of prognostic performance of models built using radiomic biomarkers. We compare the prognostic performance of models derived from the heterogeneity-mitigated features with that of models obtained from raw features, to assess whether reproducibility of prognostic scores improves upon application of our methods. We used two datasets: The Breast I-SPY1 dataset-Baseline DCE-MRI scans of 156 women with locally advanced breast cancer, treated with neoadjuvant chemotherapy, publicly available via The Cancer Imaging Archive (TCIA); The NSCLC IO dataset-Baseline CT scans of 107 patients with stage 4 non-small cell lung cancer (NSCLC), treated with pembrolizumab immunotherapy at our institution. Radiomic features (n = 102) are extracted from the tumor ROIs. We use a variety of resampling and harmonization scenarios to mitigate the heterogeneity in image parameters. The patients were divided into groups based on batch variables. For each group, the radiomic phenotypes are combined with the clinical covariates into a prognostic model. The performance of the groups is assessed using the c-statistic, derived from a Cox proportional hazards model fitted on all patients within a group. The heterogeneity-mitigation scenario (radiomic features, derived from images that have been resampled to minimum voxel spacing, are harmonized using the image acquisition parameters as batch variables) gave models with highest prognostic scores (for e.g., IO dataset; batch variable: high kernel resolution-c-score: 0.66). The prognostic performance of patient groups is not comparable in case of models built using non-heterogeneity mitigated features (for e.g., I-SPY1 dataset; batch variable: small pixel spacing-c-score: 0.54, large pixel spacing-c-score: 0.65). The prognostic performance of patient groups is closer in case of heterogeneity-mitigated scenarios (for e.g., scenario-harmonize by voxel spacing parameters: IO dataset; thin slice-c-score: 0.62, thick slice-c-score: 0.60). Our results indicate that accounting for heterogeneity in image parameters is important to obtain more reproducible prognostic scores, irrespective of image site or modality. For non-heterogeneity mitigated models, the prognostic scores are not comparable across patient groups divided based on batch variables. This study can be a step in the direction of constructing reproducible radiomic biomarkers, thus increasing their application in clinical decision making.

Understanding Sources of Variation to Improve the Reproducibility of Radiomics

Enhancing the Clinical Utility of Radiomics: Addressing the Challenges of Repeatability and Reproducibility in CT and MRI

Radiomics feature reproducibility under inter-rater variability in segmentations of CT images

Repeatability and Reproducibility of Computed Tomography Radiomics for Pulmonary Nodules

Radiomics reproducibility in computed tomography through changes of ROI size, resolution, and hounsfield unit: A phantom study

Repeatability and Reproducibility of Computed Tomography Radiomics for Pulmonary Nodules A Multicenter Phantom Study

Open-radiomics: A Collection of Standardized Datasets and a Technical Protocol for Reproducible Radiomics Machine Learning Pipelines

Feature selection methods and predictive models in CT lung cancer radiomics

Assessing the Reproducibility, Stability, and Biological Interpretability of Multimodal Computed Tomography Image Features for Prognosis in Advanced Non‐small Cell Lung Cancer

Radiomics: the process and the challenges

CT and MRI radiomics of bone and soft-tissue sarcomas: a systematic review of reproducibility and validation strategies

Resampling and harmonization for mitigation of heterogeneity in image parameters of baseline scans

Repeatability of radiomics studies in colorectal cancer: a systematic review

Could the underlying biological basis of prognostic radiomics and deep learning signatures be explored in patients with lung cancer? A systematic review

PET radiomics in lung cancer: advances and translational challenges

[Research Advances and Obstacles of CT-based Radiomics in Diagnosis and Treatment of Lung Cancer].

Radiomics as a personalized medicine tool in lung cancer: Separating the hope from the hype

Effects of interobserver and interdisciplinary segmentation variabilities on CT-based radiomics for pancreatic cancer

Quantifying the reproducibility and longitudinal repeatability of radiomics features in magnetic resonance Image-Guide accelerator Imaging: A phantom study

SU-D-207B-01: Radiomics Feature Reproducibility from Repeat CT Scans of Patients with Rectal Cancer

Reproducibility of Radiomics Features Derived from Intravoxel Incoherent Motion Diffusion-Weighted MRI of Cervical Cancer.