Abstract:IMPORTANCE: The Prostate Imaging Reporting & Data System (PI-RADS) scoring guidelines were developed to address the substantial variation in interpretation and reporting of prostate cancer (PCa) multiparametric MRI (mpMRI) results, and subsequent updates have sought to further improve inter-reader reliability. Nonetheless, the variability of PI-RADS scoring in real-world settings may represent a continuing challenge to the widespread standardization of prostate mpMRI and limit its overall clinical benefit.OBJECTIVE: To assess variability in mpMRI interpretation and reporting of PCa, we evaluated the discrepancies in PI-RADS scoring between community practices and a tertiary academic care center.DESIGN, SETTING, AND PARTICIPANTS: We identified 262 mpMRI studies from nonacademic facilities, reinterpreted by radiologists at our institution between January 2016 and July 2022. Results of targeted MRI fusion biopsy were identified for 193 of these patients, totaling 302 lesions. PI-RADS scoring from both community and academic interpreters were recorded in addition to presence of clinically significant PCa (csPCa) on pathological analysis of targeted cores.MAIN OUTCOME AND MEASURES: The primary outcome was inter-reader reliability via intraclass correlation (ICC) and the kappa statistic. We also assessed the diagnostic accuracy of PI-RADS scoring for detecting csPCa for both cohorts via receiver operator characteristics (ROC) analysis and compared these findings using paired-sample area difference under curve analysis.RESULTS: Inter-reader agreement and reliability of PI-RADS scoring per lesion was generally poor (absolute agreement ICC = 0.393, 95% CI: 0.288-0.488; consistency ICC = 0.407, 95% CI: 0.308-0.497; kappa = 0.336, 95% CI: 0.267-0.406). Reliability results from studies obtained after the publication of PI-RADSv2.1 were similar to those of the overall analysis. No agreement was observed in the subgroup of lesions scored as PIRADS 3 by community interpreters. No statistically significant difference in diagnostic accuracy was observed between cohorts (ROC area under curve [AUC]: 0.759 vs. 0.785, respectively; P = 0.337). PI-RADS 3 was determined to be the optimal cutoff for detecting clinically significant disease in both cohorts.CONCLUSIONS AND RELEVANCE: Our results suggest that mpMRI diagnostic accuracy for detecting csPCa is not significantly different between academic and community practices. However, significantly poor reliability of mpMRI was observed between cohorts, suggesting the risk of introducing practice variation for community prostate cancer management. Variability, particularly for PI-RADS 3 lesions, can lead to inconsistent biopsy recommendations, which may result in missed csPCa or unnecessary biopsies.

Blinded Independent Central Review (BICR) in New Therapeutic Lung Cancer Trials

Analysis of the cause of discordance between two radiologists on the assessment of radiographic response and progression for subjects enrolled in breast cancer clinical trials employing blinded independent central review.

RECIST 1.1 assessments variability: a systematic pictorial review of blinded double reads

The value of blinded independent central review (BICR) in recent trials in non-small cell lung cancer (NSCLC).

Blinded independent central review versus local investigator assessment of PFS in RCTs of immunotherapy in advanced cancers: A systematic review and meta-analysis

Endoscopy and central reading in inflammatory bowel disease clinical trials: achievements, challenges and future developments

Inter-observer Variability in Chemotherapy Trials for Pancreatic Cancer

Systematic bias between blinded independent central review and local assessment: literature review and analyses of 76 phase III randomised controlled trials in 45 688 patients with advanced solid tumour

Assessing the Validity of a a Priori Patient-Trial Generalizability Score Using Real-world Data from a Large Clinical Data Research Network: A Colorectal Cancer Clinical Trial Case Study.

Inter-reader variability of imaging biomarkers for therapeutic response in pancreatic ductal adenocarcinoma.

Inter-reader reliability and diagnostic accuracy of PI-RADS scoring between academic and community care networks: How wide is the gap?

Incorporation of Eye-Tracking and Gaze Feedback to Characterize and Improve Radiologist Search Patterns of Chest X-rays: A Randomized Controlled Clinical Trial

Assessing the quality of reports of randomized clinical trials: Is blinding necessary?

Blinded, independent, central image review in oncology trials: how the study endpoint impacts the adjudication rate

Seeing the Truth About Double Blinding

A Multicenter Assessment of Interreader Reliability of LI-RADS Version 2018 for MRI and CT

Risk of Bias in Liver Imaging Reporting and Data System Studies Using QUADAS-2

Suboptimal reliability of liver biopsy evaluation has implications for randomized clinical trials

<i>Implementing Real-World RECIST-based Tumor Response Assessment in Patients With Metastatic Non-small Cell Lung Cancer</i> Comment

[Data Quality in Clinical Trials: the Role of Blind Review].