Abstract:IMPORTANCE: The Prostate Imaging Reporting & Data System (PI-RADS) scoring guidelines were developed to address the substantial variation in interpretation and reporting of prostate cancer (PCa) multiparametric MRI (mpMRI) results, and subsequent updates have sought to further improve inter-reader reliability. Nonetheless, the variability of PI-RADS scoring in real-world settings may represent a continuing challenge to the widespread standardization of prostate mpMRI and limit its overall clinical benefit.OBJECTIVE: To assess variability in mpMRI interpretation and reporting of PCa, we evaluated the discrepancies in PI-RADS scoring between community practices and a tertiary academic care center.DESIGN, SETTING, AND PARTICIPANTS: We identified 262 mpMRI studies from nonacademic facilities, reinterpreted by radiologists at our institution between January 2016 and July 2022. Results of targeted MRI fusion biopsy were identified for 193 of these patients, totaling 302 lesions. PI-RADS scoring from both community and academic interpreters were recorded in addition to presence of clinically significant PCa (csPCa) on pathological analysis of targeted cores.MAIN OUTCOME AND MEASURES: The primary outcome was inter-reader reliability via intraclass correlation (ICC) and the kappa statistic. We also assessed the diagnostic accuracy of PI-RADS scoring for detecting csPCa for both cohorts via receiver operator characteristics (ROC) analysis and compared these findings using paired-sample area difference under curve analysis.RESULTS: Inter-reader agreement and reliability of PI-RADS scoring per lesion was generally poor (absolute agreement ICC = 0.393, 95% CI: 0.288-0.488; consistency ICC = 0.407, 95% CI: 0.308-0.497; kappa = 0.336, 95% CI: 0.267-0.406). Reliability results from studies obtained after the publication of PI-RADSv2.1 were similar to those of the overall analysis. No agreement was observed in the subgroup of lesions scored as PIRADS 3 by community interpreters. No statistically significant difference in diagnostic accuracy was observed between cohorts (ROC area under curve [AUC]: 0.759 vs. 0.785, respectively; P = 0.337). PI-RADS 3 was determined to be the optimal cutoff for detecting clinically significant disease in both cohorts.CONCLUSIONS AND RELEVANCE: Our results suggest that mpMRI diagnostic accuracy for detecting csPCa is not significantly different between academic and community practices. However, significantly poor reliability of mpMRI was observed between cohorts, suggesting the risk of introducing practice variation for community prostate cancer management. Variability, particularly for PI-RADS 3 lesions, can lead to inconsistent biopsy recommendations, which may result in missed csPCa or unnecessary biopsies.

Analysis of the cause of discordance between two radiologists on the assessment of radiographic response and progression for subjects enrolled in breast cancer clinical trials employing blinded independent central review.

Blinded Independent Central Review (BICR) in New Therapeutic Lung Cancer Trials

Inter-observer Variability in Chemotherapy Trials for Pancreatic Cancer

RECIST 1.1 assessments variability: a systematic pictorial review of blinded double reads

The value of blinded independent central review (BICR) in recent trials in non-small cell lung cancer (NSCLC).

Discordance Between Angiographic Assessment and Fractional Flow Reserve or Intravascular Ultrasound in Intermediate Coronary Lesions: A Post-hoc Analysis of the FLAVOUR Trial.

Systematic bias between blinded independent central review and local assessment: literature review and analyses of 76 phase III randomised controlled trials in 45 688 patients with advanced solid tumour

Blinded independent central review versus local investigator assessment of PFS in RCTs of immunotherapy in advanced cancers: A systematic review and meta-analysis

Supplementary Table S3 from Proportional Hazards Violations in Phase III Cancer Clinical Trials: A Potential Source of Trial Misinterpretation

Breast imaging reporting and data system (BI-RADS) lexicon for breast MRI: interobserver variability in the description and assignment of BI-RADS category

Blinded, independent, central image review in oncology trials: how the study endpoint impacts the adjudication rate

Exploring the Interobserver Agreement in Computer-Aided Radiologic Tumor Measurement and Evaluation of Tumor Response

Inter-reader variability of imaging biomarkers for therapeutic response in pancreatic ductal adenocarcinoma.

Measurement variability of radiologists when measuring brain tumors

Endoscopy and central reading in inflammatory bowel disease clinical trials: achievements, challenges and future developments

Inter-reader reliability and diagnostic accuracy of PI-RADS scoring between academic and community care networks: How wide is the gap?

18F-FDG PET/CT versus Diagnostic Contrast-Enhanced CT for Follow-Up of Stage IV Melanoma Patients Treated by Immune Checkpoint Inhibitors: Frequency and Management of Discordances over a 3-Year Period in a University Hospital

Endpoint comparison for bone mineral density measurements in North Central Cancer Treatment Group cancer clinical trials N02C1 and N03CC (Alliance)

Towards Treatment Effect Interpretability: A Bayesian Re-analysis of 194,129 Patient Outcomes Across 230 Oncology Trials

Concordance in Molecular Tumor Board Case Reviews in the ASCO TAPUR Study

Comparison of Clinically Adjudicated Versus Flow-Based Adjudication of Revascularization Events in Randomized Controlled Trials.