Inter-rater concordance and operating definitions of radiologic nodal feature assessment in human papillomavirus-positive oropharyngeal carcinoma
Olivia Chin,Laila Alshafai,Brian O'Sullivan,Jie Su,Andrew Hope,Eric Bartlett,Aaron R Hansen,John Waldron,Douglas Chepeha,Wei Xu,Shao Hui Huang,Eugene Yu,Aaron R. Hansen
DOI: https://doi.org/10.1016/j.oraloncology.2022.105716
IF: 5.972
2022-02-01
Oral Oncology
Abstract:BACKGROUND AND PURPOSE: This study aims to evaluate the reliability of radiologic nodal feature assessment in clinical node-positive human papillomavirus-positive oropharyngeal carcinoma.MATERIALS AND METHODS: Baseline CTs or MRIs of clinical node-positive human papillomavirus-positive oropharyngeal carcinoma diagnosed between 2012 and 2015 were reviewed independently by two neuroradiologists for seven nodal features: radiologic nodal involvement, cystic change, presence of necrosis, clustering, conglomeration, coalescence, and extranodal extension. Consensus operating definitions were derived after discussion. The features were re-reviewed in a randomly selected cohort. Levels of certainty (probability of presence: <25%, ∼50%, ∼75%, and >90%) were recorded. Interrater concordance was calculated using Cohen's kappa coefficient.RESULTS: A total of 413 patients (826 necks) were eligible. At initial review, the inter-rater kappa values for: radiologic nodal involvement, cystic change, necrosis, clustering, conglomeration, coalescence, and extranodal extension were 0.92, 0.64, 0.48, 0.32, 0.32, 0.62, and 0.56, respectively. A re-review of 94 randomly selected cases (188 necks) after consolidation of operating definitions for nodal features showed that the inter-rater kappa values of these features were 0.83, 0.62, 0.58, 0.32, 0.18, 0.68, and 0.74 when considering ≥50% certainty as positive, and improved to 0.94, 0.66, 0.59, 0.33, 0.19, 0.76, and 0.86 when considering ≥75% certainty as positive.CONCLUSION: Clearly defined nomenclature results in improved interrater reliability when assessing radiologic nodal features, especially for coalescent adenopathy and extranodal extension. Higher levels of certainty are associated with higher inter-rater agreement. Radiology reporting should include clear definitions of clinically relevant nodal features as well as levels of certainty to serve various needs in clinical care and research.
dentistry, oral surgery & medicine,oncology