The bronchiolitis severity score: An assessment of face validity, construct validity, and interobserver reliability

Shaila Siraj,Wayne Stark,Scott Daniel McKinley,John Michael Morrison,Anthony Alexander Sochet
DOI: https://doi.org/10.1002/ppul.25337
IF: 4.0899
2021-03-08
Pediatric Pulmonology
Abstract:<section class="article-section__content"><h3 class="article-section__sub-title section1"> Objective</h3><p>To assess face validity, interobserver reliability, and the ability to discriminate escalations of care within 24‐hours of admission (late rescues) for the Bronchiolitis Severity Score (BSS) for children hospitalized for acute bronchiolitis.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Hypotheses</h3><p>The BSS will yield variable face validity, have clinically relevant interobserver reliability (kappa &gt;0.7), and distinguish late rescues during hospitalization.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Methods</h3><p>We performed a combined retrospective and prospective, mixed methods study where (1) interobserver agreement was prospectively assessed by overall and subcategory congruence (kappa) calculations, (2) face value were qualitatively assessed from aggregate questionnaire responses, and (3) construct validity for late rescues were assessed using receiver operator characteristic (ROC) curve analyses.</p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Results</h3><p>Face validity, assessed from 39 questionnaire respondents, were generally positive for BSS utility, reliability, and usability. The BSS exhibited weak interobserver reliability (kappa=0.22, 95%CI:0.11‐0.31) calculated from 72 sequential, blinded calculations. Retrospectively, 181 children &lt;2 years of age admitted to the general pediatric ward for acute bronchiolitis from November 2017 to April 2019 were identified of which 18 (9.9%) experienced late rescues. Admission BSS values were no different for children with and without late rescues (6[3,6] vs 4[3,6], <i>P</i>=0.09). An ROC curve analysis revealed an area under the curve of 0.61(95%CI:0.48‐0.75; threshold ≥6 with sensitivity=56%, specificity=69%) for BSS to discriminate late rescues. </p></section><section class="article-section__content"><h3 class="article-section__sub-title section1"> Conclusion</h3><p>Although clinicians expressed favorable perceptions of BSS face and content validity, we noted weak interobserver reliability and limited construct validity. Further development and validation are needed to strengthen the BSS prior to routine use.</p><p>This article is protected by copyright. All rights reserved.</p></section>
pediatrics,respiratory system
What problem does this paper attempt to address?