Health Care Delivery Innovations—Implementation Matters in Understanding What Works

Teryl K. Nuckols

DOI: https://doi.org/10.1001/jamanetworkopen.2024.3675

2024-04-03

JAMA Network Open

Abstract:Innovations designed to reduce hospital readmissions remain of great interest nationally, and a leading strategy for reducing readmissions is monitoring and managing symptoms after discharge. Many primary care practices place transitional care calls to recently discharged patients to check in and schedule follow-up visits. However, reaching patients by telephone is labor-intensive and challenging. Mobile text-messaging has emerged as an effective, evidence-based approach to monitoring patients and improving health outcomes. 1 ,2 As such, Bressman and colleagues 3 ,4 developed a 2-way texting program that enables primary care practices to send automated transitional care check-in messages, asking "Is there anything we can help you with today?" Patients receive enrollment texts within 2 days after discharge, initial check-in texts within 3 days, and a tapering schedule of 7 total check-in texts over 30 days. The program records patients' responses, categorizes requests for help, and prompts follow-up communication from practice nurses. 3 ,4 The randomized clinical trial by Bressman et al 3 assesses this text message intervention at 30 primary care practices in 1 health care system from 2022 to 2023. At baseline, practice nurses routinely placed transitional care calls within 2 days of discharge. Bressman et al 3 randomized 4736 individual patients to the intervention or usual care. The pragmatic trial used a practice-based context, broad eligibility criteria, automatic enrollment, routinely collected data, and analyses that retained all patients. 5 Among 2352 intervention patients, retention was 90.6%, 79.5% of patients texted back, and patients texted a mean (SD) of 0.72 (1.1) requests for help. Yet there was no difference between the intervention and control groups in 30-day returns to acute care (emergency department visits plus readmissions, 23.9% vs 23.4%; adjusted relative risk, 1.02; 95% CI, 0.92-1.13), or other measures of health care utilization, including primary care visits. 3 Interestingly, Bressman et al 4 evaluated the same texting program in 2020 to 2021 using a rigorous quasi-experimental difference-in-differences study design. At 1 primary care practice, nurses enrolled patients in the texting program during routine transitional care calls. Patients at another practice served as controls. The difference-in-differences approach involved calculating, for each practice site, the change from a baseline period to the intervention period in returns to acute care and then comparing these changes between practice sites. 4 The analysis included 1885 discharges, with 430 enrolled out of 604 eligible for the intervention. Methods resembled the trial in most other ways, including context, baseline conditions, intervention design, eligibility criteria, measures, data sources, and funders. Participant retention exceeded 90%, 83% of participants texted back, and patients texted a mean (SD) of 0.70 (1.04) requests, similar to the randomized clinical trial by Bressman et al. 3 ,4 Relative to the control practice, 30-day returns to acute care declined substantially more at the intervention practice (adjusted odds ratio, 0.59; 95% CI, 0.38-0.92). 4 Why did the studies have such divergent results, despite their similarities? In general, pragmatic randomized clinical trials and quasi-experimental analyses share many features, and both are used to evaluate health care delivery innovations. Pragmatic randomized clinical trials can inform decisions about whether to implement an intervention in clinical practice, rather than determining its effects under ideal circumstances. 5 In contrast, quasi-experimental designs are used to infer causal associations based on observational data, typically when randomization would be challenging. Pragmatic randomized clinical trials are increasingly common in research, whereas quasi-experimental designs are often used for operational quality improvement activities. Using both approaches to evaluate the same intervention is rare, so this pair of studies offers a unique opportunity to compare methods and findings. 3 ,4 One potential explanation for the studies' divergent findings is scientific rigor. The randomized clinical trial by Bressman et al 3 was large, and randomization achieved well-balanced comparison groups, attributes not shared by the difference-in-differences analysis. 4 In that 2020 to 2021 study, 4 people at high risk for readmission were overrepresented at the intervention practice, and, if effectiveness was greater in high-risk populations, this could have exaggerated the intervention's apparent effectiveness. Furthermore, unadjusted readmission rates increased more at the control practice (6.9% to 11.3%) than -Abstract Truncated-

medicine, general & internal