Adopting the RE-AIM analytic framework for rural program evaluation: experiences from the Advance Care Planning via Group Visits (ACP-GV) national evaluation
Monica M Matthieu,Laura D Taylor,David A Adkins,J Silas Williams,Bo Hu,Ciara M Oliver,Jane Ann McCullough,Mary J Mallory,Ian D Smith,Jacob T Painter,Songthip T Ounpraseuth,Kimberly K Garner
DOI: https://doi.org/10.3389/frhs.2024.1210166
2024-03-25
Abstract:Introduction: To support rigorous evaluation across a national portfolio of grants, the United States Department of Veterans Affairs (VA) Office of Rural Health (ORH) adopted an analytic framework to guide their grantees' evaluation of initiatives that reach rural veterans and to standardize the reporting of outcomes and impacts. Advance Care Planning via Group Visits (ACP-GV), one of ORH's Enterprise-Wide Initiatives, also followed the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. ACP-GV is a national patient-centered intervention delivered in a large, veterans integrated healthcare system. This manuscript describes how RE-AIM was used to evaluate this national program and lessons learned from ORH's annual reporting feedback to ACP-GV on their use of the framework to describe evaluation impacts. Methods: We used patient, provider, and site-level administrative health care data from the VA Corporate Data Warehouse and national program management databases for federal fiscal years (FY) spanning October 1, 2018-September 30, 2023. Measures included cumulative and past FY metrics developed to assess program impacts. Results: RE-AIM constructs included the following cumulative and annual program evaluation results. ACP-GV reached 54,167 unique veterans, including 19,032 unique rural veterans between FY 2018 to FY 2023. During FY 2023, implementation adherence to the ACP-GV model was noted in 91.7% of program completers, with 55% of these completers reporting a knowledge increase and 14% reporting a substantial knowledge increase (effectiveness). As of FY 2023, 66 ACP-GV sites were active, and 1,556 VA staff were trained in the intervention (adoption). Of the 66 active sites in FY 2023, 27 were sites previously funded by ORH and continued to offer ACP-GV after the conclusion of three years of seed funding (maintenance). Discussion: Lessons learned developing RE-AIM metrics collaboratively with program developers, implementers, and evaluators allowed for a balance of clinical and scientific input in decision-making, while the ORH annual reporting feedback provided specificity and emphasis for including both cumulative, annual, and rural specific metrics. ACP-GV's use of RE-AIM metrics is a key step towards improving rural veteran health outcomes and describing real world program impacts.