Adopting the RE-AIM analytic framework for rural program evaluation: experiences from the Advance Care Planning via Group Visits (ACP-GV) national evaluation

Monica M Matthieu; Laura D Taylor; David A Adkins; J Silas Williams; Bo Hu; Ciara M Oliver; Jane Ann McCullough; Mary J Mallory; Ian D Smith; Jacob T Painter; Songthip T Ounpraseuth; Kimberly K Garner

doi:10.3389/frhs.2024.1210166

Adopting the RE-AIM analytic framework for rural program evaluation: experiences from the Advance Care Planning via Group Visits (ACP-GV) national evaluation

Front Health Serv. 2024 Mar 25:4:1210166. doi: 10.3389/frhs.2024.1210166. eCollection 2024.

Authors

Affiliations

¹ U.S. Department of Veterans Affairs Medical Center, Central Arkansas Veterans Healthcare System, HSR&D Center of Innovation: Center for Mental Healthcare & Outcomes Research, Little Rock, AR, United States.
² School of Social Work, Saint Louis University, Saint Louis, MO, United States.
³ U.S. Department of Veterans Affairs Medical Center, Central Arkansas Veterans Healthcare System, Geriatric Research, Education and Clinical Center, Little Rock, AR, United States.
⁴ Division of Pharmaceutical Evaluation & Policy, College of Pharmacy, University of Arkansas for Medical Sciences, Little Rock, AR, United States.
⁵ Department of Biostatistics, Fay W. Boozman College of Public Health, University of Arkansas for Medical Sciences, Little Rock, AR, United States.
⁶ Department of Psychiatry, College of Medicine, University of Arkansas for Medical Sciences, Little Rock, AR, United States.

Abstract

Introduction: To support rigorous evaluation across a national portfolio of grants, the United States Department of Veterans Affairs (VA) Office of Rural Health (ORH) adopted an analytic framework to guide their grantees' evaluation of initiatives that reach rural veterans and to standardize the reporting of outcomes and impacts. Advance Care Planning via Group Visits (ACP-GV), one of ORH's Enterprise-Wide Initiatives, also followed the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. ACP-GV is a national patient-centered intervention delivered in a large, veterans integrated healthcare system. This manuscript describes how RE-AIM was used to evaluate this national program and lessons learned from ORH's annual reporting feedback to ACP-GV on their use of the framework to describe evaluation impacts.

Methods: We used patient, provider, and site-level administrative health care data from the VA Corporate Data Warehouse and national program management databases for federal fiscal years (FY) spanning October 1, 2018-September 30, 2023. Measures included cumulative and past FY metrics developed to assess program impacts.

Results: RE-AIM constructs included the following cumulative and annual program evaluation results. ACP-GV reached 54,167 unique veterans, including 19,032 unique rural veterans between FY 2018 to FY 2023. During FY 2023, implementation adherence to the ACP-GV model was noted in 91.7% of program completers, with 55% of these completers reporting a knowledge increase and 14% reporting a substantial knowledge increase (effectiveness). As of FY 2023, 66 ACP-GV sites were active, and 1,556 VA staff were trained in the intervention (adoption). Of the 66 active sites in FY 2023, 27 were sites previously funded by ORH and continued to offer ACP-GV after the conclusion of three years of seed funding (maintenance).

Discussion: Lessons learned developing RE-AIM metrics collaboratively with program developers, implementers, and evaluators allowed for a balance of clinical and scientific input in decision-making, while the ORH annual reporting feedback provided specificity and emphasis for including both cumulative, annual, and rural specific metrics. ACP-GV's use of RE-AIM metrics is a key step towards improving rural veteran health outcomes and describing real world program impacts.

Keywords: RE-AIM framework; United States Department of Veterans Affairs; advance care planning; advance directive; implementation science; veteran.

© 2024 This work is authored by Matthieu, Taylor, Adkins, Williams, Hu, Oliver, McCullough, Mallory, Smith, Painter, Ounpraseuth and Garner on behalf of the US Government and as regards Matthieu, Taylor, Adkins, Williams, Hu, Oliver, McCullough, Mallory, Smith, Painter, Ounpraseuth and Garner, and the US Government, is not subject to copyright protection in the United States. Foreign and other copyrights may apply.

Grants and funding

This work was supported by a Quality Enhancement Research Initiative (QUERI) Partnered Evaluation Initiative (PEC 18-206) from the United States (U.S.) Department of Veterans Affairs Health Services Research & Development awarded to MM and VA VISN 16 and Office of Rural Health funding awarded to the senior author, KG. This project was also supported by the Department of Veterans Affairs Office of Rural Health, Office of Care Management and Social Work Service, and the South-Central Mental Illness, Clinical, Research, Education, and Clinical Center (MIRECC). The Central Arkansas Veterans Health Care System, North Little Rock, Arkansas provided infrastructure resources. The sponsoring VA Program Office determined that this program evaluation was not research.