Using simulation modeling to inform intervention and implementation selection in a rapid stakeholder-engaged hybrid effectiveness-implementation randomized trial

Abstract

Background: Implementation research generally assumes established evidence-based practices and prior piloting of implementation strategies, which may not be feasible during a public health emergency. We describe the use of a simulation model of the effectiveness of COVID-19 mitigation strategies to inform a stakeholder-engaged process of rapidly designing a tailored intervention and implementation strategy for individuals with serious mental illness (SMI) and intellectual/developmental disabilities (ID/DD) in group homes in a hybrid effectiveness-implementation randomized trial.

Methods: We used a validated dynamic microsimulation model of COVID-19 transmission and disease in late 2020/early 2021 to determine the most effective strategies to mitigate infections among Massachusetts group home staff and residents. Model inputs were informed by data from stakeholders, public records, and published literature. We assessed different prevention strategies, iterated over time with input from multidisciplinary stakeholders and pandemic evolution, including varying symptom screening, testing frequency, isolation, contact-time, use of personal protective equipment, and vaccination. Model outcomes included new infections in group home residents, new infections in group home staff, and resident hospital days. Sensitivity analyses were performed to account for parameter uncertainty. Results of the simulations informed a stakeholder-engaged process to select components of a tailored best practice intervention and implementation strategy.

Results: The largest projected decrease in infections was with initial vaccination, with minimal benefit for additional routine testing. The initial level of actual vaccination in the group homes was estimated to reduce resident infections by 72.4% and staff infections by 55.9% over the 90-day time horizon. Increasing resident and staff vaccination uptake to a target goal of 90% further decreased resident infections by 45.2% and staff infections by 51.3%. Subsequent simulated removal of masking led to a 6.5% increase in infections among residents and 3.2% among staff. The simulation model results were presented to multidisciplinary stakeholders and policymakers to inform the "Tailored Best Practice" package for the hybrid effectiveness-implementation trial.

Conclusions: Vaccination and decreasing vaccine hesitancy among staff were predicted to have the greatest impact in mitigating COVID-19 risk in vulnerable populations of group home residents and staff. Simulation modeling was effective in rapidly informing the selection of the prevention and implementation strategy in a hybrid effectiveness-implementation trial. Future implementation may benefit from this approach when rapid deployment is necessary in the absence of data on tailored interventions.

Trial registration: ClinicalTrials.gov NCT04726371.

Fig. 1

Stakeholders for simulation modeling to inform an implementation trial of best practices to mitigate COVID-19 disease in group homes for individuals with serious mental illness and intellectual disability/developmental disability in Massachusetts. This figure depicts the multiple stakeholder groups with whom the simulation modeling team collaborated in order to derive model inputs reflective of the population of interest, namely the residents and staff of a sample of 415 group homes in Massachusetts

Fig. 2

Total new infections over 3-month projection, grouped by vaccination group. This figure displays the projected cumulative number of new COVID-19 infections among residents and staff of 415 group homes for individuals with serious mental illness and intellectual disability/developmental disability in Massachusetts under a variety of modeled vaccination strategies, over the 90-day modeled time horizon. Model input parameters are informed by data from the group homes, as well as Massachusetts publicly-available public health data and the published literature. Projected cumulative infections in the setting of vaccination levels current at the time of the model analysis are displayed, as well as strategies with staff turnover and increased resident and staff vaccination uptake. Given the lower baseline vaccine uptake among staff, increasing staff uptake makes the largest incremental difference in preventing both resident and staff infections

Fig. 3

Factors affecting development of new infections among residents and staff of group homes for individuals with serious mental illness and intellectual disability/developmental disability in Massachusetts. This figure displays the projected impact on the of varying uncertain input parameters, including vaccine efficacy and vaccination uptake among residents, staff, and community members, on the development of new COVID-19 infections among a residents and b staff of group homes for individuals with serious mental illness and intellectual/developmental disability in Massachusetts included in a hybrid implementation-effectiveness trial. The values of each parameter are indicated in parentheses in the following format: (smallest value examined - largest value examined; value in the base case Vaccination Current Levels strategy). The orange line in (a) indicates the number of projected resident infections (84) for the base case Vaccination Current Levels strategy. The orange line in (b) indicates the number of projected staff infections (519) for the base case Vaccination Current Levels strategy. The results summarized in the figure helped to inform targets of the trial’s tailored best practice intervention package for COVID-19 risk mitigation