Developing and validating a model to predict the success of an IHCS implementation: the Readiness for Implementation Model

Abstract

Objective: To develop and validate the Readiness for Implementation Model (RIM). This model predicts a healthcare organization's potential for success in implementing an interactive health communication system (IHCS). The model consists of seven weighted factors, with each factor containing five to seven elements.

Design: Two decision-analytic approaches, self-explicated and conjoint analysis, were used to measure the weights of the RIM with a sample of 410 experts. The RIM model with weights was then validated in a prospective study of 25 IHCS implementation cases.

Measurements: Orthogonal main effects design was used to develop 700 conjoint-analysis profiles, which varied on seven factors. Each of the 410 experts rated the importance and desirability of the factors and their levels, as well as a set of 10 different profiles. For the prospective 25-case validation, three time-repeated measures of the RIM scores were collected for comparison with the implementation outcomes.

Results: Two of the seven factors, 'organizational motivation' and 'meeting user needs,' were found to be most important in predicting implementation readiness. No statistically significant difference was found in the predictive validity of the two approaches (self-explicated and conjoint analysis). The RIM was a better predictor for the 1-year implementation outcome than the half-year outcome.

Limitations: The expert sample, the order of the survey tasks, the additive model, and basing the RIM cut-off score on experience are possible limitations of the study.

Conclusion: The RIM needs to be empirically evaluated in institutions adopting IHCS and sustaining the system in the long term.

Figure 1

The RIM development process. The headings describe the phases of the process and beneath are descriptions of the key components in each phase. Phases 3 and 4 are the focus of this paper.

Figure 2

RIM predictive scores compared with perceived implementation success. The bolded lines indicate the cut-off points. ○, Correctly predicted successful IHCS initiatives; ◊, under-predicted successful IHCS initiatives; +, correctly predicted unsuccessful IHCS initiatives; ×, falsely predicted unsuccessful IHCS initiatives.