Objective: An enduring problem in the field of rehabilitation has been the lack of standardization in the protocols of treatments and tests. To develop a process evaluation method to standardize the administration of rehabilitation procedures used in the Extremity Constraint Induced Therapy Evaluation (EXCITE) Trial, a randomized controlled trial of upper-extremity constraint-induced therapy implemented across 7 sites.
Design: Process evaluation.
Setting: Research laboratory.
Participants: Convenience sample or research personnel.
Interventions: Not applicable.
Main outcome measures: Checklist scoring sheets were developed to rate videotapes using systematic application of prescribed steps for each of 5 procedures across 3 time periods. Time periods were immediately after training, and 1 and 2 years later. A performance score of at least 90% was required before individual research personnel were allowed to participate in the trial.
Results: Overall performance scores ranged from 85.8% to 95% of performance items correctly executed. There was a significant improvement in standard performance of procedures between the first time period (immediately after training) and each of the subsequent time periods for all but 1 procedure. The scoring of standardized performance when carried out with routine participant testing and training did not differ significantly from scoring from videotaped sessions submitted for standardization rating for 2 of the procedures, suggesting adequate validity of scoring from videotape.
Conclusions: The present method was successful in assessing protocol fidelity for the EXCITE research personnel and represents 1 means of addressing the longstanding problem in rehabilitation of the lack of standardization in administering different treatments and tests.