Objective: This report examines the impact of collecting and analyzing sequential data from the same sample using the chi-square test. Researchers in rehabilitation frequently analyze categorical data collected repeatedly from the same sample.
Design: Simulations were conducted using the chi-square test to analyze data with correlations between pretest and posttest ranging from .20 to .80. The simulations were analyzed using the standard chi-square based on a 2 x 2 contingency table.
Results: An incidence of type 1 errors of approximately .30 for an expected alpha of .05 was found when the correlation (serial dependence) between pretest and posttest was high (phi = .80). For simulations using an alpha = .01, the type 1 error rate was .18 for data with substantial correlation (phi = .80).
Conclusion: Rehabilitation researchers who use the chi-square test with data collected repeatedly from the same sample risk committing a type 1 error and misinterpreting their results. Alternatives to the standard chi-square test, such as McNemar's test, should be considered when correlated data are analyzed using a 2 x 2 contingency table.