Permutation tests for detecting treatment effect heterogeneity in cluster randomized trials

Stat Methods Med Res. 2025 Aug;34(8):1617-1632. doi: 10.1177/09622802251348999. Epub 2025 Jun 17.

Abstract

Cluster randomized trials are widely used in healthcare research for the evaluation of intervention strategies. Beyond estimating the average treatment effect, it is often of interest to assess whether the treatment effect varies across subgroups. While conventional methods based on tests of interaction terms between treatment and covariates can be used to detect treatment effect heterogeneity in cluster randomized trials, they typically rely on parametric assumptions that may not hold in practice. Adapting existing permutation tests from individually randomized trials, however, requires conceptual clarification and modification due to the multiple possible interpretations of treatment effect heterogeneity in the cluster randomized trial context. In this work, we develop variations of permutation tests and clarify key causal definitions in order to assess treatment effect heterogeneity in cluster randomized trials. Our procedure enables investigators to simultaneously test for effect modification across a large number of covariates, while maintaining nominal type I error rates and reasonable power in simulation studies. In the Pain Program for Active Coping and Training (PPACT) study, the proposed methods are able to detect treatment effect heterogeneity that was not identified by conventional methods assessing treatment-covariate interactions.

Keywords: Cluster randomized trials; estimands; generalized additive mixed model; intracluster correlation coefficient; linear mixed models; permutation test.

MeSH terms

  • Cluster Analysis
  • Computer Simulation
  • Data Interpretation, Statistical
  • Humans
  • Models, Statistical
  • Randomized Controlled Trials as Topic* / statistics & numerical data
  • Treatment Effect Heterogeneity
  • Treatment Outcome