What if we wait? Using synthetic waiting lists to estimate treatment effects in routine outcome data

Psychother Res. 2023 Nov;33(8):1043-1057. doi: 10.1080/10503307.2023.2182241. Epub 2023 Mar 1.

Abstract

Objective: Due to the lack of randomization, pre-post routine outcome data precludes causal conclusions. We propose the "synthetic waiting list" (SWL) control group to overcome this limitation.

Method: First, a step-by-step introduction illustrates this novel approach. Then, this approach is demonstrated using an empirical example with data from an outpatient cognitive-behavioral therapy (CBT) clinic (N = 139). We trained an ensemble machine learning model ("Super Learner") on a data set of patients waiting for treatment (N = 311) to make counterfactual predictions of symptom change during this hypothetical period.

Results: The between-group treatment effect was estimated to be d = 0.42. Of the patients who received CBT, 43.88% achieved reliable and clinically significant change, while this probability was estimated to be 14.54% in the SWL group. Counterfactual estimates suggest a clear net benefit of psychotherapy for 41% of patients. In 32%, the benefit was unclear, and 27% would have improved similarly without receiving CBT.

Conclusions: The SWL is a viable new approach that provides between-group outcome estimates similar to those reported in the literature comparing psychotherapy with high-intensity control interventions. It holds the potential to mitigate common limitations of routine outcome data analysis.

Keywords: causal inference; machine learning; practice-based evidence; practice-oriented research.

MeSH terms

  • Cognitive Behavioral Therapy*
  • Humans
  • Psychotherapy
  • Treatment Outcome
  • Waiting Lists*