Objective: Understanding patient responses to psychotherapy is important in developing effective interventions. However, coding patient language is a resource-intensive exercise and difficult to perform at scale. Our aim was to develop a deep learning model to automatically identify patient utterances during text-based internet-enabled Cognitive Behavioural Therapy and to determine the association between utterances and clinical outcomes. Method: Using 340 manually annotated transcripts we trained a deep learning model to categorize patient utterances into one or more of five categories. The model was used to automatically code patient utterances from our entire data set of transcripts (∼34,000 patients), and logistic regression analyses used to determine the association between both reliable improvement and engagement, and patient responses. Results: Our model reached human-level agreement on three of the five patient categories. Regression analyses revealed that increased counter change-talk (movement away from change) was associated with lower odds of both reliable improvement and engagement, while increased change-talk (movement towards change or self-exploration) was associated with increased odds of improvement and engagement. Conclusions: Deep learning provides an effective means of automatically coding patient utterances at scale. This approach enables the development of a data-driven understanding of the relationship between therapist and patient during therapy.
Keywords: cognitive behaviour therapy; outcome research; technology in psychotherapy research & training.