Establishing new cutoffs for Cohen's d: An application using known effect sizes from trials for improving sleep quality on composite mental health

Abstract

Objective: Cohen's d conventional effect size cutoffs [small (0.2), medium (0.5), and large (0.8)] might not be representative of the reported distribution of effect sizes across the different areas of health. Effect size cutoffs might vary not only depending on the area of research, but also on the type of intervention and population. That is, they are context dependent. Therefore, we present strategies to redefine small, medium, and large effect size based on 25, 50, and 75th percentile, respectively.

Methods: We illustrate these techniques applying them to 72 effect sizes, derived from 65 randomized controlled trials described in a recent meta-analysis (10.1016/j.smrv.2021.101556) of improving sleep quality on composite mental health. Such percentiles are equally distanced from the average effect size as suggested by Jacob Cohen and checked for potential attenuation effects (via weight selection model) and outliers (via OutRules).

Results: new cutoffs for effect size distribution of -0.177, -0.329, and -0.557, for small, medium, and large effect size were found, respectively. applying Cohen's effect size thresholds (0.2, 0.5, and 0.8) for trials of improving sleep quality on composite mental health might over-estimate effect sizes compared to the real-world context, especially around medium and large effect sizes.

Keywords: Cohen's d; effect size; mental health; randomized controlled trials; sleep quality; treatment.