Homeopathy can offer empirical insights on treatment effects in a null field

Matthew K Sigurdson; Kristin L Sainani; John P A Ioannidis

doi:10.1016/j.jclinepi.2023.01.010

Homeopathy can offer empirical insights on treatment effects in a null field

J Clin Epidemiol. 2023 Mar:155:64-72. doi: 10.1016/j.jclinepi.2023.01.010. Epub 2023 Feb 1.

Authors

Matthew K Sigurdson¹, Kristin L Sainani², John P A Ioannidis³

Affiliations

¹ Department of Epidemiology and Population Health, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA.
² Department of Epidemiology and Population Health, Stanford University, Stanford, CA, USA.
³ Department of Epidemiology and Population Health, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Department of Medicine, Stanford University, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University, Stanford, CA, USA; Department of Statistics, Stanford University, Stanford, CA, USA. Electronic address: jioannid@stanford.edu.

PMID: 36736709
DOI: 10.1016/j.jclinepi.2023.01.010

Abstract

Objectives: A "null field" is a scientific field where there is nothing to discover and where observed associations are thus expected to simply reflect the magnitude of bias. We aimed to characterize a null field using a known example, homeopathy (a pseudoscientific medical approach based on using highly diluted substances), as a prototype.

Study design and setting: We identified 50 randomized placebo-controlled trials of homeopathy interventions from highly cited meta-analyses. The primary outcome variable was the observed effect size in the studies. Variables related to study quality or impact were also extracted.

Results: The mean effect size for homeopathy was 0.36 standard deviations (Hedges' g; 95% confidence interval: 0.21, 0.51) better than placebo, which corresponds to an odds ratio of 1.94 (95% CI: 1.69, 2.23) in favor of homeopathy. 80% of studies had positive effect sizes (favoring homeopathy). Effect size was significantly correlated with citation counts from journals in the directory of open-access journals and CiteWatch. We identified common statistical errors in 25 studies.

Conclusion: A null field like homeopathy can exhibit large effect sizes, high rates of favorable results, and high citation impact in the published scientific literature. Null fields may represent a useful negative control for the scientific process.

Keywords: Bias; Homeopathy; Meta-Research; Null field; Replication crisis; Research integrity; Treatment effects.

MeSH terms

Bias
Homeopathy* / methods
Humans
Odds Ratio