How to quantify the evidence for the absence of a correlation

Abstract

We present a suite of Bayes factor hypothesis tests that allow researchers to grade the decisiveness of the evidence that the data provide for the presence versus the absence of a correlation between two variables. For concreteness, we apply our methods to the recent work of Donnellan et al. (in press) who conducted nine replication studies with over 3,000 participants and failed to replicate the phenomenon that lonely people compensate for a lack of social warmth by taking warmer baths or showers. We show how the Bayes factor hypothesis test can quantify evidence in favor of the null hypothesis, and how the prior specification for the correlation coefficient can be used to define a broad range of tests that address complementary questions. Specifically, we show how the prior specification can be adjusted to create a two-sided test, a one-sided test, a sensitivity analysis, and a replication test.

Keywords: Bayes factor; Hypothesis test; Statistical evidence.

Fig. 1

Data for the nine replication experiments from Donnellan et al. (in press). Scores for the loneliness scale are on the x-axis and scores for the physical warmth index are on the y-axis. Each panel also shows the sample Pearson correlation coefficient r, the number of observations N, and the two-sided p value

Fig. 2

Prior and posterior distributions for the correlation ρ between loneliness and the physical warmth index across the nine replication experiments from Donnellan et al. (in press). The statistical model is defined as ${\mathcal{\mathscr{H}}}_1:\rho \sim U(-1,1)$. The filled dots indicate the height of the prior and posterior distributions at ρ = 0; the ratio of these heights equals the evidence that the data provide for ${\mathcal{\mathscr{H}}}_1$ versus ${\mathcal{\mathscr{H}}}_0$ (Wagenmakers et al. 2010)

Fig. 3

Sensitivity analysis for the Bayes factor BF₀₁ across the nine replication experiments from Donnellan et al. (in press). The log of the Bayes factor BF₀₁ is on the x-axis and the prior width γ is on the y-axis. When γ = 0 the alternative hypothesis equals the null hypothesis; when γ = 1 the alternative hypothesis is ρ∼U(−1,1). The Bayes factor is qualitatively robust in the sense that the evidence favors the null hypothesis across a wide range of prior beliefs. See text for details

Fig. 4

Sensitivity analysis for the Bayes factor BF₀₁, collapsing data across studies 1–4 (left panel) and across studies 5–9 (right panel) from Donnellan et al. (in press). The log of the Bayes factor BF₀₁ is on the x-axis and the prior width γ is on the y-axis. When γ = 0 the alternative hypothesis equals the null hypothesis; when γ = 1 the alternative hypothesis is ρ∼U(−1,1). See text for details

Fig. 5

Prior and posterior distributions for the correlation ρ between loneliness and the physical warmth index across the nine replication experiments from Donnellan et al. (in press). The statistical model is defined as ${\mathcal{\mathscr{H}}}_{\text{r}}:\rho \sim \text{“posterior distribution from original study”}$. The filled dots indicate the height of the prior and posterior distributions at ρ = 0; the ratio of these heights equals the evidence that the data provide for the proponent’s ${\mathcal{\mathscr{H}}}_{\text{r}}$ versus the skeptic’s ${\mathcal{\mathscr{H}}}_0$ (Wagenmakers et al. 2010)