Improving standards in brain-behavior correlation analyses

Abstract

Associations between two variables, for instance between brain and behavioral measurements, are often studied using correlations, and in particular Pearson correlation. However, Pearson correlation is not robust: outliers can introduce false correlations or mask existing ones. These problems are exacerbated in brain imaging by a widespread lack of control for multiple comparisons, and several issues with data interpretations. We illustrate these important problems associated with brain-behavior correlations, drawing examples from published articles. We make several propositions to alleviate these problems.

Keywords: Pearson correlation; Spearman correlation; confidence intervals; multiple comparisons; multivariate statistics; outliers; robust statistics; skipped correlation.

Figure 1

Examples of Pearson and Spearman correlations. In each subplot, r is Pearson correlation, r_s is Spearman correlation, and r_p is a skipped correlation. Potential univariate and bivariate outliers are marked by circles and other points marked by disks. A skipped correlation is significant if t is larger than t_crit.

Figure 2

Questionable correlations observed in published articles. Each subplot contains a reproduction of the original data and regression line. Outliers are marked by circles and other points marked by disks. Analyses are reported above each subplot: the first line contains the correlation reported by the authors; the second line contains the Pearson correlation r from our analyses; the third line contains Spearman correlation r_s; the fourth line contains the skipped correlation r_p. Subplots A, B, C illustrate correlations that could be due to outliers. Subplot D illustrates a potential case of masking. Subplots E and F show the potentially incorrect use of correlation for what could be split data clouds.

Figure A1

Examples of false correlations potentially due to outliers. Each subplot contains a regression line and a scatterplot with outliers marked by circles and other points marked by disks. Analyses are reported above each subplot: the first line contains the Pearson or Spearman correlation reported by the authors; the second line contains the Pearson correlation we obtained; the third line contains Spearman correlation r_s; the fourth line contains the skipped correlation r_p.

Figure A2

Examples of false correlations potentially due to outliers. See Figure A1 caption for details.

Figure A3

Examples of false correlations potentially due to outliers. See Figure A1 caption for details.

Figure A4

Examples of correlations potentially masked by outliers. See Figure A1 caption for details.

Figure A5

Examples of false correlations potentially due to split data clouds. See Figure A1 caption for details.