PCAtest: testing the statistical significance of Principal Component Analysis in R

Abstract

Principal Component Analysis (PCA) is one of the most broadly used statistical methods for the ordination and dimensionality-reduction of multivariate datasets across many scientific disciplines. Trivial PCs can be estimated from data sets without any correlational structure among the original variables, and traditional criteria for selecting non-trivial PC axes are difficult to implement, partially subjective or based on ad hoc thresholds. PCAtest is an R package that implements permutation-based statistical tests to evaluate the overall significance of a PCA, the significance of each PC axis, and of contributions of each observed variable to the significant axes. Based on simulation and empirical results, I encourage R users to routinely apply PCAtest to test the significance of their PCA before proceeding with the direct interpretation of PC axes and/or the utilization of PC scores in subsequent evolutionary and ecological analyses.

Keywords: PCAtest; Permutation; Principal component analysis; R function; Statistical significance.

Figure 1. Null distributions and empirical statistics derived from PCAtest analysis of simulated data consisting of five uncorrelated variables and 100 observations.

Figure 2. Null distributions and empirical statistics derived from PCAtest analysis of simulated data consisting of five correlated variables (r = 0.25) and 100 observations.

Lower plots show mean observed values (red dots), 95%-confidence interval (CI) based on 1,000 bootstrap replicates (red bars), mean values and 95%-CI based on 1,000 random permutations (gray dots and bars, respectively).

Figure 3. Null distributions and empirical statistics derived from PCAtest analysis of simulated data consisting of five correlated variables (r = 0.50) and 100 observations.

Lower plots show mean observed values (red dots), 95%-confidence interval (CI) based on 1,000 bootstrap replicates (red bars), mean values and 95%-CI based on 1,000 random permutations (gray dots and bars, respectively).

Figure 4. Null distributions and empirical statistics derived from PCAtest analysis of seven morphological variables measured in 29 ant species (data from Wong & Carmona (2021)).

Lower plots show mean observed values (red dots), 95%-confidence interval (CI) based on 1,000 bootstrap replicates (red bars), mean values and 95%-CI based on 1,000 random permutations (gray dots and bars, respectively).

Figure 5. Results of R-mode PCAtest analysis of microarray data from Ringnér (2008).

Null distributions and empirical statistics derived from PCAtest analysis of 8,534 genes screened in 105 samples (data from Ringnér, 2008). Lower plots show mean observed values (red dots), 95%-confidence interval (CI) based on 1,000 bootstrap replicates (red bars), mean values and 95%-CI based on 1,000 random permutations (gray dots and bars, respectively).

Figure 6. Results of Q-mode PCAtest analysis of microarray data from Ringnér (2008).

Null distributions and empirical statistics derived from PCAtest analysis of 8,534 genes (observations) screened in 105 samples (variables). The original data set of Ringnér (2008) was transposed to perform a Q-mode PCA analysis. Lower plots show mean observed values (red dots), 95%-confidence interval (CI) based on 1,000 bootstrap replicates (red bars), mean values and 95%-CI based on 1,000 random permutations (gray dots and bars, respectively).