Applying univariate vs. multivariate statistics to investigate therapeutic efficacy in (pre)clinical trials: A Monte Carlo simulation study on the example of a controlled preclinical neurotrauma trial

PLoS One. 2020 Mar 26;15(3):e0230798. doi: 10.1371/journal.pone.0230798. eCollection 2020.

Abstract

Background: Small sample sizes combined with multiple correlated endpoints pose a major challenge in the statistical analysis of preclinical neurotrauma studies. The standard approach of applying univariate tests on individual response variables has the advantage of simplicity of interpretation, but it fails to account for the covariance/correlation in the data. In contrast, multivariate statistical techniques might more adequately capture the multi-dimensional pathophysiological pattern of neurotrauma and therefore provide increased sensitivity to detect treatment effects.

Results: We systematically evaluated the performance of univariate ANOVA, Welch's ANOVA and linear mixed effects models versus the multivariate techniques, ANOVA on principal component scores and MANOVA tests by manipulating factors such as sample and effect size, normality and homogeneity of variance in computer simulations. Linear mixed effects models demonstrated the highest power when variance between groups was equal or variance ratio was 1:2. In contrast, Welch's ANOVA outperformed the remaining methods with extreme variance heterogeneity. However, power only reached acceptable levels of 80% in the case of large simulated effect sizes and at least 20 measurements per group or moderate effects with at least 40 replicates per group. In addition, we evaluated the capacity of the ordination techniques, principal component analysis (PCA), redundancy analysis (RDA), linear discriminant analysis (LDA), and partial least squares discriminant analysis (PLS-DA) to capture patterns of treatment effects without formal hypothesis testing. While LDA suffered from a high false positive rate due to multicollinearity, PCA, RDA, and PLS-DA were robust and PLS-DA outperformed PCA and RDA in capturing a true treatment effect pattern.

Conclusions: Multivariate tests do not provide an appreciable increase in power compared to univariate techniques to detect group differences in preclinical studies. However, PLS-DA seems to be a useful ordination technique to explore treatment effect patterns without formal hypothesis testing.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Brain Injuries / therapy*
  • Clinical Trials as Topic*
  • Humans
  • Monte Carlo Method
  • Multivariate Analysis
  • Sample Size
  • Statistics as Topic / methods*

Grant support

The work of HT was funded by Fresenius Kabi Deutschland GmbH. This does not alter our adherence to PLOS ONE policies on sharing data and materials. The work of SG was partly supported by the CRC 1193. ESW was supported by Center for Computational Sciences in Mainz (CSM). The funder provided support in the form of salaries for author HT, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. This does not alter our adherence to PLOS ONE policies on sharing data and materials. The specific roles of all authors are articulated in the ‘author contributions’ section.