Statistical and clinical significance, and how to use confidence intervals to help interpret both

Abstract

Statistical significance is a statement about the likelihood of findings being due to chance. Classical significance testing, with its reliance on p values, can only provide a dichotomous result - statistically significant, or not. Limiting interpretation of research results to p values means that researchers may either overestimate or underestimate the meaning of their results. Very often the aim of clinical research is to trial an intervention with the intention that results based on a sample will generalise to the wider population. The p value on its own provides no information about the overall importance or meaning of the results to clinical practice, nor do they provide information as to what might happen in the future, or in the general population. Clinical significance is a decision based on the practical value or relevance of a particular treatment, and this may or may not involve statistical significance as an initial criterion. Confidence intervals are one way for researchers to help decide if a particular statistical result (whether significant or not) may be of relevance in practice.