Beyond statistical significance: clinical interpretation of rehabilitation research literature

Abstract

Evidence-based practice requires clinicians to stay current with the scientific literature. Unfortunately, rehabilitation professionals are often faced with research literature that is difficult to interpret clinically. Clinical research data is often analyzed with traditional statistical probability (p-values), which may not give rehabilitation professionals enough information to make clinical decisions. Statistically significant differences or outcomes simply address whether to accept or reject a null or directional hypothesis, without providing information on the magnitude or direction of the difference (treatment effect). To improve the interpretation of clinical significance in the rehabilitation literature, researchers commonly include more clinically-relevant information such as confidence intervals and effect sizes. It is important for clinicians to be able to interpret confidence intervals using effect sizes, minimal clinically important differences, and magnitude-based inferences. The purpose of this commentary is to discuss the different aspects of statistical analysis and determinations of clinical relevance in the literature, including validity, significance, effect, and confidence. Understanding these aspects of research will help practitioners better utilize the evidence to improve their clinical decision-making skills.

Level of evidence: 5.

Keywords: Clinical significance; evidence based practice; statistical significance.

Figure 1.

Normal distribution bell‐shaped curve with standard deviations (From http://en.wikipedia.org/wiki/File:Standard_deviation_diagram.svg)

Figure 2.

Confidence intervals presented as a curve graph (a), line graph (b), and bar graph (c), each representing a point estimate and CI of 0.3(‐0.1,0.7).

Figure 3.

Graphical representation of different types of CI: point estimate (a); effect size (b); odds ratio (c).

Figure 4.

Harmful, trivial, and beneficial ranges within a CI representing effect sizes. Adapted from Batterham & Hopkins.

Figure 5.

Line graph of CI 8(4,12) and MCID of 10. Note that the CI still contains the MCID; therefore, the treatment may be beneficial but a larger sample is needed.

Figure 6.

Forest plot used in meta‐analysis studies to summarize the effects of different studies.

Figure 7.

Using Magnitude‐based inferences to describe CIs and clinical relevance.