Background and Purpose:: P-values and “statistical significance” (e.g., set α = 0.05, test if P < α) play a central role in modern medical research. However, an overreliance on significance has led to misinterpretation, questionable research practices, and oversimplified conclusions. Reformers have made mutually exclusive calls to abandon statistical significance, to redefine statistical significance (α = 0.005 instead of α = 0.05), or to develop decision rules based on the study context (balancing Type I Errors, α, against Type II Errors, β).
Summary of Key Points:: In light of these debates, we have revised the statistical reporting guidelines at JNPT. Across different methods, P-values should always be viewed as one piece of a larger statistical framework.
1. P-values: which indicate how surprising the data are assuming the null hypothesis is true.
2. Confidence Intervals: which show the range of values compatible with the data (i.e., uncertainty in our estimates). Careful consideration is required of all values within the interval, especially null values.
3. The uncertainty of the results with respect to the Study Context, as the same statistical results may hold different implications depending on the research phase or objectives.
4. The Cost of Different Errors: should be explicitly discussed, because false positives/negatives accrue different risks depending on the study context/goals.
5. The Probability of the Hypothesis: which is distinct from the probability of the data under the null hypothesis (the P-value); done formally, this shift requires Bayesian approaches.
Recommendations:: By integrating these principles, authors can provide nuanced, contextually relevant conclusions that go beyond binary significance thresholds.