Abstract:Much statistical teaching and many research reports focus on the 'null hypothesis significance test'. Yet the correct meaning and interpretation of statistical significance tests is elusive. Misinterpretations are both common and persistent, leading many to question whether significance tests should be used at all. While most take aim at the arbitrary declaration of p < 0.05 as a threshold for determining 'significance', others extend the critique to suggest the 'p-value' should be dispensed with entirely. P-values and significance tests are still widely used as if they give a measure of the size and importance of relationships, even though this misunderstanding has been observed and discussed for many years. We argue that p-values and significance tests are intrinsically misleading. Point estimates of relationships and confidence intervals give direct information about the effect and the uncertainty of the estimate without recourse to interpreting how a particular p-value might have arisen or indeed referring to them at all. In this paper we briefly outline some of the problems with significance testing, offer a number of examples selected from a recent issue of the International Journal of Nursing Studies and discuss some proposed responses to these problems. We conclude by offering some guidance to authors reporting statistical tests in journals and present a position statement that has been adopted by the International Journal of Nursing Studies to guide its' authors in reporting the results of statistical analyses. While stopping short of calling for an outright ban on reporting p-values and significance tests we urge authors (and journals) to place more emphasis on measures of effect and estimates of precision/uncertainty and, following the position of the American Statistical Association, emphasise that authors (and readers) should avoid using 0.05 or any other cut off for a p-value as the basis for a decision about the meaningfulness/importance of an effect. If point estimates and confidence intervals are used, then the p-value may be redundant and can be omitted from reports. When authors talk about 'significance' they need to be explicit when referring to statistical significance and we recommend authors adopt the language of 'importance' when talking about effect sizes to avoid any confusion.

The p value wars (again)

[Statistical P values do not dominate scientific research].

Is It Time to Ban the P Value?

Statistical witchhunts: Science, justice & the p-value crisis

New relevance and significance measures to replace p-values

Statistical significance testing and p-values: Defending the indefensible? A discussion paper and position statement

Thou Shalt Not Reject the P-value

Evidence-based medicine or statistically manipulated medicine? Are we slaves to the P -value?

The Practical Alternative to the p Value Is the Correctly Used p Value

The Search for Significance: A Few Peculiarities in the Distribution of P Values in Experimental Psychology Literature

Comment on "The statistics wars and intellectual conflicts of interest" by D. Mayo

P-value: A Bless or A Curse for Evidence-Based Studies?

Second-generation p-values: Improved rigor, reproducibility, & transparency in statistical analyses

When More Is Less: Pitfalls of significance testing

High-Dimensional Randomized Crossover Studies: A Clarification of P-Values Interpretation

Understanding p-values and significance

Trap of trends to statistical significance: likelihood of near significant P value becoming more significant with extra data

Reexamining Statistical Significance and P-Values in Nursing Research: Historical Context and Guidance for Interpretation, Alternatives, and Reporting

Addressing Common Misuses and Pitfalls of P values in Biomedical Research

The evidence contained in the P-value is context dependent

Connecting Simple and Precise P-values to Complex and Ambiguous Realities