The myth of null-hypothesis significance testing in scientific research

Last modified: May 23, 2017

Null-hypothesis significance testing (NHST) causes researchers to draw erroneous conclusions from their data. NHST is still ubiquitously used as a default procedure to make inferences about population effects. Despite it’s popularity, NHST has received much criticism in the past decades. The core of this criticism is that NHST is a form of indirect inference; a conclusion about a population effect (the null hypothesis) is drawn from the p-value that provides the probability of the data or more extreme data, given the null hypothesis.

However, the probability of a null population effect depends greatly on the prior probability of a null-hypothesis being ‘true’. This is an important underlying reason that many conclusions from research are false and it makes NHST unsuitable as the default procedure to draw conclusions from empirical data. A number of alternatives have been developed that overcome this pitfall, such as Bayesian inference methods, informative hypothesis testing and a-priori inferential statistics. However the uptake of these methods has been limited. Therefore, the questions are why many scientists keep on using NHST and how this information can be used to create a paradigm shift in the scientific community.

NRIN devotes a great deal of attention to the website’s content and would greatly appreciate your suggestions of documents or links you believe to belong on this website.

This selection is an incomplete convenience sample, and does not reflect NRIN’s vision or opinion. Including an item does not necessarily mean that we agree with the authors nor does it imply we think unmentioned items are of poorer quality.

Please report any suggestions, inaccuracies or nonfunctioning hyperlinks etc. that you discover. Thanks in advance!