Thursday, November 2, 2017

Noah Smith — Why 'Statistical Significance' Is Often Insignificant

The knives are out for the p-value. This statistical quantity is the Holy Grail for empirical researchers across the world -- if your study finds the right p-value, you can get published in a credible journal, and possibly get a good university tenure-track job and research funding. Now a growing chorus of voices wants to de-emphasize or even ban this magic number. But the crusade against p-values is likely to be a distraction from the real problems afflicting scientific inquiry....
The real danger is that when each study represents only a very weak signal of scientific truth, science gets less and less productive. Ever more researchers and ever more studies are needed to confirm each result. This process might be one reason new ideas seem to be getting more expensive to find.
If we want to fix science, p-values are the least of our problems. We need to change the incentive for researchers to prove themselves by publishing questionable studies that just end up wasting a lot of time and effort.
There is a difference in proving that one has the ability to use the tools of one's trade and using the tools to produce authentic, useful and elegant output.

No comments: