Lars P. Syll — On the applicability of statistics in social sciences

David Salsburg quote.

Lars P. Syll’s Blog
On the applicability of statistics in social sciences

Lars P. Syll | Professor, Malmo University

Lars P. Syll — Randomization — a philosophical device gone astray

When giving courses in the philosophy of science yours truly has often had David Papineau’s book Philosophical Devices (OUP 2012) on the reading list. Overall it is a good introduction to many of the instruments used when performing methodological and science theoretical analyses of economic and other social sciences issues.

Unfortunately, the book has also fallen prey to the randomization hype that scourges sciences nowadays....

Lars P. Syll’s Blog
Randomization — a philosophical device gone astray
Lars P. Syll | Professor, Malmo University

Lars P. Syll — P-hacking and data dredging

I think there are two separate issues here that depend on intent. "P-hacking" likely implies intent, and that is not necessarily a factor in all cases, and it may well not be in many if not most cases.

In some cases there may be intent to persuade by playing loose, or even to deceive. I recall that How to Lie with Statistics was required reading in the Stat 101 course I took over fifty years ago. But this is not the only issue.

As Richard Feynman famously observed, science is about not fooling ourselves. This applies to each of us individually owing to cognitive bias. Humans are smart, but we mare still primates. 

Nobody is entirely free of cognitive-affective bias. So we have to take steps to counter this tendency. Science was developed as an instrument to address this.

The reason we use rigorous method is to avoid, or at least minimize, our tendency toward being shaped by cognitive biases such as confirmation bias and anchoring. 

Methodology is about using instruments on good data in a rigorous fashion that reduces not only error in application but also bias.

There is no method that completely eliminates error and bias. On one hand, GIGO, and being human, on the other.

A lot of the problems in doing science as well as applying other rigorous instruments lies in measurement. The highest level of formality does nothing to affect errors in measurement. I happened to be thinking about the issues around measurement just prior to reading this post.

And lot of the most interesting things are difficult to measure when humans are involved and psychology enters into the data significantly. History also present issues regarding not only data quality and availability but also changing context that affects the data.

It's good we are having a debate about p-values, since there are issues there than do seem to be influential in a negative way.  And it is not only the stat, but also the data that the method is being applied to.

There are essentially three areas of interest. The first is the method, in this case probability and statistics  as formal method. The second is data and its reliability and precision, along with data collection. The third is data processing and selection. All of these are subject to error and manipulation. This is especially a problem when data sets are proprietary and are not transparent.

But the debate should not stop there. The methodological debate is not over, as some would have it. Science is always tentative on discovery and it is a work in progress. Science is often viewed as a fixed body of true knowledge. That is not a good approach to doing science. The fundamental principle of science is questioning authority, especially that of received belief, intuition and common sense.

Humans are fallible, and it is doubtful that we can ever finally work out all the kinks epistemologically and methodologically. We are a work in progress, too, just as is science.

As a discipline becomes more formalized, there is a greater tendency to emphasize formal rigor at the expense of data and evidence, especially when there are issues around data and evidence. Such tendencies are fertile ground for cognitive-affective bias.

Epistemology, logic, and methodology are foundational to gaining reliable knowledge. We need to keep this in mind.

On one hand, the search for absolute knowledge is a chimera since no criteria can be established as absolute. Criteria are stipulated. This realization should make us humble — and careful.

On the other hand, humans are not lost in a sea of relativity either. History has shown that it is possible to arrive at knowledge that is reliable and practical if intelligence is applied and bias reduced.

Lars P. Syll’s Blog
P-hacking and data dredging
Lars P. Syll | Professor, Malmo University

G.A. Barnard: The “catch-all” factor: probability vs likelihood — Debate between G. A. Barnard and Leonard Jimmie Savage

Similar to there Bayesian versus frequentist debate in statistical reasoning.

Likelihood Principle

My epistemological view on this is that the border between them is fuzzy and needs to be approached on a case by case basis, along with acknowledging a cognitive bias toward greater certainty than is attainable from the given and the reasoning about it.

Humans don't like uncertainty and have a strong bias toward minimizing it at the risk of fooling themselves. Even statisticians.

Error Statistics

G.A. Barnard: The “catch-all” factor: probability vs likelihood

Debate between G. A. Barnard and Leonard Jimmie Savage
Posted by Deborah Mayo, professor in the Department of Philosophy at Virginia Tech and visiting professor at the Center for the Philosophy of Natural and Social Science of the London School of Economics.

Lars P. Syll — Econometrics and the art of putting the rabbit in the hat

More GIGO. How statistical inference requires necessary conditions with respect to population, e.g, random sampling a population with a normal distribution.  If these conditions are not met, inference is not valid and the reasoning fails the test. If the conclusion happens to be true, it is coincidentally and not shown by the model.

Clint Ballinger gets a shout out, too.

Econometrics and the art of putting the rabbit in the hat

Lars P. Syll | Professor, Malmo University