Diane Coyle — Statistics vs truthiness

[Howard Wainer's] Truth or Truthiness a collection of essays in effect, published as a response to this brave new world of truthiness (ie. lies that people believe because they want to) in politics and public debate. Wainer writes very clearly about statistics in general, and his main theme here, causal inference. This is of course dear to the heart of economists, and gratifyingly Wainer recognises that the profession is more scrupulous than most disciplines about causation. The book starts by underlining the importance of having a clear counterfactual in mind and thinking – thinking! – about how it might be possible to estimate the size of any causal effect. As Wainer puts it, “The real world is hopelessly multivariate,” so untangling the causality is never going to happen without careful thought.

I also discovered that one aspect of something that’s bugged me since my thesis days – when I started disaggregating macro data – namely the pitfalls of aggregation, has a name elsewhere in the scholarly forest: “The ecological fallacy, in which apparent structure exists in grouped (eg average) data that disappaears or even reverses on the individual level.” It seems it’s a commonplace in statistics – here’s one clear explanation I found. Actually, I think the aggregation issues are more extensive in economics; for example I once heard Dave Giles do a brilliant lecture on how time aggregation can lead to spurious autocorrelation results....

Any competent logician can explain how it is illogical to proceed necessarily from individual to general owing to the fallacies of composition and hasty generalization, or to proceed from the general to the individual without regard for synergy, that is, the whole is greater than the sum of the parts.

Consequently, assuming methodological individualism and microfoundations is fraught with pitfalls. Getting the causality right is difficult in social science, even in specific cases, as the difficulty in replicating studies shows.

The Enlightened Economist
Statistics vs truthiness
Diane Coyle | freelance economist and a former advisor to the UK Treasury. member of the UK Competition Commission, and acting Chairman of the BBC Trust, the governing body of the British Broadcasting Corporation