The ladder of social science reasoning, 4 statements in increasing order of generality, or Why didn’t they say they were sorry when it turned out they’d messed up? — Andrew Gelman

Reinhart and Rogoff. Why didn't they take responsibility, a student asked Andrew Gelman. Statistics professor Gelman answers:  It wasn't actually about the data in the minds of R & R, so being wrong about it apparently made no significant difference to them. Empirical result? Meh.

Rationalists, or just ideologues with a cognitive bias?

Statistical Modeling, Causal Inference, and Social Science
The ladder of social science reasoning, 4 statements in increasing order of generality, or Why didn’t they say they were sorry when it turned out they’d messed up?
Andrew Gelman | Professor of Statistics and Political Science and Director of the Applied Statistics Center, Columbia University

See also

Paper by Mohsen Javdani and Ha-Joon Chang
Marginal Revolution
Ideological bias and argument from authority among economists
Tyler Cowen | Holbert C. Harris Chair of Economics at George Mason University and serves as chairman and general director of the Mercatus Center

Daniel Little — Modeling the social

Brief review of Scott Page, The Model Thinker: What You Need to Know to Make Data Work for You.

Understanding Society
Modeling the social
Daniel Little | Chancellor of the University of Michigan-Dearborn, Professor of Philosophy at UM-Dearborn and Professor of Sociology at UM-Ann Arbor

Lars P. Syll — P-hacking and data dredging

I think there are two separate issues here that depend on intent. "P-hacking" likely implies intent, and that is not necessarily a factor in all cases, and it may well not be in many if not most cases.

In some cases there may be intent to persuade by playing loose, or even to deceive. I recall that How to Lie with Statistics was required reading in the Stat 101 course I took over fifty years ago. But this is not the only issue.

As Richard Feynman famously observed, science is about not fooling ourselves. This applies to each of us individually owing to cognitive bias. Humans are smart, but we mare still primates. 

Nobody is entirely free of cognitive-affective bias. So we have to take steps to counter this tendency. Science was developed as an instrument to address this.

The reason we use rigorous method is to avoid, or at least minimize, our tendency toward being shaped by cognitive biases such as confirmation bias and anchoring. 

Methodology is about using instruments on good data in a rigorous fashion that reduces not only error in application but also bias.

There is no method that completely eliminates error and bias. On one hand, GIGO, and being human, on the other.

A lot of the problems in doing science as well as applying other rigorous instruments lies in measurement. The highest level of formality does nothing to affect errors in measurement. I happened to be thinking about the issues around measurement just prior to reading this post.

And lot of the most interesting things are difficult to measure when humans are involved and psychology enters into the data significantly. History also present issues regarding not only data quality and availability but also changing context that affects the data.

It's good we are having a debate about p-values, since there are issues there than do seem to be influential in a negative way.  And it is not only the stat, but also the data that the method is being applied to.

There are essentially three areas of interest. The first is the method, in this case probability and statistics  as formal method. The second is data and its reliability and precision, along with data collection. The third is data processing and selection. All of these are subject to error and manipulation. This is especially a problem when data sets are proprietary and are not transparent.

But the debate should not stop there. The methodological debate is not over, as some would have it. Science is always tentative on discovery and it is a work in progress. Science is often viewed as a fixed body of true knowledge. That is not a good approach to doing science. The fundamental principle of science is questioning authority, especially that of received belief, intuition and common sense.

Humans are fallible, and it is doubtful that we can ever finally work out all the kinks epistemologically and methodologically. We are a work in progress, too, just as is science.

As a discipline becomes more formalized, there is a greater tendency to emphasize formal rigor at the expense of data and evidence, especially when there are issues around data and evidence. Such tendencies are fertile ground for cognitive-affective bias.

Epistemology, logic, and methodology are foundational to gaining reliable knowledge. We need to keep this in mind.

On one hand, the search for absolute knowledge is a chimera since no criteria can be established as absolute. Criteria are stipulated. This realization should make us humble — and careful.

On the other hand, humans are not lost in a sea of relativity either. History has shown that it is possible to arrive at knowledge that is reliable and practical if intelligence is applied and bias reduced.

Lars P. Syll’s Blog
P-hacking and data dredging
Lars P. Syll | Professor, Malmo University

Lars P. Syll — Mainstream economics — like butter scraped over too much bread

Wassily Leontief quote from "Theoretical Assumptions and Nonobserved Facts."

Lars P. Syll’s Blog
Mainstream economics — like butter scraped over too much bread
Lars P. Syll | Professor, Malmo University

Noah Smith — "Theory vs. Data" in statistics too

Important.

I think Noah has this right. Fit the tool to the job, rather than the job to the tool.

Aristotle defined speculative knowledge in terms of causal explanation. This definition stuck although Aristotle's analysis of causality did not.

In the Posterior Analytics, Aristotle places the following crucial condition on proper knowledge: we think we have knowledge of a thing only when we have grasped its cause (APost. 71 b 9–11. Cf. APost. 94 a 20). That proper knowledge is knowledge of the cause is repeated in the Physics: we think we do not have knowledge of a thing until we have grasped its why, that is to say, its cause (Phys. 194 b 17–20). Since Aristotle obviously conceives of a causal investigation as the search for an answer to the question “why?”, and a why-question is a request for an explanation, it can be useful to think of a cause as a certain type of explanation. (My hesitation is ultimately due to the fact that not all why-questions are requests for an explanation that identifies a cause, let alone a cause in the particular sense envisioned by Aristotle.) — Stanford Encyclopedia of Philosophy

There is a distinction between reasons and causes. Some types of explanation seek only reasons, while other seek causes. Causation subsequently came to be viewed in terms of articulating mechanisms or lines of transmission (models) that are substantiated in evidence.

Explanation by reasons is different since the strict criterion of articulating mechanisms or lines of transmission that can be checked against evidence is not required.

Explanation by reasons rather than strictly by establishing causation is based on the principle of sufficient reason, which is usually credited to Spinoza and Leibnitz.

In philosophical logic, two negative criteria are foundational. Valid reasoning is vitiated by 1) arguing in a circle and 2) infinite regress.

Without recourse to checking against evidence there is no stopping point in assigning causes other than stipulation, e.g. of a first cause.

However, there may be a reason for a stopping point that doesn't involve causality based on evidence from observation or only stipulation, for example, principles that are "self-evident" based on intuition such as Aristotle's conception of intellectual intuition, or Kant's synthetic a priori propositions as articulated in the Critique of Pure Reason. 

On the other hand, Hume argued that causality is merely over-interpretation of constant correlation, there being no knowledge of the world other than that based on sense data. There is no observable causal link.

Cutting to the chase, scientific explanation based on causality is grounded in models that articulate causal mechanisms or lines of transmission that show how things change invariantly, which is the basis for deterministic functions. Where this is not possible, then there are two other avenues. The first is explanation by giving reasons, which is the domain of speculative philosophy. The second is employing statistics to explore patters of correlation. The question then is to what degree causal models can be gained from statistical methods, or whether it is possible at all. 

This is the issue that Noah Smith's post is getting at.

Noahpinion
"Theory vs. Data" in statistics too
Noah Smith | Bloomberg View columnist

Selim Yaman — It’s gotta be true, because data says so

Nicely framed post on the use and misuse of data. Remember Reinhart & Rogoff.

What they don't mention is that often the data are not made available publicly because it comes from proprietary sources. "Trust us."

The Minskys
It’s gotta be true, because data says so
Selim Yaman, TRT World Research Centre and graduate student in Political Economy of Development, SOAS, London.

Andrew Gelman — The role of models and empirical work in political science

I’m more and more becoming convinced of Dan Kahan’s idea that the paradigmatic task of empirical science is not the testing of hypotheses but the gathering of data in order to distinguish between competing models of the world.

Statistical Modeling, Causal Inference, and Social Science
The role of models and empirical work in political science
Andrew Gelman | Professor of Statistics and Political Science and Director of the Applied Statistics Center, Columbia University

Source — The People and Tech Behind the Panama Papers – Interview with Mar Cabra, ICIJ Data & Research

If you are into the nuts and bolts.

Source
The People and Tech Behind the Panama Papers
Interview with Mar Cabra, head of the Data & Research Unit at ICIJ

Noah Smith — Elegant Economic Theories Get Shoved Aside by Data

The first approach -- relying on an assist from econ theory -- is called structural estimation. Basically, you make a theory of how the economy works -- how consumers behave, who owns what and what kind of costs companies face. Then you use available data to see how well the model fits the data, and to figure out the most likely values for the model’s parameters. Those parameters, or the specifics of the model, could include how risk-averse people are or how much it costs companies to change the rate at which they purchase new capital equipment.

The advantage of this method is that it allows you to pose all kinds of interesting questions. You can ask what would happen if racial discrimination suddenly vanished from the workplace -- even if that’s never actually happened in the past.

The downside is that your theory might be wrong and give you the incorrect answers. All those parameter values might be estimates of things that don’t even exist! Ideally, you can check whether the theory fits the data, and only use the theories that check out, but in reality very few models fit the data well enough to pass these tests. Structural estimation is powerful if you have a very good working theory of the world, but that’s a luxury we rarely have.

The second main approach is called the quasi-experimental technique, or natural experiments. This is when you look for a random variation in economic conditions or policy, and you observe the effect of that random variation. For example, suppose that a crazy dictator in a Caribbean nation suddenly decides to send a whole bunch of low-skilled refugees to Miami. You can use that random decision to get an idea of how an influx of low-skilled immigrants affects local wages and employment levels. Or suppose a city runs a lottery, and lets the winners go to whatever school they want. By comparing the lottery winners to the losers (who are stuck with their old schools by pure random chance), you can get an estimate of how much difference school choice makes.

This is a very powerful technique, because it doesn’t require you to have the correct theory. Just look at the effect of A on B. But it also has a severe limitation -- the more the world changes from when and where you did the study, the less useful the result is. Suppose you look at a minimum wage hike that raises hourly pay to $5.05 in New Jersey in 1992. What does that tell you about the likely effect of a plan adopted to raise the minimum wage to $15 in Seattle? Maybe a lot, but maybe nothing at all. Quasi-experimental research becomes less reliable the further you move away from the conditions where the experiment happened -- and you don’t even know how fast the reliability vanishes.…

Bloomberg View

Elegant Economic Theories Get Shoved Aside by Data

Noah Smith | Assistant Professor of Finance, Stony Brook University

Noah Smith — Unlearning economics

Data.

Noahpinion
Unlearning economics
Noah Smith | Assistant Professor of Finance, Stony Brook University

Beatrice Cherrier — Theory vs data, computerization, old wine and new bottles: Morgenstern and Econometric Society fellows, 1953

[Oskar] Morgenstern proposed that candidates be required to“have done some econometric work in the strictest sense” and be “in actual contact with data they have explored and exploited for which purpose they may have even developed new methods.” Though remembered as the coauthor of one of economics’ most prominent theoretical treatise, The Theory of Games, Morgenstern also had a longstanding interest in statistical work. His feeling was that the Econometric Society and Econometrica editors overvalued “purely abstract work,” while stimulus should be give, to “papers involving data.”

The Undercover Historian
Theory vs data, computerization, old wine and new bottles: Morgenstern and Econometric Society fellows, 1953
Beatrice Cherrier

Don Quijones— Did the European Court of Justice Just Torpedo the Mother of All US Trade Agreements?

Europe’s already rocky trading relationship with the U.S. just got a whole lot worse. Thanks to one young man’s battle against one of the world’s biggest tech companies, data traffic underpinning the world’s largest trading relationship has been thrown into jeopardy.

As the Wall Street Journal warns, hanging in the balance could be billions of dollars of trade in the online advertising business, as well as more quotidian tasks such as storing human-resources documents about European colleagues.

When, in 2013, the Austrian law graduate Max Schrems filed a data-privacy-infringement lawsuit against Facebook after Edward Snowden had revealed the full extent of the company’s collusion with the NSA, little could he have imagined the impact he would end up having. Now, two years later, the European Court of Justice has ruled that the Safe Harbor Agreement that has governed EU data flows across the Atlantic for some 15 years is no longer valid.…

As WOLF STREET previously reported, TiSA appears to have three primary goals: 1) privatize all services; 2) rip up national and regional financial regulations and 3) spread the U.S. approach to data protection — i.e. no protection — around the world…

Battle brewing against US overreach.

Raging Bull-Shit
Did the European Court of Justice Just Torpedo the Mother of All US Trade Agreements?
Don Quijones

Merijn Knibbe — Estimating capital. Robert Gallman edition

In economics, there is an unfortunate rift between academics and the economists who actually measure the economy. Which means that academic economists give little attention to the extremely important question how economic concepts relate to actual measurements – one reason why so much of their work is naïve (‘Ricardian’ households which spend more when taxes go up and the like). Fortunately, economic historians, who often have to do the measurements themselves, often bridge part of the gap. Robert Gallman has some highly relevant remarks about different ways to measure (nineteenth century USA) capital – and how these relate to the future, the past, uncertainty, savings, consumption foregone and replacement costs. This still leaves out important parts of the concept of capital like liquidity, ownership and the ‘overlapping generations’ problem – which however does not make these remarks less valuable.

"Naïve" or BS?

Real-World Economics Review Blog
Estimating capital. Robert Gallman edition
Merijn Knibbe

Mark Thoma — Why Economists Can’t Always Trust Data

Why economics is not a science, other than in very narrow studies where reliable data can be obtained with a reasonable degree of assurance. That excludes most areas of interest and contention in economics, the consequence that economics resembles philosophy more than science and ideology prevails in choice of assumptions and method.

This will likely improve in the Information Age, with Big Data and the development of AI. But we aren't there yet and probably never will be regarding historical data, where context has been lost.

The Fiscal Times
Why Economists Can’t Always Trust Data
Mark Thoma | Professor of Economics, University of Oregon

Tim Johnson — Piketty and the problems of data interpretation

A data scientist looks at Piketty and says not to worry, journalists like Chris Giles (and the FT staff that may have helped him) are not data scientists. If you want to throw out Piketty on this count, then prepare to throw out most of science.

Magic, Maths, and Money — The Relationship between Science and Finance
Piketty and the problems of data interpretation
Tim Johnson | Lecturer in Financial Mathematics at Heriot-Watt University, Edinburgh

Tim Johnson appends an observation about capitalism as an economic system:

I have not read Piketty's book but I will, I will do so on the basis that I am dubious about his conclusions and I base this doubt on an intuition that he believes that capitalism inevitably leads to wealth inequality. I see this is a marxist (as distinct from Marxist) interpretation that rests on a sense of determinism. I believe that capitalism can take on many characters, just as eating is done differently in East Asian and West European cultures. In this respect we can construct a capitalism that does not lead to wealth inequality. (Of course, this may rest on how capitalism is defined, if it is defined on the basis of profit maximisation rather than market exchange, there is no hope for capitalism, in my opinion).

Yes, I think that the highlighted sentence is the crux of it. A lot of the kerfuffle about capitalism results from different definitions of it. When I say that capitalism is antithetical to democracy, I specify a definition of capitalism that privileges capital ownership consisting of money and machines over labor, signifying the vast majority of the world's people.

Keeping Up With FULL Context: OpenSourced Hardware AND Software As One Key To Distributed OBT&E

   (Commentary posted by Roger Erickson)

NVIDIA Unveils First Mobile Supercomputer for Embedded Systems

I wonder if ARM licensees can keep up? Or other CPU mass producers. Maybe CPU will also soon become Open-Sourced commodities, rather like grass for cattle, or ants for insectivores. :)

Regardless of hardware vendors, cheaper data-intensive hardware will allow more distributed access to serious data-crunching and complex model simulations.

Full context simulations? We're nearly there with sophisticated gaming, even though developing gaming technology is still mostly applied to culturally trivial distractions.

Sophisticated, full-context modeling could nevertheless soon follow. Simply by using real-time, real data feeds, instead of only artificial-reality imaging engines.

This will slowly make it more feasible for more people in all settings and all disciplines to begin to model their entire supply chains or "eco-systems" - as well as the full context (the whole that grows as less/equal/more than the sum of all sub-eco-systems).

The "Singularity" is more likely to accelerate human coordination. Enslavement of humans by their smart-phones/watches/glasses still seems improbable. :)

Keeping up with context is how all organizations reduce internal frictions to low enough levels that all sub-members aren't simply fighting one another. That's what OBT&E is all about. Outcomes-Based-Training-&-Education, aka, cultural evolution.

Making it harder for citizens to work at cross-purposes is always a function of processing key data flow. Just running enough model simulations (i.e., thinking) to even keep recognizing minimal-data short-cuts for given contexts is itself a considerable task.

Distributed awareness of full context - aka, an informed public. It's not just the data that matters. Without context, any amount of data is still meaningless. So getting slowly improving, full-context models in front of more people is the name of the game.

Real-time, full-context awareness, across full groups? That requires:

streaming even far more data flows, in real-time, 

RAPID pattern recognition within and across more data flows, 

massively parallel testing of distributed tuning steps (local actions), 

and most of all ... an ongoing ASSESSMENT model ("Public Purpose" as the agile, floating Desired Outcome).

All inter-linked, through massively parallel feedback data flows, of course. No, corporate owned advertising & propaganda channels will NOT suffice. Don't even ask. It's already just in the way, and a significant paywall obstacle that is slowing the evolution of the USA.

How about personal "Citizen Dashboards" able to serve distributed citizens as well as the cockpit data displays serve airplane pilots? That should help +320 million people run a more agile culture and country. It might even save our collective butts. The aggregate return on coordinating REQUIRED distribution of nearly all available info to nearly all "tribal" or citizen members ... will be a far higher return than ANYTHING any individual can currently imagine. Even legendary King Midas - or anyone else from 2000 years ago - couldn't imagine the distributed wealth we argue over today. Why should we constrain even our own, following grandchildren, let alone the 7th generation yet unborn?

Why NOT OpenSource our survival ... the challenge is wide open, and not yet sourced. Supposedly, none of us is as smart as all of us, so lets just figure out ways to USE all of us?

As Ben Franklin (and every tribal member throughout history) said, it seems we still must all hang together, or we'll surely experience a common demise.

The Campaign to Neuter Our Fiat

   (Commentary by Roger Erickson)

Neuter our Fiat?

May the gods, universe and our own MiddleClass please take pity upon these sad people for their lack of insight, and lack of situational awareness.

This is a growing, ironically "well funded" and strangely distorted campaign. These folks do NOT grasp semantics OR fiat currency operations.

Over 2000 years ago the Sophists "proved" - using a linguistic loophole since named semantics - that if you owned a cat, and that cat was a mother, that it was therefore "your" mother. Ergo, we all have a responsibility to neuter our mothers.

It took over 2000 years before Walter Shewhart noticed the depth of the resulting problem, and publicly declared that

"Without context, data has no meaning."

Seriously, I'm more than half afraid that the Neuter-The-Fiat campaigners might endorse the same faux "logic" - and start neutering their mothers.

After all, just because the process by which a nation creates it's own, sovereign fiat currency is referred to by accountants as a "nominal," fiat debt, and is an expression of Public Initiative or "fiat," .... semantic sophism dictates that the nominal fiat accounting "debt" is OUR real "debt," right? Weepin' Erbles in a Marriner Eccles trap! Is that the best we can do here in the USA, in the year 2014?

Hence, a foolish population and it's public fiat are soon neutered, and hoisted on their own semantics. Worst, the emasculating campaign is funded, ironically, by the very people most desperate to maintain the buying power of their hoarded fiat currency. They desperately want to stop the flow of what they want! What can you do with such simpletons? Send them back to highschool?

Maybe the people running the Neuter-The-Fiat campaign will fall for a dose of their own logic? Fine. I hereby suggest that they maintain their cranial blood pressure by tightening a tourniquet around their neck, one twist per day, until their middle section gives up the battle & dies. They'll succeed, by stopping the flow of what they want.

Maybe it'll werk fer thum!

Does It Help, To Simply Say That We Need A More Distributed Steering Wheel?

   (Commentary posted by Roger Erickson)




Steering the Expanding Responses of Growing Aggregates ... To Accelerating Data Flows (we need a more distributed steering wheel)

Precisely Managing Nonsense, With Double-Entry Accounting

Commentary by Roger Erickson

John Maynard Keynes once wrote of Hayek’s book Prices and Production:

“The book, as it stands, seems to me to be one of the most frightful muddles I have ever read, with scarcely a sound proposition in it beginning with page 45 … It is an extraordinary example of how, starting with a mistake, a remorseless logician can end up in Bedlam.” (Keynes 1931: 394).

The Horror of Rothbardian Natural Rights

One has to be aware of the purpose of evolving aggregates before assuming definitions for natural flows, and managing them with double-entry accounting. Otherwise, you end up rigorously managing nonsense.

Sense vs non-sense comes ONLY by the selection pressure of combining feedback from all channels, existing and emerging. There is no adaptive path in the natural world that isn't bounded by dynamic equilibria between locally conflicting forces, and local perspectives.

In the end, you can't square double-entry accounting with quantum mechanics. That should remind us to never lose track of the fact that it's accounting that has to yield to reality, not the other way around.

This reminds me of three, simple, ancient truths.
* Data is meaningless without context.
* The difference between [operations] & theory is greater in practice than in theory.
* Theory soon becomes meaningless if not constantly serving actual aggregate [not sub-group] operations.

The definition of logic itself is looking for consistency in explaining unpredictable reality. Logic simply maps theory to discovered operations, and in the process selects sense from nonsense.

Interestingly, that seemed to be recognized by members of the Brain Trust who advise the FDR administrations through the astounding number of pragmatic experiments which they utilized, even though it was clear that they, themselves couldn't adequately model a completely new paradigm from the methods which context forced them to adopt, by trial and error. That had to wait for the operational summary called MMT.

STUART CHASE: SEMANTICIST EXTRAORDINARY

Chase, Stuart - Tyranny Of Words [pdf; $9 fee in USA]
  (Ironically, available here too, from Germany, free.)

One lesson is that we always need a survivable ratio of leaders and experimenters in policy offices responsible for our survival, and that we soon die if we instead fill those leadership offices with risk managers. We're always executing experiments on ourselves, no matter what we do. So we may as well have as leaders people able to rapidly learn from many mistakes, rather than those opposed to any change whatsoever, and terrified of making a mistake. The ultimate mistake is to not generate survivable ones as a survivable rate!

HOW REALITY IS INTERPRETED

Commentary by Roger Erickson

Know of this place? Lincoln Land Institute

It's one of many similar institutes around the world, which track fundamental data. There are many similar organizations, tracking real data - from diverse disciplines - which can be used to define real contexts.

Key Problem?

These kinds of institutes seem to be lost in the overgrowth of DC propaganda "think tanks."

That signal/noise ratio implies that we're completely sidetracked to a struggle for control of meta data. Fundamentals SEEM less relevant every year, while the battle for "HOW REALITY IS INTERPRETED" becomes increasingly abstract.

Will we still know how to adjust quickly enough, when fundamental adjustment is once again needed?*

If you really care about your grandchildren, you better do more than just hope so.



* The difference between how theorists and pragmatists answer this question differs more in practice than in theory. :(
Your results may vary. :)


This message about pragmatic, patient exploration of unpredictable reality is void where prohibition by lobbyists is accepted by fools. Tolls, taxes, fees, propaganda and orthodox economics may impinge upon your reality-field.