Saturday, January 14, 2017

Lars P. Syll — Statistical inference — a self-imposed limitation

The critique of assuming linearly, homeogeneity, additivity and shift invariance holds not only for statistical inference but also reasoning that is deterministic. Physical processes are not necessarily linear — homogenous, additive and shift invariant.

Assuming linearity concerning agents and their relationships, e.g,  through institutional arrangements, requires justification. This point was made by Keynes and later by Robert Lucas in the Lucas critique about effects of changes in policy. Lucas can be viewed as formalizing an aspect of Keynes' criticism of Tinbergen, in this case assuming shift invariance accompanying change in institutional arrangements.
This essay has been devoted to an exposition and elaboration of a single syllogism: given that the structure of all econometric model consists of optimal decision rules of economic agents, and that optimal decision rules vary systematically with changes in the structure of series relevant to the decision maker, it follows that any change in policy will systematically alter the structure of econometric models. — Robert E. Lucas, Jr., Econometric Policy Evaluation: A Critique 
I would not say that econometric models "fail" as much as they are misapplied, misunderstood, or misrepresented. A theory is a causal explanation. A model is an algorithm that expresses this formally. A formal model is not necessarily mathematical. It can also be expresses using formal logic. 

A formal model articulates the structure of an ideal world, where "ideal" means constructed of ideas. As long as the model obeys the rules of logic and mathematics, it is a "good" model. 

The problem is that anything can be stipulated as an assumption without vitiating the model. As long as the formal rules are followed scrupulously it is a "good" model in this sense.

The question is how useful models are in describing a real world world. Obviously, the model will only be as good as the fit between the ideal world articulated by the model and the real world that the model purports to describe in general terms.

There is nothing "wrong" about assuming ceteris paribus in model construction as a formal process. But if a model that assumes cet. par. is applied as an explanation of real world events, the model is vitiated because it cannot be disconfirmed — it is circular. The defender of the model can claim that all things were not held equal, so the model did not actually apply. Well, that is the point! 

An ideal world is unchanging, while the real world is ever-changing. Unchanging ideal models only apply to the real world in the limited instances where the real world is either unchanging or change is so minimal as not be to relevant to the design problem. 

Regarding human agents, this would likely be a matter of nature versus nurture, which terms out to be difficult to distinguish. For example, optimizing is an assumption that is presumed to be a matter of "human nature," as is Mises' "praxeology. " However, these are asserted as self-evident when they are actually either  empirical claims that require substantiation, or else are based on either circular reasoning or gratuitous stipulation.

There is sometime a temptation to conflate a good model in the former sense of coherent and consistent with a good model in the sense of corresponding to observed data and the pragmatic test of usefulness as an explanation. This is illogical since the meaning of "good" has shifted. This is a type of category error.

The fit of a model is determined by how well hypotheses derived from the general description  are confirmed or disconfirmed by observation and comparison with the model. This is a sine qua non of science versus speculation. 

This makes social science, including economics, somewhat tenuous as science in that this kind of testing is generally difficult, impossible, or unethical. This criterion doesn't rule out social science but limits its scope and requires great care in approach. As a result there is a murky boundary between social and political science and social and political philosophy, and speculation may be passed off as science unless care is taken.

In addition to coherence, correspondence and practicality, simplicity aka economy of explanation is also a criterion. An objective in modeling is to construct as simple a model as possible to accomplish the purpose for which the model is designed, for example, Newton's theory of planetary motion.

In Newton's theory of planetary motion, the planets and sun can be represented by points representing their center of gravity without concern with knowing the details of their chemical composition, since that is not relevant to the task of explaining the invariant principles of motion that govern change in terms of scientific "laws" that apply universally and not only to the local solar system. Because "gravity." Gravity is not observable as a data point like a planet but it is measurable as "force." Gravity is a universal law. 

The so-called economic "law of supply and demand" — actually the law of supply and the law of demand as the basis of familiar supply-demand curves that are based on price and quantity — is not a "law" in the same sense as laws of physics that apply to the real world of phenomena. The "laws" of supply and demand are heuristics used to generate graphs similar to what engineers and designers call a thumbnail. They are useful as gadgets in thinking and teaching, but they are not models articulating a scientific theory that explains real world events in that they do not apply to all markets at all times. The assumptions are too restrictive to be representational.

For example, preference is non-homogenous. There is no representative agent with fixed preferences that corresponds to the modeling assumption. The restrictive assumption used to simplify is simplistic with respect to actual agents. This oversimplification might not make a difference in special cases, but it is not universal in application and therefore does not qualify as a general law on which a general theory can be based.

Owing to the elegance of explanations in natural science, scientists in other disciplines are likely to be tempted to emulate this success. While that is a good thing in the sense of striving to observe the four criteria or coherence, correspondence, practicality and economy, it is not a good thing if scientists doing life or social science become unmindful of the limitations imposed by the subject matter of their discipline. The result will be application of inappropriate methodology and exaggerated claims.

Science is generally divided into natural, life, and social science based on subject matter. Science in general aims at causal understanding of a universe in which natural, life and social sciences are aspects of various phenomena.

Consilience is also a requirement in doing science. Scientific theories are expected to corroborate other and not contradict each other in that science aims at a general explanation of "reality."

As a philosopher looking at economics as an amateur, it appears to me that many economists are careless about applying the above criteria and therefore overstate their claims and likely overestimate their knowledge. as being scientific instead of speculative, and objective rather than interested.

This is even before getting into measurement and historical issues. There really needs to be more attention paid to philosophy of science, philosophy of social science in economics and much more work is needed in philosophy (foundations) of economics, which is underdeveloped since so few people have contributed to it. Indeed, a lot of what passes for philosophy of economics now is mostly ideology. Yet, conventional economists claim that methodological questions are settled. NOT!

It would seem that the pragmatic approach in economics would be using the logic of Adriane's thread rather than matching an ideal world to the real world or, worse, attempting to shape the real world to an ideal world determined by ideology. 

This recognizes that economics is dealing with subject matter that is historical, contingent and dependent on choices under uncertainty that are socially and politically influenced in a context of history, culture, and institutional arrangements.  It is a method for exploring alternatives and evaluating options in terms of tradeoffs so as to find the cheese and not end up in blind alleys. This would involve not only looking out the window but also getting out of the ivory tower.

Lars P. Syll’s Blog
Statistical inference — a self-imposed limitation
Lars P. Syll | Professor, Malmo University


Bob said...

Well I'm convinced. What are economists waiting for? Reform should begin NOW.

Matt Franko said...

"many economists are careless about applying the above criteria and therefore overstate their claims and likely overestimate their knowledge."

I think they are definitely doing that... from Romans again: "alleging themselves to be wise, they are made stupid..." Rom 1

But that is not all it takes, there is a previous qualifier of what these people are caught up in that gets us where we are they somehow also "retain the truth in injustice..."

Might be a cocktail of both 1. in some sort of possession of the truth while in injustice and 2. combined with an allegation of their own wisdom to go with it... the result is we are F-ed with these people large and in charge....

Whole thing:

"God's indignation is being revealed ....on all the .... men who are retaining the truth in injustice, ..... Alleging themselves to be wise, they are made stupid..." Rom 1

ie they're being made morons... to our detriment...

jrbarch said...
This comment has been removed by the author.