Showing posts with label information theory. Show all posts
Showing posts with label information theory. Show all posts

Thursday, July 12, 2018

Jason Smith — One purpose of information theory

Unfortunately, Shannon's entropy, often referred to as information entropy, and then shortened to just information, is often confused with the colloquial term "information". This brings connotations of data, of knowledge, of specific sets of symbols with specific meaning. But as Shannon and Weaver said in their book from a year later, we must not confuse information theory information with meaning. This collision of terminology is amplified when it encounters economics, where information economics deals specifically with the economic value of meaningful information.

I believe the best way to understand this difference is to understand what information theory illuminates. Information theory gives us a way to quantify concepts when we have limited knowledge about what underlies those concepts. For example, information theory is essentially a more general framework that encompasses thermodynamics in physics — thermodynamics is the science of how collections of atoms behave despite not having remotely enough knowledge about the trillions upon trillions of atoms to make a model. We give up talking about what a single atom in a gas is doing for what an atom could be doing and with what probability. We cease talking about atoms are doing and instead talk about the realm of possibilities (the state space) and the most likely states....
Information Transfer Economics
One purpose of information theory
Jason Smith

Wednesday, May 24, 2017

Jason Smith — More on Hayek and information theory


Longish but not wonkish. Of interest to those interested in this stuff and probably not of general interest.

Information Transfer Economics
More on Hayek and information theory
Jason Smith

Saturday, April 11, 2015

Jason Smith — Information theory and economics, a primer


A physicist looks at economics and simplifies the formal modeling that economists conventionally use.
The physicist Eugene Wigner once referred to the "unreasonable effectiveness of mathematics in the natural sciences". One comment I frequently see out there on the economics blogs is the economics equivalent: economists equations and models don't anything that looks like a real human in them. Some economists actually defend this -- they say they don't need realistic humans or assumptions (microfoundations).

What this information equilibrium framework seems to say is that when markets are functioning properly, this is ok. To turn Tolstoy upside down, all functioning markets are alike; each market failure fails in its own way. Those functioning markets represent a common Platonic ideal -- something that can be described by mathematics. Interestingly, the information equilibrium framework also has some suggestions for how to approach the problem when markets fail, too! It's called non-ideal information equilibrium and it allows us not only to see markets that are functioning (those well-described by ideal information equilibrium), but point out ones that are failing. This also represents a divergence from mainstream -- markets aren't assumed to be functioning unless they can be shown to be functioning.

The basics of the framework come down to a couple of equations and what is effectively a supply and demand diagram -- which should be thought of as an entropic force diagram.
The analogy of the "scissors" of supply and demand can be called upon to summarize this albeit simplistically.  As long as each blade is functioning as it should, e.g., sharp enough to do the work, and the scissors is working correctly as a system, with the blades operating in alignment, the mechanism cuts as it should. However, if one of the blades is not functioning as it should, .e.g., is dull, or the scissors is out of alignment, e.g., the fulcrum screw loosens, then the scissors no longer operates correctly and the cuts are either off or done happen at all, that is, the system fails.

This analogy can be complicated, .e.g, by re;pacing the schism with a milling machine. The difference here is the assumptions underlying the different models. The milling machine has a more parts and more detailed design than a scissors, but it does essentially the same thing — cuts the material precisely.

Ideally, a complicated real process can be modeled usefully though a rather simple model, e.g., in economics using a "gadget" like ISLM. Ockham's razor holds that the simplest explanation for the purpose is to be preferred. Here, the less is less information.

The advantage of the information transfer model is that it simplifies the assumptions while still producing useful results using only "market forces" in modeling.
One major divergence from mainstream economic approaches is the lack of assumptions about what it is that is mediating economic activity. You really don't need any economic agents, firms, households, rational expectations, or really any kind of human thought at all ... when markets are functioning properly. Market forces are like entropic forces in thermodynamics -- diffusion and osmosis are a couple of well known ones. Molecules in a gas don't know about their density at the other side of the room, but collectively they will distribute themselves to achieve an almost equal density across their container. People and prices can be described by the same information equilibrium framework that describes the behavior of molecules. The equilibrium states in this model are the ones with maximum entropy [1].
I think this actually solves a really tough philosophical problem. If humans have free will and don't behave perfectly rationally all the time, why does it seem that when markets are functioning properly, why is mathematics of economics work so well -- as if they were atoms in an ideal gas?
Of course, the really interesting stuff in economics occurs when the system is not in equilibrium at efficient use of available resources, which is described by the equilibrium model.

This is the problem that conventional equilibrium economic modeling like DSGE deals with as the result of exogenous "shock" that could not be foreseen using the model alone.

Thus, for some economists, doing economics involves the "art" of choosing the right model at the right time.

The question then arises is whether and how information transfer economics deals with this in more satisfactory way, e.g., by enabling anticipation from within the model and ideally preventative maintenance to prevent breakdowns that result in idling resources and costly declines in efficiency and effectiveness. Smith seems to suggest that this is not possible in the case of simple information models.

However, it is possible in physics to detect when a situation is becoming unstable based on feedback from data. Let's see where Smith goes with this. As he points out, economics has been quite successful in dealing with static equilibrium models, but challenges arise with introduction of dynamic models as the rate of change of key variables shifts over time with economies in flux.

Hayek is often credited with introducing the concept of information to economics, e.g., in
"The Use of Knowledge in Society". But Hayek's essay raises more questions than it answers, as I think it was intended to do.
It is in many ways fortunate that the dispute about the indispensability of the price system for any rational calculation in a complex society is now no longer conducted entirely between camps holding different political views. The thesis that without the price system we could not preserve a society based on such extensive division of labor as ours was greeted with a howl of derision when it was first advanced by von Mises twenty-five years ago. Today the difficulties which some still find in accepting it are no longer mainly political, and this makes for an atmosphere much more conducive to reasonable discussion. When we find Leon Trotsky arguing that "economic accounting is unthinkable without market relations"; when Professor Oscar Lange promises Professor von Mises a statue in the marble halls of the future Central Planning Board; and when Professor Abba P. Lerner rediscovers Adam Smith and emphasizes that the essential utility of the price system consists in inducing the individual, while seeking his own interest, to do what is in the general interest, the differences can indeed no longer be ascribed to political prejudice. The remaining dissent seems clearly to be due to purely intellectual, and more particularly methodological, differences.
I would venture to say that no serious thinkers today assume that the "price system," that is, "the market, " is in any way dispensable. The question that remains is whether the market is sufficient as an institutional tool for organizing and governing a society. The simple answer is that it will never be discovered because "the market"is based on other institutions, and these institutions are  grounded in non-economy factors such as government not to mention emergence that occurs in complex adaptive systems. Politics follows on the introduction of government, and with it, networks and strata of social relationships based on status (class), power, and property ownership (wealth).

The question then becomes how to incorporate this apparently extraneous information into economics and economic information ("data").  It would seem that this is related to the art of choose the correct model for changing conditions. For example, it would be significant to know where an economy is headed on the curve of financial instability that Hyman Minsky hypothesized based on financial and other information (in the recent crisis, fraud) that economists generally have not taken into account in the conventional approach.

Information Transfer Economics

Tuesday, September 2, 2014

Jason Smith — Which way does the information flow?


Hayek's The Uses of Knowledge in Society (1945) established information as the basis of modern economics. This accords, for example, with the view that the primary function of money is as a unit of account, and that accounting records are primary information sources for the study of both economies and economic principles.

Then the questions become questions about information and its use. This is much clearer in business and finance than economics, however. What emerges is recognition that information is not only a matter of records but also a factor that enters into choices that affect outcomes especially through feedback.

When information becomes the basis for theory in addition to record-keeping, then direction of information flow becomes relevant to causality. Jason Smith present arguments for why the direction of flow of economic information is chiefly from demand to supply rather than vice versa.

Information Transfer Economics 

Thursday, May 10, 2012

Interview with James A. Shapiro, author of Evolution:  A View from the 21st Century

“Given the exemplary status of biological evolution, we can anticipate that a paradigm shift in our understanding of that subject will have repercussions far outside the life sciences. . . .  How such an evolutionary paradigm shift will play out in the physical and social sciences remains to be seen. But it is possible to predict that the cognitive (psychological) and social sciences will have an increased influence on biology, especially when it comes to the acquisition and processing of information.” –James A. Shapiro, Evolution:  A View from the 21st Century

Read it at Counterpunch
The Evolution Paradigm Shift
by Suzan Mazur

How cognitive and information science are challenging existing scientific paradigms. The evolutionary paradigm underlies virtually all other paradigms and so a different understanding of evolution will have reverberating effects across the spectrum of knowledge. While this is not about economics, it is not difficult to see the parallels and implications.

Wednesday, May 9, 2012

Surplus theory, the Information Age, and the Knowledge Society

Surplus theory originated with Ricardo and was later developed by Sraff[a]. The use of surplus theory as a theory of price morphed into the cost limit of price. Surplus theory is not a good theory of price but is a good analysis of production.

On it’s own, surplus theory cannot explain how to achieve full employment and neither can MMT. MMT stops at explaining how full employment can be funded but doesn’t explain how governments should spend money. Using surplus theory and disequilibrium theories, it appears that the way to achieve full employment is by achieving an increasing surplus. Evolution has managed to achieve this consistently over a period of billions of years. Information theory has been used to demonstrate that evolution is a process of genetic information gain, therefore each generation produces an information surplus.

Evolution is often portrayed as a single process, however evolution evolves new processes as well as new individuals. Information gain has accelerated as the initial process of mutation has been enhanced by permutation, sexual selection and intelligence. It’s even arguable that the human ability to manipulate information is a new meta-evolutionary process.

Information gain is a form of accumulation of real capital. According to Paul Romer, this is the most important form of capital and this is how governments should invest surplus labour (the unemployed). Train more people. Employ more teachers. Allow people to study. Allow people to conduct research. Allow people to try and fail because this is also learning. According to evolutionary principles, the most important research is the meta-research, i.e. research about how best to conduct and fund research.
Reposted from heteconomist.com
Comment on Fiat Money is Logically Prior to Capital
by Hacky The Hufrex