Andrew Gelman — Rosenbaum (1999): Choice as an Alternative to Control in Observational Studies

Paul Rosenbaum’s 1999 paper “Choice as an Alternative to Control in Observational Studies” is really thoughtful and well-written. The comments and rejoinder include an interesting exchange between Manski and Rosenbaum on external validity and the role of theories....

Important in the most studies in social science, including economics, are necessarily observational rather than experimental. The question is how to design observational studies to make them as close as possible to experimental studies where tight control of variables is available.

Design problems involves choice that are implicit assumptions. Designers need to carefully choose (assume) consciously and intentionally rather than presume, which runs the risk of hidden assumptions that might have been avoided through greater advertence.

A good example is the Reinhart and Rogoff historical study on the effect of public debt that was vitiated by inadvertence to the different consequences of public debt under different monetary systems. MMT economists immediately pointed out that the presumption that all public debt is the same in its effects is false, owing to operational differences under different monetary regimes historically. This is actually more significant than the computational errors that were discovered subsequently and highly publicized in the media.

The R&R study was highly influential in policy formulation even though MMT economists had pointed out at the time of its release, and this led to very damaging effects when policy based on the study was implemented. This should not have happened in a professional environment.

Statistical Modeling, Causal Inference, and Social Science
Rosenbaum (1999): Choice as an Alternative to Control in Observational Studies
Andrew Gelman | Professor of Statistics and Political Science and Director of the Applied Statistics Center, Columbia University