Empirical Testing Of Macro Models — Brian Romanchuk

From a bond market practitioner's perspective, model testing is straightforward: does it make money? The ability to make money after model implementation (and not just "out of sample") is a simple quantitative metric -- although one might need to wait for a large enough sample to test this. Not everyone is a market participant, and they want to evaluate models on other metrics (e.g., does it help guide policy decisions?). However, the key insight of the "does is make money?" metric is that it is related to the more vague: "does it offer useful information about the future?" It is entirely possible for a model to have some statistical properties that are seen as "good" -- yet offer no useful information about the future.

Usefulness in Forecasting and Pricing

Although one might be able to make money from a mathematical model any number of ways, I am considering two types of financial models that are of interest.

• Forecasting models that generate buy/sell signals.
• Pricing models. Although this sounds unusual in a macroeconomics context, this is related to DSGE models, given their similarity to arbitrage-free pricing models (link to previous discussion).

Forecasting models are probably what most people would think of, the structure of DSGE models implies a need to worrying about pricing concepts. The key observation one can make about financial forecasting models is that they are not evaluated based on statistical tests (r-squared, whatever), rather the profits they generate after model creation. That is, passing statistical tests does not translate into a useful model.

Since there is less to say about pricing models, I will discuss them first....

Bond Economics
Empirical Testing Of Macro Models
Brian Romanchuk