Kira Radinsky and Yoni Acriche — How to Make Better Predictions When You Don’t Have Enough Data

"Transfer learning. "

.. in order to stay relevant, statisticians will have to get out of the purist position of fitting models that are based solely on direct historical data, and to enrich their models with recent data from similar domains that could better capture current trends. 

This is known as Transfer Learning, a field that helps to solve these problems by offering a set of algorithms that identify the areas of knowledge which are “transferable” to the target domain. This broader set of data can then be used to help “train” the model. These algorithms identify the commonalities between the target task, recent tasks, previous tasks, and similar-but-not-the-same tasks. Thus, they help guide the algorithm to learn only from the relevant parts of the data.

This has been used by military strategist from time immemorial. No battle is the same but familiarity with past operations and outcomes is needed to provide a structure for thinking about present cases by identifying similar patterns, for example terrain. Military strategists have incredibly detailed knowledge of historical operations extending back millennia.

There is a similar field in logic called fuzzy logic.

Bayesianism is a similar technique in statistics.

These are rigorous approaches to heuristics that are appropriate when it is neither practical or possible to do more rigorous analysis. 

Heuristics may be superior to rigorous analysis is time-critical situations, for example. There may also be transaction costs that make more rigorous analysis impractical.

As always, it is a matter of choosing the right tool for the situation and using the tool skillfully. The executives that can do this efficiently and effectively are the most highly paid, and to a great degree it accounts for outsized CEO compensation as long as results corroborate.

Richard Feynman and Enrico Fermi were known as exceptionally good at heuristics. At the time of the testing of the first atomic bomb, the physicists present were, of course, intent on determining the yield and had in fact made bets on it. Enrico Fermi threw some pieces of paper in the air simultaneously with the blast and watched their behavior to estimate the yield. He was remarkably close using this heuristic device. Obviously, a lot of background went into that.

These proto-algorithms are held in the brain as what Michael Polanyi, Karl's equally brilliant brother, called "tacit knowledge." This is a basis for "intuition" as "educated guessing."

The take-away is that there is no sharp distinction between heuristic thinking and rigorous thinking. They generally overlap. Focusing on the contrast between them leads to a false dichotomy. They are complementary modes of thought that blend as they approach each other and more data becomes available.

For example, engineers regularly begin with heuristics, first to estimate the problem and its solution, and secondly, to provide a reference against which to judge the results of more rigorous thinking. If rigor thinking produces a significantly different result from previous estimates, then the solution has to be checked closely to see which went off the rails and how.

Harvard Business Review
How to Make Better Predictions When You Don’t Have Enough Data
Kira Radinsky and Yoni Acriche