Bad Assumptions, Bad Results
One of the most important questions facing decision-makers is: Why are forecasts from so many of today’s sophisticated economic models so often disappointing?
The easy answer is, GIGO.
“GIGO,” as everyone knows and Wikipedia explains, “is an acronym for Garbage In, Garbage Out. It is a fundamental computing and data science principle stating that the quality of an output is strictly determined by the quality of the input. If flawed, biased, or poor-quality data (garbage) is fed into a system, the resulting analysis or output will be equally flawed (garbage), regardless of how advanced the algorithm is.”
So, bad stuff in, bad stuff out.
It is not a new idea.
Charles Babbage was using it in France a hundred years before computers existed.
Its current popularity began in the middle of the 20th century when computer programmers and engineers used it to describe how flawless code can deliver bad results when it is fed bad data.
But the failures of so many forecasting models used by governments as well as those used throughout the business and investing world are about more than bad data.
Sometimes it is bad data.
Sometimes it is researchers and programmers breaking the rules.
Far too often, it is ignoring the universal and timeless truth that false assumptions lead to false results.
The rules are simple: A legitimate statistical study begins by stating a hypothesis that describes a relationship to be tested. The hypothesis is based on observations, and the more accurate the descriptions of the hypothesis and the observations behind it, the more valid the study. Relevant data, which is defined by the hypothesis, is then collected, and an applicable statistical procedure is used to conclude that the hypothesis should either be rejected or not rejected. Concluding that the hypothesis should be accepted is not an option because a hypothesis, when restated, becomes a theory, and one of the conditions of a theory is that it is a statement that can be disproved.
And because of the difficulty of obtaining and processing data by hand, early statistical work was conducted by researchers who followed the rules, spending a lot of time and energy developing reasoned hypotheses based on observations and careful assumptions before collecting data.
Computers changed all that by making it easy to dump whatever data was available into a computer and letting a model identify any possible (but not previously explained) relationships between variables.
The faster and more accessible computers became, the more researchers focused on feeding data into models instead of carefully examining the assumptions on which their models were built.
Without any big announcements, it had become acceptable to ignore the rules.
Which is why, whether it is AI or widely-used statistical programs, someone has to know if the data being fed into a computer is or is not garbage. Someone has to know if the assumptions in the model are or are not knowingly false. And someone has to know if the results are or are not garbage.
Computers have become faster.
Models have become more sophisticated.
Data has become almost limitless.
But none of that changes the fact that mistakes can be made whenever assumptions do not match reality.
And there may be no better description of the importance of reality than Mike Tyson’s famous and often-quoted response to a reporter who asked him if he was worried about Evander Holyfield and his fight plan. Tyson’s answer was, “Everyone has a plan until they get punched in the mouth.”
Models built on market-based economics are no different. They can look elegant, sophisticated, and convincing until they get punched in the mouth by reality.
Which is why eliminating false assumptions may be the most important thing investors, businesspeople, and policymakers can do to avoid doing too many stupid things.
Bad stuff in, bad stuff out, no matter how sophisticated the models are.
That is something everyone should already know.
What individuals, businesses, investors, governments, and publications may not already know is how much better the world could be if economic thinking were connected to reality.
DENNIS F. PAULAHA
