The Theory Fits Perfectly, So What’s Wrong?

If you like my blog, please tell others.
If you don’t like it, please tell me.

“Hey! Look at this!” Your coworker shows you a chart of the closing price of a certain stock every day for the last month. Then he shows you a graph of a mathematical equation that follows the ups and downs of that stock price almost perfectly. He continues, “I’ve discovered the equation of how the price of this stock goes up and down. It’s going to go up tomorrow. All we have to do is buy now, then sell tomorrow to make some money.” Would you follow his advice? If you do you will probably set yourself up for disappointment.

“But wait,” you might ask. “How can the stock price fit that equation so well up through today and not predict what it will do tomorrow?” The answer is that there can be more than one theory that fits the data and those theories can be quite different. In fact there are an unlimited number of equations that fit your coworker’s data, and they can’t all be right. Some of those equations show the price increasing tomorrow; some show it decreasing. We could even find an equation that shows the stock price as negative tomorrow while still fitting the existing data!*

The same applies to non-mathematical theories. Chicken farmers are well aware that the sun comes up shortly after the rooster crows. They might theorize that the rooster crowing is what causes the sun to rise, but that is hardly proof of the theory. We cannot accept that a theory is correct just because it fits existing data. That is true for things like the coworker’s equation and for other theories. The theory may be true, it may be true in certain circumstances, or it may be only chance that allowed it to fit the data.

The history of science is littered with the ruins of theories later shown to be only approximations or in some cases completely false. Yet the human mind still tends to regard agreement with data as proof of theoretical correctness. Why? Almost certainly because we try to make sense of the world. To do so, we make theories to explain the facts. Then we believe those theories, sometimes even in the face of new facts that contradict them.

That is not all bad. Having a theory of how the world works allows us to live better lives. For example, if the farmer did not have a theory of how the seasons work, he would not be able to effectively plant and harvest his crops. More sophisticated theories allow us to build everything from homes to supercomputers, even rockets to take us to the moon. Those theories can be very useful – but only to the extent that they reflect reality. The theory that vaccines can prevent disease has helped wipe out smallpox and otherwise greatly improved our health. However the old theory that disease was often caused by too much blood in the body led to physicians draining badly needed blood from sick people. That theory probably killed thousands of people.

Humans and other animals seem programmed to seek theories or mental models to explain the world. That can be useful but it can also go too far, leading us into incorrect and sometimes dangerous actions as we try to control important aspects of our lives. B.F. Skinner showed that pigeons fed at random times will develop strange ways of trying to control the food delivery. Whatever action they happen to be doing when food arrives becomes associated in their minds with the food, so they repeat that action when they get hungry. One bird would turn counter-clockwise; others would swing their heads back and forth in a pendulum motion. The birds did that consistently, long after the event that caused them to associate those actions with food. Their theories about what caused the food to appear simply didn’t fit reality.

Humans too are subject to such misleading theories. What sports fan has not tried repeating something he happened to be doing when things went well for his team? I have to admit having been tempted to do such myself. My team was doing poorly while I listened to the game. I had to go do something else and when I got back the team had done much better. It occurred to me that I should turn off the radio so the team would continue to do well. Come on sports fans out there, admit it. What have you done to try to help your team? Is there any rational reason to believe it would work?

Think is limited to people who don’t understand statistics? Think again. Nassim Nicholas Taleb is probably as statistically sophisticated a person as walks the face of this earth. He worked as a trader and one day the taxi dropped him at a different door than where he usually entered the building. That day his account skyrocketed, one of his best days ever. Next morning, without even thinking about it, he asked the taxi driver to drop him at that same door. Then he noticed that he had unconsciously put on the same stained tie he wore the day before! He knows enough to realize that neither the tie nor the door had anything to do with his success, but his natural tendency was still to repeat irrelevant actions from that successful day.

Following theory without or even in spite of evidence is probably nowhere more prevalent than in politics and government. The big government, controlled economy model remains very popular in the world today. In many circles it is considered naïve to even consider anything else. Yet which type of economy has consistently caused problems by over-production and which consistently creates a surplus only of misery? In spite of that abundant evidence, many people still want government to control the economy and make centralized decisions for all of us. (For more on why that does not work well, see my blog on this site, “Who Pays, Who Uses, Who decides?”)

So how can we know that our mental models reflect reality, not just happenstance or some partial data? How can we be certain new data won’t overturn them? Actually we can’t, but with care we can be much more confident and at least weed out the worst of our erroneous theories. Next blog I intend to discuss how to do that.

*For those of mathematical bent and interested in fitting equations to data, we can always fit n + 1 data points with a polynomial of order n. We can fit two data points with a polynomial of the form A + BX. For three points we can use A +BX +CX^2. If our stock price data is for 20 days, a polynomial including values up to X^19 can fit those data perfectly. Then by adding one more term, for X^20, we can fit any next point we want, including negative numbers.