[Note: Since today is Pearl Harbor day I am posting this Monday instead of Tuesday. I’m also interrupting the “Health Care – Something’s Missing” sequence. I plan to return to that series on Friday.]
The fighter planes, dive bombers, and torpedo planes dropped out of the clouds over the 2800-foot Koolau Range. Surprise was complete. Strafing fighters knocked out the U.S. planes before they could get off the ground. Bombers and torpedo planes sunk every warship in Pearl Harbor. Then the attackers returned to their carriers and escaped.
You almost certainly never heard of that attack. No it did not happen on December 7, 1941. It took place on February 7, 1932. The attackers were under the command of U.S. Admiral H. E. Yarnell and of course the bombs and other weapons were simulated, part of a war game.* That day that does not live in infamy – but it should.
The sad fact is that Yarnell’s operation offered some lessons that our naval authorities missed. The Japanese attack in 1941 was a near duplicate of that simulation. Both were on Sunday when defenses were down, both used the same pattern of attack, even using clouds over the same mountain range to conceal their approach. However the Japanese bombs and torpedoes were real and in 1941 the ships were sunk in reality instead of in simulation.
What was the reaction of the admiralty? Some officers saw the lesson and wanted to incorporate it into naval operations. They were overruled. The navy remained organized around battleships and cruisers, with aircraft carriers as a stepchild. Worse, decision-makers ignored the possibility that the Japanese might copy Yarnell’s plan.
The Japanese did not ignore it. Their spies were watching and reports quickly made their way to Tokyo. Those reports may have played a part in Yamamoto’s planning (though there were other sources he may also have used). What is clear is that U.S. involvement in World War II would have started very differently had our authorities learned from Yarnell’s operation and taken measures to defend against such an attack in a real war. The total lack of preparation for air attack made it easy for the Japanese.
That day in 1932 should live in infamy because of U.S. refusal to learn the obvious lessons. Furthermore it should live in infamy as a reminder to each of us that we are subject to similar blind spots. That is part of being human.
The fact is that we do not see the world as it is, the world is much too complicated for that. Our minds filter what we perceive so that we see only what seems important to us. A group of Harvard psychologists demonstrated this by showing a video of basketball players passing the ball. They asked people to count the number of passes made. During the video either a woman with an umbrella or a man in a gorilla costume would walk through the action. Only half noticed the gorilla and only 65% noticed the woman. Those distractions were not necessary to the task so many people filtered them out.
No, we don’t see the world as it is, we see a model of that world. Our mind creates that model by paying selective attention to what we regard as important. That is the only manner we can make sense of this world. So it has always been, and so it will always be unless we somehow become omniscient. The difference between success and failure is not whose model contains the greatest amount of information. It is not even necessarily whose model is most accurate. No, that difference is whose model is most accurate in characteristics relevant to the issue at hand.
Air power was not an important part of the model held by U.S. Navy decision-makers so they filtered out Yarnell’s success. Their model of warfare had been effective during World War I but the world had changed. Following the standard procedure of using the time between wars to learn how to fight the last war better, U.S. commanders ignored important information. Failure to incorporate that information into their model led to disaster.
So what does a failure from 77 years ago have to do with us today? The answer is that human nature has not changed. We still see only our own model of the world and human nature still militates against changing that model. Each of us has a model of the world, correct in some regards, incorrect in others, and simply not including other parts of the world. This affects how we live our individual, family, and work lives. It also affects how we vote and how those we elect govern us. Parents, employees, managers, politicians. All see their own model of the world, not the real world itself. Those models are all imperfect. The effectiveness of their decisions depends on how those imperfections fit with important aspects of those decisions. If the errors in the model are important to their decisions, they will have no choice but to make bad decisions.
Can we overcome this problem? Not completely but we can do better. Gonzales book, Deep Survival, points out that survivors are people who are willing to recognize the imperfections in their model of the world, and to change that model to fit new information. Those who refuse to do this may get by but are setting themselves up for disaster when their model does not match important aspects of reality. Sowell in his book, The Vision of the Anointed, makes a similar point about political life. He points out that many with the “unconstrained vision” simply refuse to admit that they might be wrong. It is not that their model is faulty, that happens to everybody. Their problem is that they do not adapt their model of the world to available information. Like the navy brass after 1932, they continue down the path to disaster.
What can we do about all this? Perfection is not available to humans so we have to do the best we can. That means recognizing that our models of the world are imperfect and always will be. However it also includes continually improving in those models by seeking and accepting new information. We can also insist that politicians do the same. Only in that way can we improve how we see the world with consequent improvement in our decisions and lives.
*My source is Edwin Muller’s article, “The Inside Story of Pearl Harbor, Reader’s Digest, April 1944 reprinted in Secrets & Spies, Reader’s Digest Association, 1964. Several shorter but more readily available accounts can be found by a web search for “1932 Pearl Harbor Attack.”
If you like my blog, please tell others.
If you don’t like it, please tell me.
Showing posts with label model. Show all posts
Showing posts with label model. Show all posts
Monday, December 7, 2009
Wednesday, July 1, 2009
Figures Don't Lie - But...
If you like my blog, please tell others.
If you don’t like it, please tell me.
Figures don’t lie, but liars figure. So do people who figure with good intentions but inadequate skill or thought.
It was all over the news. “U.S. gun stores and gun shows are the source of more than 90 percent of the weapons being used by Mexico's ruthless drug cartels, according to U.S. and Mexican law enforcement officials.” However the story had a bit of a problem – it was dead wrong, the result of misuse of data.
William La Jeunesse and Maxim Lott of Fox News published the facts. That number comes from a limited and abnormal sample of guns recovered by Mexican authorities. During 2007 and 2008 Mexico recovered 29,000 guns at crime scenes. They submitted 11,000 of those to the US for tracing. Only 6,000 could actually be traced with 90% of those 6,000 coming from the US. Over 23,000 of the guns either could not be traced or weren’t even submitted because there was no reason to do so. Many obviously came from Russia, South America, China etc
This is one example of the errors caused by poor understanding of what data means. The claim that 90% of those guns came from the US fooled a lot of people who should have known better, even the US secretary of state. Some may have been deliberately trying to mislead us, but most were probably just victims of the normal human failure to understand statistics. We tend to think that what is easily visible is representative of the whole.
Worse yet, we often think that anything precisely calculated must be correct. The claim that 90% of Mexican criminal weapons came from the U.S. was very precise – and very wrong. That number comes from equations that somebody manipulated to perfection, based on poor input. The source seems to have forgotten that applied mathematics is much more than the manipulation of equations, it requires understanding of the “physics” behind those equations and of the information fed to the equations. Even if the equations are perfect, they will give the wrong answer if the numbers used in the calculation are bad, Garbage In Garbage Out (GIGO).
This is not limited to politics. Two winners of the Nobel Memorial Prize for Economics served on the board of Long Term Capital Management, an investment firm which failed spectacularly. Their calculation was impeccable but accuracy in calculation was not enough to warn them of the oncoming problem.
I admit that I have a certain affinity for mathematics. I can enjoy grinding through equations and calculating results. However in this blog it would be a mistake to provide a treatise on how to calculate probabilities (though I would have fun doing that). First, I would probably bore any readers I have and lose them. Secondly, the bigger issue is not the actual calculation but determination of what goes into those equations. That is why Taleb* claims that “mathematics is a tool to meditate, not compute.” Anyone interested in the means of calculation can take any number of classes or read any of a variety of good books on the subject. Let us here confine ourselves to the “meditation” aspect of the problem.
What ought we to “meditate” on? First we should think about why we even consider any theory or model of the world in the first place. What makes it attractive to us? Its simplicity? Its usefulness? It’s beauty? (Make no mistake, scientists and mathematicians prefer beauty in their theories.) Are we being mislead because of our preferences?
Second, what other theories might explain the facts? Is there any reason to prefer one over another?
Third, are we considering all the facts? Are the facts we consider pertinent to the question at hand?
Forth, how can we know if the theory or model works? I’ve previously indicated that in any situation there are probably an unlimited number of theories that fit the facts. We need a way of distinguishing between those likely to be true and the rest of them. This is a knotty problem, one has troubled science for centuries and will almost certainly continue to cause trouble. The best answer we know comes from philosopher of science Karl Popper. His answer is that the theory must be falsifiable. By that me meant that it must make predictions that can be tested with clear results that might disagree with the theory.
For example, Newton’s famous second law of motion says that the acceleration of a body is directly proportional to the applied force. We can measure the force and the acceleration on a wide variety of objects under many circumstances. When we do so we find that in most cases the law holds to as accurately as we are able to measure. However at speeds approaching the velocity of light the data contradict that law. Newton’s second law is wrong, though it is very useful at ordinary speeds. At higher speeds we must use Einstein’s theory of relativity.
In the political arena we have many theories of how government should work. Frankly, none of them are perfect, probably because human beings are imperfect. However some work better than others. The unfortunate problem is that politicians seldom pay attention to the results when they try to modify a system. Market economies consistently out-produce controlled economies, yet politicians (and many voters) consistently ignore that fact as they increase government power and restrict individual and market freedom.
Next issue I intend to address one reason that government control causes problems with the economy.
*Nassim Nicholas Taleb, Fooled by Randomness, p216
If you don’t like it, please tell me.
Figures don’t lie, but liars figure. So do people who figure with good intentions but inadequate skill or thought.
It was all over the news. “U.S. gun stores and gun shows are the source of more than 90 percent of the weapons being used by Mexico's ruthless drug cartels, according to U.S. and Mexican law enforcement officials.” However the story had a bit of a problem – it was dead wrong, the result of misuse of data.
William La Jeunesse and Maxim Lott of Fox News published the facts. That number comes from a limited and abnormal sample of guns recovered by Mexican authorities. During 2007 and 2008 Mexico recovered 29,000 guns at crime scenes. They submitted 11,000 of those to the US for tracing. Only 6,000 could actually be traced with 90% of those 6,000 coming from the US. Over 23,000 of the guns either could not be traced or weren’t even submitted because there was no reason to do so. Many obviously came from Russia, South America, China etc
This is one example of the errors caused by poor understanding of what data means. The claim that 90% of those guns came from the US fooled a lot of people who should have known better, even the US secretary of state. Some may have been deliberately trying to mislead us, but most were probably just victims of the normal human failure to understand statistics. We tend to think that what is easily visible is representative of the whole.
Worse yet, we often think that anything precisely calculated must be correct. The claim that 90% of Mexican criminal weapons came from the U.S. was very precise – and very wrong. That number comes from equations that somebody manipulated to perfection, based on poor input. The source seems to have forgotten that applied mathematics is much more than the manipulation of equations, it requires understanding of the “physics” behind those equations and of the information fed to the equations. Even if the equations are perfect, they will give the wrong answer if the numbers used in the calculation are bad, Garbage In Garbage Out (GIGO).
This is not limited to politics. Two winners of the Nobel Memorial Prize for Economics served on the board of Long Term Capital Management, an investment firm which failed spectacularly. Their calculation was impeccable but accuracy in calculation was not enough to warn them of the oncoming problem.
I admit that I have a certain affinity for mathematics. I can enjoy grinding through equations and calculating results. However in this blog it would be a mistake to provide a treatise on how to calculate probabilities (though I would have fun doing that). First, I would probably bore any readers I have and lose them. Secondly, the bigger issue is not the actual calculation but determination of what goes into those equations. That is why Taleb* claims that “mathematics is a tool to meditate, not compute.” Anyone interested in the means of calculation can take any number of classes or read any of a variety of good books on the subject. Let us here confine ourselves to the “meditation” aspect of the problem.
What ought we to “meditate” on? First we should think about why we even consider any theory or model of the world in the first place. What makes it attractive to us? Its simplicity? Its usefulness? It’s beauty? (Make no mistake, scientists and mathematicians prefer beauty in their theories.) Are we being mislead because of our preferences?
Second, what other theories might explain the facts? Is there any reason to prefer one over another?
Third, are we considering all the facts? Are the facts we consider pertinent to the question at hand?
Forth, how can we know if the theory or model works? I’ve previously indicated that in any situation there are probably an unlimited number of theories that fit the facts. We need a way of distinguishing between those likely to be true and the rest of them. This is a knotty problem, one has troubled science for centuries and will almost certainly continue to cause trouble. The best answer we know comes from philosopher of science Karl Popper. His answer is that the theory must be falsifiable. By that me meant that it must make predictions that can be tested with clear results that might disagree with the theory.
For example, Newton’s famous second law of motion says that the acceleration of a body is directly proportional to the applied force. We can measure the force and the acceleration on a wide variety of objects under many circumstances. When we do so we find that in most cases the law holds to as accurately as we are able to measure. However at speeds approaching the velocity of light the data contradict that law. Newton’s second law is wrong, though it is very useful at ordinary speeds. At higher speeds we must use Einstein’s theory of relativity.
In the political arena we have many theories of how government should work. Frankly, none of them are perfect, probably because human beings are imperfect. However some work better than others. The unfortunate problem is that politicians seldom pay attention to the results when they try to modify a system. Market economies consistently out-produce controlled economies, yet politicians (and many voters) consistently ignore that fact as they increase government power and restrict individual and market freedom.
Next issue I intend to address one reason that government control causes problems with the economy.
*Nassim Nicholas Taleb, Fooled by Randomness, p216
Wednesday, June 24, 2009
The Theory Fits Perfectly, So What’s Wrong?
If you like my blog, please tell others.
If you don’t like it, please tell me.
“Hey! Look at this!” Your coworker shows you a chart of the closing price of a certain stock every day for the last month. Then he shows you a graph of a mathematical equation that follows the ups and downs of that stock price almost perfectly. He continues, “I’ve discovered the equation of how the price of this stock goes up and down. It’s going to go up tomorrow. All we have to do is buy now, then sell tomorrow to make some money.” Would you follow his advice? If you do you will probably set yourself up for disappointment.
“But wait,” you might ask. “How can the stock price fit that equation so well up through today and not predict what it will do tomorrow?” The answer is that there can be more than one theory that fits the data and those theories can be quite different. In fact there are an unlimited number of equations that fit your coworker’s data, and they can’t all be right. Some of those equations show the price increasing tomorrow; some show it decreasing. We could even find an equation that shows the stock price as negative tomorrow while still fitting the existing data!*
The same applies to non-mathematical theories. Chicken farmers are well aware that the sun comes up shortly after the rooster crows. They might theorize that the rooster crowing is what causes the sun to rise, but that is hardly proof of the theory. We cannot accept that a theory is correct just because it fits existing data. That is true for things like the coworker’s equation and for other theories. The theory may be true, it may be true in certain circumstances, or it may be only chance that allowed it to fit the data.
The history of science is littered with the ruins of theories later shown to be only approximations or in some cases completely false. Yet the human mind still tends to regard agreement with data as proof of theoretical correctness. Why? Almost certainly because we try to make sense of the world. To do so, we make theories to explain the facts. Then we believe those theories, sometimes even in the face of new facts that contradict them.
That is not all bad. Having a theory of how the world works allows us to live better lives. For example, if the farmer did not have a theory of how the seasons work, he would not be able to effectively plant and harvest his crops. More sophisticated theories allow us to build everything from homes to supercomputers, even rockets to take us to the moon. Those theories can be very useful – but only to the extent that they reflect reality. The theory that vaccines can prevent disease has helped wipe out smallpox and otherwise greatly improved our health. However the old theory that disease was often caused by too much blood in the body led to physicians draining badly needed blood from sick people. That theory probably killed thousands of people.
Humans and other animals seem programmed to seek theories or mental models to explain the world. That can be useful but it can also go too far, leading us into incorrect and sometimes dangerous actions as we try to control important aspects of our lives. B.F. Skinner showed that pigeons fed at random times will develop strange ways of trying to control the food delivery. Whatever action they happen to be doing when food arrives becomes associated in their minds with the food, so they repeat that action when they get hungry. One bird would turn counter-clockwise; others would swing their heads back and forth in a pendulum motion. The birds did that consistently, long after the event that caused them to associate those actions with food. Their theories about what caused the food to appear simply didn’t fit reality.
Humans too are subject to such misleading theories. What sports fan has not tried repeating something he happened to be doing when things went well for his team? I have to admit having been tempted to do such myself. My team was doing poorly while I listened to the game. I had to go do something else and when I got back the team had done much better. It occurred to me that I should turn off the radio so the team would continue to do well. Come on sports fans out there, admit it. What have you done to try to help your team? Is there any rational reason to believe it would work?
Think is limited to people who don’t understand statistics? Think again. Nassim Nicholas Taleb is probably as statistically sophisticated a person as walks the face of this earth. He worked as a trader and one day the taxi dropped him at a different door than where he usually entered the building. That day his account skyrocketed, one of his best days ever. Next morning, without even thinking about it, he asked the taxi driver to drop him at that same door. Then he noticed that he had unconsciously put on the same stained tie he wore the day before! He knows enough to realize that neither the tie nor the door had anything to do with his success, but his natural tendency was still to repeat irrelevant actions from that successful day.
Following theory without or even in spite of evidence is probably nowhere more prevalent than in politics and government. The big government, controlled economy model remains very popular in the world today. In many circles it is considered naïve to even consider anything else. Yet which type of economy has consistently caused problems by over-production and which consistently creates a surplus only of misery? In spite of that abundant evidence, many people still want government to control the economy and make centralized decisions for all of us. (For more on why that does not work well, see my blog on this site, “Who Pays, Who Uses, Who decides?”)
So how can we know that our mental models reflect reality, not just happenstance or some partial data? How can we be certain new data won’t overturn them? Actually we can’t, but with care we can be much more confident and at least weed out the worst of our erroneous theories. Next blog I intend to discuss how to do that.
*For those of mathematical bent and interested in fitting equations to data, we can always fit n + 1 data points with a polynomial of order n. We can fit two data points with a polynomial of the form A + BX. For three points we can use A +BX +CX^2. If our stock price data is for 20 days, a polynomial including values up to X^19 can fit those data perfectly. Then by adding one more term, for X^20, we can fit any next point we want, including negative numbers.
If you don’t like it, please tell me.
“Hey! Look at this!” Your coworker shows you a chart of the closing price of a certain stock every day for the last month. Then he shows you a graph of a mathematical equation that follows the ups and downs of that stock price almost perfectly. He continues, “I’ve discovered the equation of how the price of this stock goes up and down. It’s going to go up tomorrow. All we have to do is buy now, then sell tomorrow to make some money.” Would you follow his advice? If you do you will probably set yourself up for disappointment.
“But wait,” you might ask. “How can the stock price fit that equation so well up through today and not predict what it will do tomorrow?” The answer is that there can be more than one theory that fits the data and those theories can be quite different. In fact there are an unlimited number of equations that fit your coworker’s data, and they can’t all be right. Some of those equations show the price increasing tomorrow; some show it decreasing. We could even find an equation that shows the stock price as negative tomorrow while still fitting the existing data!*
The same applies to non-mathematical theories. Chicken farmers are well aware that the sun comes up shortly after the rooster crows. They might theorize that the rooster crowing is what causes the sun to rise, but that is hardly proof of the theory. We cannot accept that a theory is correct just because it fits existing data. That is true for things like the coworker’s equation and for other theories. The theory may be true, it may be true in certain circumstances, or it may be only chance that allowed it to fit the data.
The history of science is littered with the ruins of theories later shown to be only approximations or in some cases completely false. Yet the human mind still tends to regard agreement with data as proof of theoretical correctness. Why? Almost certainly because we try to make sense of the world. To do so, we make theories to explain the facts. Then we believe those theories, sometimes even in the face of new facts that contradict them.
That is not all bad. Having a theory of how the world works allows us to live better lives. For example, if the farmer did not have a theory of how the seasons work, he would not be able to effectively plant and harvest his crops. More sophisticated theories allow us to build everything from homes to supercomputers, even rockets to take us to the moon. Those theories can be very useful – but only to the extent that they reflect reality. The theory that vaccines can prevent disease has helped wipe out smallpox and otherwise greatly improved our health. However the old theory that disease was often caused by too much blood in the body led to physicians draining badly needed blood from sick people. That theory probably killed thousands of people.
Humans and other animals seem programmed to seek theories or mental models to explain the world. That can be useful but it can also go too far, leading us into incorrect and sometimes dangerous actions as we try to control important aspects of our lives. B.F. Skinner showed that pigeons fed at random times will develop strange ways of trying to control the food delivery. Whatever action they happen to be doing when food arrives becomes associated in their minds with the food, so they repeat that action when they get hungry. One bird would turn counter-clockwise; others would swing their heads back and forth in a pendulum motion. The birds did that consistently, long after the event that caused them to associate those actions with food. Their theories about what caused the food to appear simply didn’t fit reality.
Humans too are subject to such misleading theories. What sports fan has not tried repeating something he happened to be doing when things went well for his team? I have to admit having been tempted to do such myself. My team was doing poorly while I listened to the game. I had to go do something else and when I got back the team had done much better. It occurred to me that I should turn off the radio so the team would continue to do well. Come on sports fans out there, admit it. What have you done to try to help your team? Is there any rational reason to believe it would work?
Think is limited to people who don’t understand statistics? Think again. Nassim Nicholas Taleb is probably as statistically sophisticated a person as walks the face of this earth. He worked as a trader and one day the taxi dropped him at a different door than where he usually entered the building. That day his account skyrocketed, one of his best days ever. Next morning, without even thinking about it, he asked the taxi driver to drop him at that same door. Then he noticed that he had unconsciously put on the same stained tie he wore the day before! He knows enough to realize that neither the tie nor the door had anything to do with his success, but his natural tendency was still to repeat irrelevant actions from that successful day.
Following theory without or even in spite of evidence is probably nowhere more prevalent than in politics and government. The big government, controlled economy model remains very popular in the world today. In many circles it is considered naïve to even consider anything else. Yet which type of economy has consistently caused problems by over-production and which consistently creates a surplus only of misery? In spite of that abundant evidence, many people still want government to control the economy and make centralized decisions for all of us. (For more on why that does not work well, see my blog on this site, “Who Pays, Who Uses, Who decides?”)
So how can we know that our mental models reflect reality, not just happenstance or some partial data? How can we be certain new data won’t overturn them? Actually we can’t, but with care we can be much more confident and at least weed out the worst of our erroneous theories. Next blog I intend to discuss how to do that.
*For those of mathematical bent and interested in fitting equations to data, we can always fit n + 1 data points with a polynomial of order n. We can fit two data points with a polynomial of the form A + BX. For three points we can use A +BX +CX^2. If our stock price data is for 20 days, a polynomial including values up to X^19 can fit those data perfectly. Then by adding one more term, for X^20, we can fit any next point we want, including negative numbers.
Subscribe to:
Posts (Atom)
