The Monty Hall Problem
Jeff Atwood, of Coding Horror, recently resurrected one of the most notorious probability paradoxes around, the Monty Hall problem. Here’s the version that Marilyn Vos Savant famously analyzed in several Parade magazine columns:
Suppose you’re on a game show, and you’re given the choice of three doors. Behind one door is a car, behind the others, goats. You pick a door, say #1, and the host, who knows what’s behind the doors, opens another door, say #3, which has a goat. He says to you, “Do you want to pick door #2?” Is it to your advantage to switch your choice of doors?
Vos Savant’s initial response:
Yes; you should switch. The first door has a 1/3 chance of winning, but the second door has a 2/3 chance. Here’s a good way to visualize what happened. Suppose there are a million doors, and you pick door #1. Then the host, who knows what’s behind the doors and will always avoid the one with the prize, opens them all except door #777,777. You’d switch to that door pretty fast, wouldn’t you?
Some of Vos Savant’s readers were appalled – she printed three letters, whose authors claimed to be PhDs, insisting that she was incorrect, that after the first door has been opened there is a fifty percent chance that the car is behind either of the two remaining unopened doors, and that it therefore makes no difference whether one keeps his initial selection or switches. She subsequently tried to explain why she was actually correct and her critics incorrect, only to receive many more letters, some rather condescending and insulting, insisting that she was indeed wrong. She summarized the responses:
Gasp! If this controversy continues, even the postman won’t be able to fit into the mailroom. I’m receiving thousands of letters, nearly all insisting that I’m wrong, including the Deputy Director of the Center for Defense Information and a Research Mathematical Statistician from the National Institutes of Health! Of the letters from the general public, 92% are against my answer, and and of the letters from universities, 65% are against my answer. Overall, nine out of ten readers completely disagree with my reply.
Now we’re receiving far more mail, and even newspaper columnists are joining in the fray! The day after the second column appeared, lights started flashing here at the magazine. Telephone calls poured into the switchboard, fax machines churned out copy, and the mailroom began to sink under its own weight. Incredulous at the response, we read wild accusations of intellectual irresponsibility, and, as the days went by, we were even more incredulous to read embarrassed retractions from some of those same people!
Vos Savant persevered in her mission of public education, and seems to have made some headway. She subsequently wrote:
Wow! What a response we received! It’s still coming in, but so many of you are so anxious to hear the results that we’ll stop tallying for a moment and take stock of the situation so far. We’ve received thousands of letters, and of the people who performed the experiment by hand as described, the results are close to unanimous: you win twice as often when you change doors. Nearly 100% of those readers now believe it pays to switch. (One is an eighth-grade math teacher who, despite data clearly supporting the position, simply refuses to believe it!)
But many people tried performing similar experiments on computers, fearlessly programming them in hundreds of different ways. Not surprisingly, they fared a little less well. Even so, about 97% of them now believe it pays to switch.
And plenty of people who didn’t perform the experiment wrote, too. Of the general public, about 56% now believe you should switch compared with only 8% before. And from academic institutions, about 71% now believe you should switch compared with only 35% before. (Many of them wrote to express utter amazement at the whole state of affairs, commenting that it altered their thinking dramatically, especially about the state of mathematical education in this country.) And a very small percentage of readers feel convinced that the furor is resulting from people not realizing that the host is opening a losing door on purpose. (But they haven’t read my mail! The great majority of people understand the conditions perfectly.)
And so we’ve made progress! Half of the readers whose letters were published in the previous columns have written to say they’ve changed their minds, and only this next one of them wrote to state that his position hadn’t changed at all.
But not everyone was convinced:
I still think you’re wrong. There is such a thing as female logic.
Don Edwards
Sunriver, Oregon[Vos Savant:] Oh hush, now.
Atwood cites this amazing excerpt from Leonard Mlodinow’s The Drunkard’s Walk: How Randomness Rules Our Lives:
It appears to be a pretty silly question. Two doors are available — open one and you win; open the other and you lose — so it seems self-evident that whether you change your choice or not, your chances of winning are 50/50. What could be simpler? The thing is, Marilyn said in her column that it is better to switch.
Despite the public’s much-heralded lethargy when it comes to mathematical issues, Marilyn’s readers reacted as if she’d advocated ceding California back to Mexico. Her denial of the obvious brought her an avalanche of mail, 10,000 letters by her estimate. If you ask the American people whether they agree that plants create the oxygen in the air, light travels faster than sound, or you cannot make radioactive milk by boiling it, you will get double-digit disagreement in each case (13 percent, 24 percent, and 35 percent, respectively). But on this issue, Americans were united: Ninety-two percent agreed Marilyn was wrong.
Almost 1,000 Ph.D.s wrote in, many of them math professors, who seemed especially irate. “You blew it,” wrote a mathematician from George Mason University. From Dickinson State University came this: “I am in shock that after being corrected by at least three mathematicians, you still do not see your mistake.” From Georgetown: “How many irate mathematicians are needed to change your mind?” And someone from the U.S. Army Research Institute remarked, “If all those Ph.D.s are wrong the country would be in serious trouble.” Responses continued in such great numbers and for such a long time that after devoting quite a bit of column space to the issue, Marilyn decided she whould no longer address it.
The army PhD who wrote in may have been correct that if all those PhDs were wrong, it would be a sign of trouble. But Marilyn was correct. When told of this, Paul Erdos, one of the leading mathematicians of the 20th century, said, “That’s impossible.” Then, when presented with a formal mathematical proof of the correct answer, he still didn’t believe it and grew angry. Only after a colleague arranged for a computer simulation in which Erdos watched hundreds of trials that came out 2-to-1 in favor of switching did Erdos concede that he was wrong.
Monty Fall
Atwood links to Dr. Jeffrey S. Rosenthal’s paper Monty Hall, Monty Fall, Monty Crawl, which, in the course of its analysis of the correct solution to the original problem, also discusses a couple of variants, one of which is the titular Monty Fall:
Monty Fall Problem: In this variant, once you have selected one of the three doors, the host slips on a banana peel and accidentally pushes open another door, which just happens not to contain the car. Now what are the probabilities that you will win the car if you stick with your original selection, versus if you switch to the remaining door?
[Emphasis in the original.]
Does the answer remain the same as for the original problem? See Rosenthal’s paper, and the comments to Atwood’s post, for a discussion.
Simulations
Not trusting myself with a single step through these probabalistic minefields, I ran simulations for both of the aforementioned problems:
Monty Hall
#! /usr/bin/perl -w
use strict;
my ($i, $prize, $door, $open, $win_switch, $win_keep);
for ($i = 0; $i < 500; $i++) {
$prize=int(rand(3));
$door=int(rand(3));
do {$open=int(rand(3))} while ($open == $door || $open == $prize);
($prize == $door) ? $win_keep++ : $win_switch++;
}
print "Keep:\t$win_keep\nSwitch:\t$win_switch\n";
Monty Fall
#! /usr/bin/perl -w
use strict;
my ($i, $prize, $door, $open, $win_switch, $win_keep);
for ($i = 0; $i < 500; $i++) {
$prize=int(rand(3));
$door=int(rand(3));
$open=int(rand(3));
next if ($open == $door || $open == $prize);
($prize == $door) ? $win_keep++ : $win_switch++;
}
print "Keep:\t$win_keep\nSwitch:\t$win_switch\n";
[Don't laugh, perl guys. Yes, the code can be optimized. No, it's not worth it.]
These simulations do indeed confirm the correct answers supplied by theory, although honesty demands the admission that I did not ultimately escape the minefield (or banana peel); my initial modeling of Monty Fall was incorrect, and yielded the wrong answer.1
When I mentioned having written these simulations to my brother, a mathematician, he pointed out that simulations should not really be necessary, since for any predetermined process, there will only be a small finite number of possible outcomes with well defined (unconditional) probabilities, and applying the conditional probability formula should be relatively straightforward. But as he himself conceded, simulations can be useful if one mistrust's his math or reasoning, or as I put it, if Paul Erdős - Paul Erdős, אשר קטנו עבה ממתנינו (at least in the area of mathematics) had enormous trouble accepting the correct answer even when faced with a formal proof, and required a demonstration to become convinced, אנן מה נעני אבתריה ...
Ibn Ezra and Maharshal On the Ineluctable Possibility Of Human Error
Ibn Ezra's famous criticism of the poetry of the Kallir contains an eloquent insistence upon the fundamental equality of all men and a consequent intellectual freedom to criticize others, no matter how great they may be:
יש אומרים אין משיבין את הארי אחר מותו, התשובה: רוח קל עשתנו כלנו, ומחומר קורצו הקדמונים כמונו, ואוזן מלים תבחן. וכלנו נדע כי דניאל היה נביא, ורב על כל חרטומי בבל וחכמיה. והנה אמרו חכמים ז"ל, טעה דניאל בחשבונו, והחשבון הוא דבר קל. ועוד כי ירמיה הנביא בזמן דניאל היה, ואחר שהראו חכמינו הראיה על טעותו, האמור יאמר להם אילו היה דניאל חי היה מטעה המטעים אותו?!2
The reference to Daniel's error in חשבון is apparently a reference to the Sugya in Megillah 11b - 12a, but it is unclear to me how Ibn Ezra understood the passage. In particular, I do not understand what Ibn Ezra means by his assertion that the Sages "showed [Yirmiyahu] a demonstration of [Daniel's] error".
Another famous clarion call for intellectual independence is that of Maharshal, articulating clearly a position that he is indeed noted for exemplifying. He goes so far as to call "the present generation" "weak and feeble" for its inability to comprehend the possibility of error by a great author, and to excoriate it for its overly deferential attitude toward earlier (Halachic) authority:
וזו היא שיטתי, להביא כל הדיעות, בין הקדמונים, בין האחרונים, ולא נשאתי פנים לשום מחבר. אף שהדור שלפנינו לעת ההיא, מרוב חולשת ורפיון ידם, אין יכולת בשכלם להשיג, שגדול אחד מן המחברים יטעה בדמיונו, וסוברים מה שכתב בכתב ישן אין להרהר אחריו, ואין נותנים טעם, אלא לסתור דברי חבירו. וכל מה שיוצא מפי אדם, אפילו הוצק חן בשפתים, ופיו מפיק מרגליות. אפילו הכי אומרים מה גברא מגוברין. הלא יש גם לנו לשון למודים, ויד ושם בתלמוד כמותו.
אבל האמונה, באמונת שמים, שהוכחתי כמה פעמים, בפרט מן הפוסקים האחרונים, שטעו בכמה מקומות מן התלמוד. כאחד מן התלמידים הטועים, בענין עיון הלכה. על כן שמתי פני כחלמיש. ואומר אמרתי בני עליון כולהון, אכן כאדם ידרושון. ולכן לא אאמין לשום אחד מן המחברים יותר מחבירו. אף שיש הכרע גדול בין מעלותם, למי שמורגל בהם בעיון רב. מכל מקום התלמוד הוא המכריע, וראייות ברורות יצדקו ויתנו עדיהן. ולפעמים מחבר אחד כיון להלכה, ולא מטעמיה. מי היה גדול בחכמה ובמניין כרבינו תם בזמנו. והוא התיר אשה שזינתה עם הגוי לכשיתגייר (כתובות ג' ע"ב תוספות ד"ה ולדרוש), וכתבו עליו כל האחרונים, אשר לא הגיעו למעלתו, שטעה בראיות ובמופת.. וחזרו הרא"מ והרא"ש (שם פרק א' סימן ד') וקיימו את דינו, וכתבו דהלכתא כותיה, ולא מטעמיה3
The conventional assumption is that the cause of a great man's error must be momentary carelessness, as in: כי ניים ושכיב רב אמר להאי שמעתתא4. But in light of the above, we must consider an entirely different possibility; perhaps there are some conceptual areas in which human intuition goes terribly awry, and and exerts such a powerful push in the wrong direction that even overpowering rational evidence of the correct approach can be summarily ignored, as we have see in Erdős's reaction to the Monty Hall problem.