Chemistry for Everyone
Empirical Evidence or Intuition? An Activity Involving the Scientific Method
W
Ken Overway Department of Chemistry, Bridgewater College, Bridgewater, VA 22812;
[email protected] Since a basic understanding of the scientific method is important for introductory science students to learn in the first week of classes, my wife (who teaches in the field of sport psychology) and I were both searching for an activity that would involve students in a brief foray into this cornerstone of science. Subsequently my wife found an article in Parade Magazine (1) that posed and explained the famous Monty Hall game problem, which came from the classic game-show “Let’s Make a Deal” where host Monty Hall offers contestants the choice of three doors, behind one of which is a prize and behind the other two is nothing of value. After the contestant makes a choice, the host eliminates one of the other two doors (that did not have the prize) and asks the contestant if she or he wants to “stay” with his or her original choice or “switch” to the remaining door. Since the game can be played repetitiously in a short period of time, it is ideally suited for an activity where intuition and empirical evidence concerning the best gaming strategy can be compared. When asked about the odds of winning at this game, students and faculty offer a few different answers including: “stay” with the first choice to win (gut instinct), 50兾50 odds either way—so it does not matter (a more analytical approach), or “switch” (for no concrete reason). The beauty of this exercise is that most people do not know the answer but think they know the answer, which sets the stage for an experiment. When I first played the game, I was certain I understood the odds and thus resisted the correct explanation of the true odds. I did not believe it and was going to prove I was right… but I was wrong and shocked when I first saw the empirical evidence. I still did not understand the explanation, but was left with that nonplussed feeling that occurs when you see the evidence but cannot explain it. I was in good company, however, because the same thing had happened to several mathematicians when Marilyn vos Savant presented her solution to the Monty Hall game in her initial Parade publication (1). It prompted a controversial exchange between these mathematicians and vos Savant (2, 3) that resulted in a final exchange in an article published in 1991 (4). It is a case study in misunderstanding and intuition gone awry. Good summaries of the fiasco were published in The Skeptical Inquirer (5) and The American Statistician (6). So what is the winning strategy and what are the odds? Psychologist Art Kohn developed an exercise (7) for his experimental psychology course to empirically determine the odds, which I have adapted to be completed in less than 30 minutes with minimal equipment. Its purpose is to show the players the difference between intuition and evidence and why evidence is the currency of science.
606
Journal of Chemical Education
•
Playing the Game The activity is started when the instructor places three walnut half-shells in plain view of the class and places a dried pea under one shell (just about anything will do here, such as crucibles or cups and a small object to place under one of them). After the shells have been quickly scrambled, the instructor asks the class for a volunteer who wishes to play for a prize by choosing the shell with the pea under it. The student “player” is asked to pick one shell, which is subsequently moved to the side to separate it from the other shells. The instructor then places a folder or some other object in front of the two remaining shells to obstruct the view of the player. The instructor investigates the two shells to discover which one, if any, has the pea. The instructor then presents the player with the two remaining shells and removes one shell that did not have the pea under it. The player is now issued the following challenge: As you can see I have eliminated one of the shells and the pea lies under your shell or the remaining shell. At this time I will offer you the chance to switch shells with me. Do you wish to switch or stay with your first choice?
Without uncovering either of the two shells, the instructor asks the player to clarify why she or he made the choice to switch or stay. This brief discussion is opened up to the entire class where the instructor prompts students to offer intuitionbased and probability-based reasons for success or failure in the game. At a minimum the instructor should ask students to write out their hypothesis about which strategy (to switch or stay) will lead to the highest chance of winning. If the instructor wishes to expand this exercise, optional questions have been provided in the Supplemental MaterialW that can be turned into a handout in order to guide the exercise. The instructor should not reveal the contents of the shells at this point, but will come back to this game after the class activity. After the pre-exercise questions, the instructor proposes to the class the experiment that is outlined on the game sheetW and described herein. The instructor then divides the class into groups of two, where one person is the player and the other person is the game-show host. In each group, the host is given the game sheet that lists a series of 20 trials in which the “prize” letter is listed for each trial. It is important that the player is not shown the game sheet. The host presents the player with three “shells” in the form of the letters A, B, or C. Once the player makes a choice, the host eliminates one of the other choices (knowing the correct choice) and presents the player with a remaining option by saying, for example: You chose A and the prize is not C. Would you like to switch to B or stay with A?
Vol. 84 No. 4 April 2007
•
www.JCE.DivCHED.org
Chemistry for Everyone
Table 1. Results from 43 Student Pairs Section
Probability of Success (%) –
–
(x ± sx)stay
(x ± sx)switch
01
32 ± 17
59 ± 21
02
43 ± 16
67 ± 20
03
41 ± 15
52 ± 21
04
31 ± 10
65 ± 21
05
41 ± 16
67 ± 13
Average
37 ± 14
62 ± 18
–
Note: x is the mean and sx is the pooled standard deviation.
The host records the choice (switch or stay), but does not reveal the success or failure to the player until all trials have been completed, at which point the tally of [(no. of successful switches)兾(total no. of switches)] and the [(no. of successful stays)兾(total no. of stays)] should reveal roughly a 67% and 33% probability, respectively. Twenty trials is enough to get close to these statistics, but pooling the class results on the chalkboard and determining an overall class average for each of the two percentages comes much closer (Table 1 and the two example student-game sheetsW). It does not matter whether the player chooses to always stay, always switch, or vary the choice as long as there are a variety of strategies across the entire class. If all students choose to stay, for example, a solid probability for staying can be determined, but no data for the switching probability would be available. To complete the exercise, the instructor asks each student to reread his or her original hypothesis and determine whether or not it needs amending based on the new empirical evidence. It should be emphasized here that the students have just completed one full cycle of the scientific method. Finally, the instructor should give the original player one last chance to switch or stay before both of the remaining shells are uncovered and the prize is awarded. The point here is that the original player had chosen a strategy and now has the benefit of the pooled statistics of the entire class. Has the evidence convinced him or her of the best strategy? The optional post-exercise questions and readingsW can be used to draw attention to the results, students’ original hypotheses on their gaming strategy, and misunderstandings between scientists who themselves rely on intuition instead of evidence when they communicate. In the Parade articles, many mathematicians reacted skeptically and forcefully to Marilyn vos Savant’s solution to the Monty Hall problem though her explanation was correct for the problem she posed. It is an interesting point to make that even professionals are sometimes misled by intuition and miscommunication.
Figure 1. A pictorial explanation of the Monty Hall problem showing the probabilities before (A) and after (B) one item has been removed.
the statistics correctly the exercise would not be very useful. Intuition in this case usually leads a person to the incorrect answer. That is where the scientific method comes in. Instead of settling the matter with a discussion of what people think is the best strategy, the instructor can ask the students to state their opinion in the form of a hypothesis. It is the point of this exercise to give the students the means to test their hypothesis with an experiment. Empirical evidence measured by the students will show that the probability for winning when switching is 67%, while success when staying is 33%. Figure 1 shows the origin of these odds pictorially. The probability that any one shell will have the pea under it is approximately 1 in 3. By choosing one shell out of the three, the player has grouped the probabilities into a set with 1 /3 odds of containing the pea and another group with 2/3 odds of containing the pea as in Figure 1A. When the host of the game eliminates one shell from the group with 2 /3 odds, the odds for that group does not change but entirely falls on the remaining shell in the group. Thus one shell still has a 1 in 3 chance of containing the pea and the other shell has a 2 in 3 chance of containing the pea (Figure 1B). The resulting odds make the player who switches twice as likely to guess correctly as a player who stays. Data from 5 sections of a general chemistry laboratory course with 43 student-pairs can be seen in Table 1. Discussion
An Explanation My experience so far shows that most people either choose to stay because of a gut feeling or guess the probability is 50% either way. Indeed if most people were to guess
www.JCE.DivCHED.org
•
Instructors should not treat this activity as an exercise in statistics, since its purpose is to demonstrate how the empirical evidence collected as a result of an experiment will either corroborate, modify, or reject the hypotheses the stu-
Vol. 84 No. 4 April 2007
•
Journal of Chemical Education
607
Chemistry for Everyone
dents have stated at the onset of the game. The statistics simply provide the evidence. Several exercises implementing the scientific method have been published in this Journal using history (8), mass measurements (9), organic synthesis (10), and polymer preparation (11). The strength of this activity is that (i) it is accessible to high school and college students, (ii) it can take an opinion that students may hold strongly and dispel it with empirical evidence that is student-measured in less than 30 minutes, and (iii) it requires no foreknowledge of chemistry—only some very basic math skills. This activity models one full cycle of the scientific method: take a familiar problem or game, get opinions and hypotheses about how to beat the game, run game trials to see what results fall out, and then modify the hypotheses based on the results. While definitions of the scientific method may vary from textbook to textbook and scientist to scientist, it is still instructive for students to participate in an activity that shows one example of it. If the instructor is so inclined, this activity could lead to a discussion of other historical examples of the implementation of the scientific method. Rutherford’s scattering experiment is a classic example of how new evidence changed the experiment’s hypothesis and our understanding of the nature of matter. Facets of Einstein’s general theory of relativity could not be verified until decades after its assertion, proving the scientific method is larger than any one scientist and may require several people, over several years, to complete the cycle. Also, Röntgen’s serendipitous discovery of X-rays provides an example of how luck can also be a part of the scientific method. Eurekas and Euphorias by Walter Gratzer (12) is a good source for these and other examples. At the core of the scientific method, however, is a cyclical process that uses information collected at one end to inform the process and any new information. The fact that it is self-correcting is its greatest strength and it is essential for students to understand this. As seen from the furor that erupted when vos Savant presented her solution in the Parade articles, scientists and students need to be reminded the scientific method is firmly grounded in empirical evidence. While theoreticians may perform many calculations on this or that, sooner or later those calculations must be compared against actual, physical systems. Without natural systems as a rule to follow, theoretical models are interesting but meaningless. The current climate change investigations and the general circulation models that require constant adjustment are a testament to this.
608
Journal of Chemical Education
•
Conclusion This demonstration can be used in the classroom or lab during the first week of class or when the scientific method is discussed. It takes about 20 minutes to complete if the preand post questions are excluded, leaving ample time for lab check-in procedures and safety orientation. Provide 30–45 minutes for the exercise if the questions are used. The power of this exercise lies in allowing students to dispel or reaffirm their intuition based on empirical evidence. They see, through experimentation, how observation affirms and sometimes corrects a hypothesis that is made based on intuition or prior evidence. Acknowledgments I gratefully acknowledge the assistance of Lori GanoOverway, assistant professor of Health and Exercise Science at Bridgewater College, for alerting me to the Parade Magazine and the Teaching in Psychology articles upon which this activity was based. It reaffirms my belief that despite the differences between the many disciplines in the sciences, each is built upon the same cornerstone—the scientific method. W
Supplemental Material
Two example student-game sheets, a blank game sheet, and optional questions are available in this issue of JCE Online. Literature Cited 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11.
vos Savant, M. Ask Marilyn. Parade Magazine 1990, Sep 9, 15. vos Savant, M. Ask Marilyn. Parade Magazine 1990, Dec 2, 28. vos Savant, M. Ask Marilyn. Parade Magazine 1991, Feb 17, 12. vos Savant, M. The American Statistician 1991, 45 (4), 347. Posner, G. P. The Skeptical Inquirer 1991, 15, 342–345. vos Savant, M. The American Statistician 1999, 53 (1), 43–51. Kohn, A. Teaching of Psychology 1992, 19 (4), 218–219. Giunta, C. J. J. Chem. Educ. 1998, 75, 1322–1325. Hohman, J. R. J. Chem. Educ. 1998, 75, 1578–1579. Adrian, J. C.; Hull, L. A. J. Chem. Educ. 2001, 78, 529–530. Gilbert, R. G.; Fellows, C. M.; McDonald, J.; Prescott, S. W. J. Chem. Educ. 2001, 78, 1370–1372. 12. Gratzer, W. Eurekas and Euphorias; Oxford University Press: New York, 2002.
Vol. 84 No. 4 April 2007
•
www.JCE.DivCHED.org
