Research: Science and Education edited by
Chemical Education Research
Diane M. Bunce The Catholic University of America Washington, D.C. 20064
What Are Students Thinking When They Pick Their Answer? A Content Analysis of Students’ Explanations of Gas Properties Michael J. Sanger* and Amy J. Phelps Department of Chemistry, Middle Tennessee State University, Murfreesboro, TN 37132; *
[email protected] Nurrenbern and Pickering (1) have shown that students who can solve mathematical chemistry problems often have difficulty answering conceptual problems covering the same topics, especially if these problems address concepts at the particulate level. Several chemical education researchers have subsequently corroborated these results using the particulate pictures reported by Nurrenbern and Pickering (2–10) and other pictures (11–17). At about the same time, Johnstone (18) noted that while chemists think about chemical reac-
The following diagram represents a cross-sectional area of a steel tank filled with hydrogen gas at 20 °C and 3 atm pressure. (The dots represent the distribution of H2 molecules.)
tions using three separate yet related representations (macroscopic, microscopic, and symbolic), most chemistry instruction in high school and college courses occurs at the symbolic level. Further research (3, 14, 19–22) has shown that when students receive instruction using particulate drawings, they are better able to answer these types of questions. These results imply that the difficulty students have in answering particulate questions more often stems from students’ lack of familiarity with these types of questions than from an inability to answer particulate questions. One of the particulate questions used by Nurrenbern and Pickering, which appears in Figure 1, asks students to predict how the distribution of gas particles in a steel container changes when the gas sample is cooled. Although several researchers (and countless instructors) have used this specific question (1–5, 7, 8, 10), no one has reported an analysis of students’ molecular-level explanations regarding their choice. That is the goal of this paper. In particular, we were interested to determine whether this question is a valid measure of students’ conceptual understanding of kinetic-molecular theory and the behavior of gas particles in an ideal gas. This study addressed the following research questions: 1. What are the most common ideas expressed by students in their molecular-level explanations for their choice of distractor in the question?
Which of the following diagrams illustrate the distribution of H2 molecules in the steel tank if the temperature is lowered to ᎑20 °C?
2. Do students choosing the scientifically accepted answer provide a scientifically accepted explanation of gas particle behavior?
(A)
3. Do students choosing an incorrect answer demonstrate an incorrect view of gas particle behavior?
(B)
Methodology
(C)
(D)
Figure 1. Adapted1 from a multiple-choice question reported in Nurrenbern and Pickering (1).
870
Journal of Chemical Education
•
Subjects Students were asked to answer the particulate question (Figure 1) and to explain what would happen to the gas particles at the microscopic level when the gas sample was cooled. (We altered the question slightly from its original form so the number of particles in the “before” and the four “after” pictures were the same.) The first set of data collected involved a group of students (Group I, N ⫽ 99) enrolled in a firstsemester general chemistry course who answered these questions as part of their final examination. Analysis of the responses showed that there were two options (C and D) that were chosen by fewer than 15 students. Since we were concerned that this small number of responses may not adequately reflect all of the possible reasons that students choose these
Vol. 84 No. 5 May 2007
•
www.JCE.DivCHED.org
Research: Science and Education
options, we decided to collect additional data. Consequently, a second group of students from the same university (Group II, N ⫽ 231) enrolled in a second-semester general chemistry course were asked to answer these questions at the beginning of a three-hour laboratory session, and the responses from both sets of students were combined. Since both groups of students studied gas laws in the same first-semester general chemistry course at the same university (one year apart), we felt justified in combining these two groups together. Data Analysis The responses from both groups of students were sorted by their choice of distractor and then the responses were analyzed using the content-analysis technique (23, 24). The first author read through each student’s responses, created categories of all the students’ responses, and tallied the number (frequency) of students’ responses that corresponded to each category. To test for reliability, the second author used the categories identified by the first author and analyzed the students’ responses for Group I. The agreement between the two authors’ classifications was very good; the inter-rater reliability for these students’ responses was 0.977. Since the analysis of the responses from the students in Group II did not provide any new categories, we were confident that the combined sample size was adequate and that there was no need to collect additional data from a third group of students. Reliability of the Multiple-Choice Question Several researchers (1, 2, 10) have reported the distribution of their students’ responses to this same multiple-choice question; these values appear in Table 1. The percentages of students choosing each response are very similar (especially in the last three studies), even though they were measured at four different universities over a period of two decades. These results demonstrate that this question is very reliable (a measure of the consistency or stability of the question over time). In general, just over 30% (30–36%) of the students in each group chose the correct answer, (A). The most commonly selected incorrect answer was option (B), chosen by 28–48% of the students. Option (B) was also the most frequent answer for all of the groups, except for three of the five groups of students listed in Nurrenbern and Pickering’s study. The incorrect answer in option (C) was chosen by 12–25% of the students, followed by option (D) as the least commonly selected answer (chosen by 6–8% of the students).
Analysis of Student Explanations Table 2 shows the results of the content analysis performed on the students’ explanations of molecular-level gas properties in the multiple-choice question for both groups of students. We decided to omit the responses that were expressed by fewer than 3% of the population choosing each option, under the assumption that these responses were not representative of the group as a whole and probably represented misstatements or guesses on the part of the student. We also caution readers to recognize that this summary represents a content analysis of the students’ written comments. It would be wrong to infer that simply because a student did not explicitly mention these parameters that they had no opinion on them. There were four major categories identified by the content analysis regarding particulate-level gas properties: Changes in velocity, changes in volume, changes in pressure, and changes in state of matter.
Changes in Velocity None of the students mentioned that the particles’ velocities would stay the same, and only 2 students (7%) choosing option (D) said that the particles would move faster upon cooling. These students believed that when the gas sample was cooled, the particles would move faster and that would increase the gas volume. Most of the students who mentioned a change in the velocity of the gas particles reported that they would slow down. This was reported by 85 students (77%) who chose (A), 70 students (50%) who chose (B), 24 students (46%) who chose (C), and 5 students (19%) who chose (D). A χ2 test of goodness-of-fit (25) showed a significant difference in the proportion of students choosing each response who reported a decrease in the particles’ velocities (χ2(3) ⫽ 17.65, p < .001). In particular, more students choosing the correct answer (A) and fewer students choosing incorrect option (D) mentioned this change in velocity than would have been expected by chance. The large proportion of students choosing (A) who reported a velocity change is consistent with the idea that students choosing the correct option would be more likely to provide a correct description. It may also be because they recognized that the gas volume would not change (discussed below), yet knew something would change as a result of the temperature change. Fewer students choosing (D) mentioned a decrease in velocity: this is consistent with the fact that some of them thought the particles would move faster, not slower.
Table 1. Distribution of Student Responses to a Multiple-Choice, Gas Properties Question Data Source
Student Response Choices (A Is the Only Correct Response)
(Number of Responses)
A
(%)
Nurrenbern and Pickering (1), (N ⫽ 198)a
72
(36)
Sawrey (2), (N ⫽ 285)a
89
Sanger et al. (10), (N ⫽ 210) This study (N ⫽ 330) aThe
B
(%)
C
(%)
D
(%)
56
(28)
50
(25)
11
(6)
(31)
136
(48)
34
(12)
23
(8)
62
(30)
91
(43)
40
(19)
17
(8)
110
(33)
141
(43)
52
(16)
27
(8)
sum of all four choices and the total number of students are not the same; presumably, this is because some students left this question blank.
www.JCE.DivCHED.org
•
Vol. 84 No. 5 May 2007
•
Journal of Chemical Education
871
Research: Science and Education
Changes in Volume Only students choosing (A) stated that the volume of the container or the distribution of particles within it would remain the same after cooling, reported by 70 of these 110 students (64%). One would expect that those picking (A) recognized that the distribution of particles remains constant, and it could be argued that some of them assumed that this “goes without saying” so they didn’t say it. However, 18 of these students (16%) implied in their descriptions that the volume of the gas would decrease, even though they chose (A), which shows no volume change. Interestingly, 6 students (4%) choosing (B) included comments suggesting that the particle distribution would not change upon cooling, yet still chose (B), which clearly shows a volume change. They would move slower due to decrease in temp. and pressure. They would not be hitting the walls as much as they had originally.
We would expect students who chose (B) and (C) to say that the particles moved closer together or that the gas volume decreased upon cooling, and most of them did: 126 of the 141 students (89%) choosing (B) and 37 of the 52 students (71%) choosing (D) reported these changes. However, we were surprised that 18 of the 110 students (16%) choosing (A) stated that the volume would decrease. Most of these students (11 of the 18) reported that while the volume would
decrease, it would not be as dramatic as the volume decrease shown in (B) because the temperature change was so small. Most of students choosing (B) stated that the volume of a gas decreases when the gas is cooled and that the particles would move closer. Analysis of their responses showed that some of them confused the rigid steel tank with a container whose volume could change (like a balloon), while others seemed to be quoting gas laws and confused Gay-Lussac’s law (P–T) with Charles’s Law (V–T). In either case, this drawing clearly shows that the volume of the tank did not change and consequently this drawing violates the kinetic-molecular theory, which states that gas particles will travel in random straight-line motions and will occupy the entire volume of any container. The responses of the students choosing (C) were not very different from those of the students choosing (B). Some students expressed the belief that (B) or (C) were equivalent and that either one could be correct depending on how the cross-section of the tank was cut, as evidenced by this response from a student who chose (B). As the temperature cools the energy of random movement [decreases] and the hydrogen will attract towards each other. However if the steel tank is laying on its side then the answer would be C.
The only students who reported that the volume of the tank would increase were those choosing (D), and this idea
Table 2. Distribution of Student Responsesa to a Multiple-Choice, Gas Properties Question, Including Students’ Molecular-Level Explanations of Their Answers Concept Category and Student Explanationsb
Number Correct: A (%) (N ⫽ 110)
Number Incorrect: B (%) Number Incorrect: C (%) Number Incorrect: D (%) (N ⫽ 141) (N ⫽ 52) (N ⫽ 27)
Changes in Velocity Particle velocity (speed) decreases; Particles slow down; Particles have less kinetic energy
85 (77)
070 (50)
24 (46)
05 (19)
Particle velocity (speed) increases; Particles speed up; Particles have more kinetic energy
———
———
———
02 0(7)
Volume decreases; Particles move closer together
18 (16)
126 (89)
37 (71)
02 0(7)
Volume/particle distribution does not change
70 (64)
006 0(4)
———
———
Volume increases; Particles move farther apart
———
———
———
22 (81)
Pressure decreases; Particles hit each other, walls less often
22 (20)
016 (11)
06 (12)
06 (22)
Pressure increases; Particles hit each other, walls more often
———
———
———
05 (19)
State of matter does not change; Sample remains a gas
13 (12)
———
———
———
Sample turns into a solid (freezes) or a liquid
———
015 (11)
17 (33)
———
Changes in Volume
Changes in Pressure
Changes in State of Matter
aDashes
872
indicate that the idea was supported by less than 3% of students choosing this distractor. bThe correct concept is italicized.
Journal of Chemical Education
•
Vol. 84 No. 5 May 2007
•
www.JCE.DivCHED.org
Research: Science and Education
was expressed by 22 of them (81%). It was interesting that this response came from students regardless of whether they said the particles were moving slower or faster and regardless of whether they said the pressure would increase or decrease. It also clearly shows that students are interpreting (D) as showing that particles are trying to expand the volume of the container, and not that they have condensed and “frosted out” on the sides of the container as one of the authors had originally assumed.
Changes in Pressure None of the students reported that the pressure of the gas sample would remain the same, and in general fewer students made explicit comments about changes in pressure or the number of particle collisions as a result of cooling compared to changes in velocity and volume. For each option, 10–20% of students reported that the pressure or number of collisions decreased as a result of cooling. Only students who chose (D) reported that the pressure would increase (5 of the 27 students, or 19%), which is consistent with the idea that the gas particles are pushing on the walls of the container in order to increase the container volume. Students who chose (D) were equally split regarding whether the pressure would decrease (22%) or increase (19%), which may imply that they do not have a strong sense regarding how the pressure would change upon cooling. Changes in State of Matter Few students made comments regarding phase changes, although during the data collection some students expressed uncertainty regarding the temperature at which hydrogen would liquefy or solidify. Students choosing (A) were the only ones to report that ᎑20 ⬚C was not cold enough to liquefy or solidify hydrogen gas, and this was expressed by 13 students (12%). On the other hand, 15 of the 141 students (11%) choosing (B) and 17 of the 52 students (33%) choosing (C) reported that the gas sample would liquefy or solidify (freeze). While we had expected most of the students choosing (C) to suggest that a state of matter change occurred, we were surprised to see that students choosing (B) also thought this image depicted a state of matter change. Once again, students expressed the belief that (B) and (C) are both correct representations of a phase change depending on the perspective of the tank’s cross-section (perpendicular or parallel to the ground). Validity of the Multiple-Choice Question The second and third research questions address the issue of the multiple-choice question’s validity (the extent to which the question measures what it purports to measure, in this case students’ conceptual understanding of gas laws and particle behavior). The second question asks whether students choosing the correct option provided scientifically accepted answers that do not show misconceptions. Analysis of the responses from the 110 students who chose the correct answer (A) shows that 87 of these students (79%) provided explanations that showed none of the misconceptions listed in Table 2. Of the 23 students who did exhibit misconceptions, the most common misconception was that the volume of the gas would decrease (reported by 18 students). The other four students demonstrated misconceptions related to the gas pressure staying the same or increasing, and the gas sample condenswww.JCE.DivCHED.org
•
ing into a liquid. These results support the hypothesis that most students (about 80%) choosing the correct answer have a conceptual understanding of the question that is free of misconceptions. The third research question asks whether students choosing an incorrect option provided answers indicating that they do have misconceptions about the particles’ behavior. Analysis of these students’ responses showed that 127 of the 141 students (90%) choosing (B), 48 of the 52 students (92%) choosing (C), and 24 of the 27 students (89%) choosing (D) provided answers containing at least one misconception. These results support the hypothesis that most students (about 90%) choosing an incorrect answer would demonstrate at least one misconception regarding the behavior of gas particles. Although both of the results above tend to confirm the validity of the multiple-choice question, there are other issues that may threaten the validity of this question. First, several students seem to view option (B) and (C) as similar or equivalent choices, which differ in their “correctness” depending on how the cross-section of the steel tank was made (perpendicular or parallel to the ground). And although the number of students picking (B) and (C) are very different, the proportion of student comments in each of the categories listed in Table 2 are similar for the two groups. Second, several students answered the question assuming that the sample of hydrogen gas would either liquefy or solidify at 3 atm and ᎑20 ⬚C. While it is certainly true that hydrogen would remain a gas under these conditions, is it a misconception for students to think the gas would change states of matter, or simply a lack of factual knowledge regarding the pressure and temperature at which these changes would occur? In other words, are these students demonstrating an incorrect conception about the behavior of gas particles or are they demonstrating a correct conception about the properties of solids, liquids, and gases (if a gas sample is cooled enough, it will liquefy and then solidify) but simply unaware of the conditions under which these changes will occur? One way to solve this problem would be to give the students the boiling point of hydrogen at 3 atm instead of making them guess what it would be. Nahkleh and Mitchell (4, 5) used an altered version of this question in their work in which they included the boiling point of hydrogen. Unfortunately, they did not report how these alterations changed the distribution of students choosing each option (if at all). Third, many students seemed to have trouble with this question because the critical attribute that changed as a result of cooling the gas sample (particle motion) was not shown in this question. Thus, some students were reluctant to choose the correct answer (A) because this picture looks no different from the starting picture, but they were sure that decreasing the temperature would have some effect on the particles. I picked B because it shows that the molecules are not packed onto the walls of the container which would show high pressure. C and D both have issues with pressure at that temp and A too closely resembles the original drawing at 20 ⬚C and it wouldn’t be the same at ᎑20 ⬚C.
Even students who chose the correct answer commented that the picture was not showing what was really changing as a result of the cooling.
Vol. 84 No. 5 May 2007
•
Journal of Chemical Education
873
Research: Science and Education The hydrogen will move slower but still be evenly distributed throughout the tank. There is no way to determine the speed of the molecules from this diagram.
Conclusions A sample of 330 students was asked to answer a multiple-choice, particulate-level question on gas properties (Figure 1) first used by Nurrenbern and Pickering (1). These students were also asked to explain what they believed was happening to these particles at the microscopic level when the gas sample was cooled. An analysis of these comments demonstrated that most students believed that the temperature change would lead to changes in four parameters: Particle velocity, volume (particle distribution), pressure (number of particle collisions), and state of matter. Most students choosing the correct answer (A) recognized that the particles would move slower but still occupy the same volume. However, some who chose (A) implied that the gas volume would decrease upon cooling, even though image (A) shows no volume change. While half of the students choosing (B) and (C) said that the particles would move slower, most of these students believed that the volume of the gas sample would change even if the volume of the steel tank remained the same. Some choosing (B) and (C) also provided explanations for their choice based on the idea that the temperature change was sufficient to make the gas sample become a liquid or solid. Most choosing (D) believed that the gas volume would increase upon cooling, but were evenly split in their beliefs that the pressure of the gas sample would either increase or decrease. While this question is very reliable, as evidenced by the similar distribution of responses from four different universities over a span of twenty years (Table 1), there is some question regarding the question’s validity (i.e., does it really measure students’ conceptual understanding of gas particle behaviors?). As evidence of the question’s validity, we found that 80% of students who picked the correct answer (A) provided an explanation that did not show any misconceptions regarding gas particle behavior, while 90% of students choosing an incorrect option provided explanations that showed at least one misconception regarding gas particle behaviors. While this information supports the validity of the question, we also identified three possible threats to the question’s validity. First, some students do not see a distinction between options (B) and (C), and have argued that either image could be correct depending on how the cross-section of the tank was cut. Second, students who stated that the gas sample would liquefy or solidify under these conditions may not be exhibiting a misconception. These students expressed the correct conception that when a gas sample is cooled enough, it will turn into a liquid or solid; they simply underestimated the temperature change needed to cause these changes. Finally, this question is troublesome because it does not depict the critical attribute of change (particle motion). As a result, some students viewed this as a trick question because the correct answer looks just like the initial picture even though the gas sample was cooled. Although we identified four major areas that some of the students considered when explaining this question at the microscopic level, many students didn’t mention one or more of these ideas in their explanations. Since the conclusions from this study are based only on the responses of students who 874
Journal of Chemical Education
•
explicitly mentioned each idea, these conclusions should be viewed cautiously. In order to get a better estimate of all of the students’ beliefs regarding each of these parameters, data should be collected in which students are asked to answer the multiple-choice question along with questions that specifically ask them about changes in velocity, volume, pressure, and state of matter. In addition, the threats to validity identified in this study suggest that another way to improve this question would be to animate it so that the critical attribute of particle motion can be seen and so that students will see that the initial picture and the correct answer do look different (10). Perhaps the animation could also make options (B) and (C) look different so that students will not view them as equivalent. Note 1. The distractors in this question were changed so that there is the same number of particles in the initial drawing and each choice.
Literature Cited 1. Nurrenbern, S. C.; Pickering, M. J. Chem. Educ. 1987, 64, 508–510. 2. Sawrey, B. A. J. Chem. Educ. 1990, 67, 253–254. 3. Pickering, M. J. Chem. Educ. 1990, 67, 254–255. 4. Nakhleh, M. B. J. Chem. Educ. 1993, 70, 52–55. 5. Nakhleh, M. B.; Mitchell, R. C. J. Chem. Educ. 1993, 70, 190– 192. 6. Zoller, U.; Lubezky, A.; Nakhleh, M. B.; Tessier, B.; Dori, Y. J. J. Chem. Educ. 1995, 72, 987–989. 7. Phelps, A. J. What They Don’t Know and Why: Improving the Teaching of Chemistry through Misconceptions. Presented at the 225th National Meeting of the American Chemical Society, Mar 2003, CHED #1282. 8. Deming, J. C.; Ehlert, B. E.; Cracolice, M. S. Algorithmic and Conceptual Understanding Differences in General Chemistry: A Link to Reasoning Ability. Presented at the 226th National Meeting of the American Chemical Society, Sep 2003, CHED #299. 9. Sanger, M. J. J. Chem. Educ. 2005, 82, 131–134. 10. Sanger, M. J.; Campbell, E.; Felker, J.; Spencer, C. J. Chem. Educ. 2007, 84, 875–879. 11. Gabel, D. L.; Samuel, K. V.; Hunn, D. J. Chem. Educ. 1987, 64, 695–697. 12. Smith, K. J.; Metz, P. A. J. Chem. Educ. 1996, 73, 233–235. 13. Lee, K.-W. L. J. Chem. Educ. 1999, 76, 1008–1012. 14. Sanger, M. J. J. Chem. Educ. 2000, 77, 762–766. 15. Raviolo, A. J. Chem. Educ. 2001, 78, 629–631. 16. Mulford, D. R.; Robinson, W. R. J. Chem. Educ. 2002, 79, 739–744. 17. Pınarbası, T.; Canpolat, N. J. Chem. Educ. 2003, 80, 1328–1332. 18. Johnstone, A. H. J. Comp. Assist. Learn. 1991, 7, 75–83. 19. Gabel, D. L. J. Chem. Educ. 1993, 70, 193–194. 20. Williamson, V. M.; Abraham, M. R. J. Res. Sci. Teach. 1995, 32, 521–534. 21. Sanger, M. J.; Greenbowe, T. J. J. Chem. Educ. 1997, 74, 819–823. 22. Sanger, M. J.; Badger, II, S. M. J. Chem. Educ. 2001, 78, 1412–1416. 23. Borg, W. R.; Gall, M. D. Educational Research, 4th ed.; Longman: New York, 1983; pp 511–522. 24. Domin, D. S. J. Chem. Educ. 1999, 76, 109–111. 25. Hinkle, D. E.; Wiersma, W.; Jurs, S. G. Applied Statistics for the Behavioral Sciences, 3rd ed.; Houghton Mifflin: Boston, MA, 1994; pp 539–542.
Vol. 84 No. 5 May 2007
•
www.JCE.DivCHED.org
