Symposium: What Is Research in Chemistry Education?
Think-Aloud Methods in Chemistry Education Understanding Student Thinking
-
Craia W. Bowen Center for lnsrr~ct~onal Development and Research. Parrmgton rlall 109. DC-07, Tne Jn~versityol Wasn ngron. Sean e. WA 98195
Imagine you are teaching a firsbterm chemistry class. The class has discussed Lewis structures, and you have moved into intermolecular forces before leading into properties of gases. In lab the students have carried out a decomposition to demonstrate stoichiometric relations and recently have done a lab in which they collected a gaseous product over water. After several days working with the class discussing the hehavior of gases, you ask them to work in small groups on the following problem: Which is denser a t the same temperature and pressure, dry air or air saturated with water vapor? Explain (1). As you walk around the room and listen to the small groups working on this problem, you hear several approaches to working the problem. The first p u p you see working on the task draws pictures a s a way to represent the problem. At first they recall that air is mainlv a mixture of oxveen (20%) and nitroeen " !80PiI so they draw two containers ol'air (shown in Fig. 1 Next. the students remember that Avoeadro's law will hclo them figure out that if the containers are the same size (at the same temperature and pressure), then they have to have the same number of particles. So they modify one of their original drawings by reducing the number of oxygen and nitrogen molecules and replacing them with water (see Fig. 2). From this modified drawing they realize that, for a given volume, the overall mass of the moist air is less than the dry air (based on the total mass of molecules in their drawing). So this group of students concludes the moist air is less dense. As you continue to walk around the room, another group figures out a way they could go into a lab and answer the question. (One member of the group recalls an experiment she had done in another class where they determined the identity of various gases based on their densities.) The students continue to discuss how they could get a sample of dry air by passing some air through a drying agent. They also figure they could get a moist sample by putting water
-
Container $1
Container #2: Moist Air
Figure 2.Modified drawing by small group #I into a container holding a sample of air. The students then reason that it would be easy to answer the question by just drawing samples of a given volume for each gas and weighing them. The sample with the greater mass would be the more dense form of air. A third group of students you watch takes t h e approach you expected all the students would take because i t was t h e approach you used. They begin by listing equations and information they know about gases (see Fig. 3). The students in the small group realize that they need to figure out the "molecular weight" of dry and moist air, so they work a particular case of a certain mole fraction (shown in Fig. 4). These students conclude t h a t if there is any water vapor in the air, then the density of that air will be less than dry air because of the reduction of the "molecular weight" and application of Avogadro's law. What sorts of things did you learn about your students by using this small-group activity and listening to their discussions? Perhaps you learned that students can represent the same problem in several different ways. Second, it was noticed that students draw on their different experiences for solving problems (e.g., thinking about what they had done in lab). Finally, and probably most importantly,
Container #2
Figure 1 . Initial drawing by small group #I.
184
I
Container U1: Dry Air
Journal of Chemical Education
Figure 3. Written work of small group #3
L
Figure 4. Wrinen work of small group #3 you realized that listening to the students'explanations of what they were doing can be valuable in understanding how they think about chemistry and solve problems. This third outcome-learning from listening to your studentsis the focus of this paper. The purpose of this article is to describe research techniques in chemistry education that are based on listening to learners. This article gives a brief history of think-aloud techniques, and works through an example study to show the kinds of considerations that need to be made when doing this sort of research. Historical Overview Before behaviorism, verbal reports were used as data for psychological research because they provided information about "what was going on in the mind." However, when behaviorism came to power, and there was a shift to observing behavior rather than studying the mind, verbal reports fell into disfavor. Behaviorists claimed that verbal reuorts were unscientific.,irreoroducible. unverifiable. and uifalsifiable. The new paradigm suggested verbal reports were not scientific because hvuotheses could not be tested. This claim was no longer true with the rise of Piagetian research methods in which hvwtheses could be tested dur-. ing clinical interviews based on subjects' responses. Schoenfeld (21, a mathematics education researcher, suggests that the use of verbal reports as data received most of its new leeitimization from information-urocessine -.ow" chologists and artificial intelligence (A11 researchers. Information-processingresearchers try to explain human behavior using symbolic and computer models, while A1 researchers attempt to make computers do intelligent things (either mimicking human thought, or using entirely different approaches). The beginnings of these fields were based on analysis of human problem-solving protocols. For example, researchers reported on artificial intelligence research in such areas as chess, cryptarithmetic, etc. (3).Because the use of think-aloud methods led to success in these fields, the collecting and analysis of human protocols has been deemed useful by psychologists and educational researchers. With the resurgence of verbal methods, Schoenfeld raises issues surrounding their use:
. ".
(1) Are such methods reliable? Do speak-aloud transcripts really reflect the processes used by the individuals as they solve problems, or the processes they would use if they had not been asked to work out loud? (2) Are protocol analysis schemes useful for research in mathematics education? That is, do such schemes, generally adapted from those used successfully in artificial intelligence, provide relevant and useful information about human problem solving? (p 171). These questions were examined through study of variables that are suggested to affect verbal reports including: the numhrr of people bem: mumewed, 121 the dcgrce ofrnterventmn in t h e i n t m w w , 13) the nature of the envlrnnmrnr a n d now mrnfurtdble the ( 11
subjects feel in it; and, (4)the task variables.
The number of people being interviewed affects cognitive behavior because of consequent social interactions. Acommon assumution is that single-resoondent orotocols urovide the cognitions because they are thought tdoperate without soeial concerns. Schoenfeld points out that the effects of social dynamics becomes more apparent as more subjects are studied together. However, multi-respondent interviewing has advantages in that researchers can examine decision-making behaviors with less inkrvention because other subjects are likely to ask questions researchers want answered. The degree of intervention also can affect prohlem-solving behaviors. Asking a simple question such as, "Why did you do X?" can lead the subject to exhibiting artificial metacognitive behaviors. These behaviors can have enormous consequences on problem-solving behaviors. For example, subjects might start asking and answering these, and other metacognitive, questions to themselves. The nature and deerees of freedom in instruction and intervention during inierviews can influence the quality of verbal r e ~ o r t sof data. The manner in which subiects report thei'thought affects those thought processes.~eflecting on their thought after solving a task is different from describingthem as they solve the problem. Anexpost facto commentary on the solution to a problem makes the problem solver appear to know what was going on at every step. In contrast, while verbalizing thoughts as they solve problems, subjects appear disjointed and disoriented about how to proceed-probably a better indication of how they are processing information. Only after they solve the problem do their thoughts become more organized. The nature of the environment and how comfortable subjects feel in it also affects problem-solvina behaviors. Subjects may exhibit atypicalbehaviors if t&y are in uncomfortable situations. Schoenfeld notes that some of his own subjects demonstrated mathematical behaviors just because they thought they were supposed to, and these hehaviors had nothing to do with solving the problem! Another source of error in gathering high-quality verbal data is that the subject might try to guess the researchers' intentions and give them what they want (or the opposite). Schoenfeld suggests another factor related to collecting useful verbal data is task variables. Although he does not elaborate what these variables are, they;nclude those things that the subjects can use to solve the problems (and the problems themselves). When difficult tasks are repeatedly given to a subject, this might affect concentration and enthusiasm for doing the work. Tools such as paper and pencil, calculators, models, and texts also are task variable that can influence prohlem-solving behaviors. For example, students may estimate values if they are not given access to a calculator. A Sample Study This section illustrates aspects of designing, implementing, and reporting a study in which a think-aloud technique is utilized. Formulating the Research Question Think back to the opening context of this paper in which groups used different approaches to answer the question, "Which is more dense, moist or dry air?" Imagining students in those groups work on that problem may have raised several questions in your mind:
.How are laboratory experiences related to ways students represent gas-law problems? How do students think about Avogadrds law? How does the context (e.g.,elassraom or laboratory)in which questions are presented influence how students represent , and salve problems? Volume 71
Number3 March 1994
185
.What processes take place in small groups that lead to certain ways of thinking.about solving a problem? How are prior student experiences in chemistry lecture and labs (or their cognitive developmental level) related to the ways students represent and solve gas problems? These auestions represent a good starting point for designing'a research siudy. This also demonstr%s a n important source of research ideas-the classroom. I t is analbgous to a chemist noticing something taking place during a reaction in a lab, and wondering what is occurring. As with anv studv. asking a useful research question in chemistry ejucation is
