In the Classroom
Demonstration-Based Cooperative Testing in General Chemistry: A Broader Assessment-of-Learning Technique Craig W. Bowen Department of Chemistry and Biochemistry, University of Southern Mississippi, Box 5043, Hattiesburg, MS 39406-5043 Amy J. Phelps Department of Chemistry, University of Northern Iowa, Cedar Falls, IA 50614 During the past several years there have been numerous calls for changing the curriculum in general chemistry (1–6). Although it is difficult to make changes in an ill-defined practice like curriculum, reformers for the most part have focused on changing content (e.g., including molecular modeling or statistical analysis) and/or instruction (e.g., incorporating computer-based tutorials or using cooperative groups). However, more often than not these changes in content or instruction are not carried over into the realm of assessment. In fact, students learning the new content areas through new instructional techniques are tested in the same old ways (typically paper-and-pencil multiple choice or short answer tests, quizzes, and homework). These current reform efforts need to focus on not only content and instruction but also assessment. This paper addresses three areas related to assessment of learning. First, it provides a consideration of possible relations among instruction, content and assessment. In the second section a rationale is provided and examples are given of demonstration-based testing activities. Finally, student performance and reaction to these assessment methods are given. Curriculum: Relations among Assessment, Content, and Instruction Figure 1 shows a linear relationship among content, instruction, and assessment as they are normally practiced in many college chemistry classes. The approach is to decide what to teach, teach it, and see what students learned. However, recent literature reviews indicate that assessment of student learning can play a larger role in curriculum (7). Figure 2 shows an interactive relationship among content, instruction, and assessment. For example, diagnostic assessment can be used by an instructor to determine what content needs to be taught before moving on with the rest of the course material. Or oral questioning of a sample of
Figure 1. A linear model of curriculum components.
Figure 2. An interactive model of curriculum components.
students in a class can be used to enhance instructional practices by providing an instructor feedback on how well new instructional techniques (e.g., small group activities during class) were working. Figure 2 emphasizes that assessment of learning can inform content selection and instructional practices. A Rationale for Broadening Assessment Practices In summary, rethinking the chemistry curriculum involves thinking how components such as content, instruction and assessment are interrelated. One strategy for refining curriculum is to change assessment practices. This strategy is based on the assumption that by changing assessment practices there will be changes in instruction and content. But upon what areas are these new assessment strategies to be based? Below we provide a brief review of two areas that are pertinent to these changes: research on problem solving in chemistry; and research on cooperative learning. Through this overview we provide a rationale for demonstration-based cooperative testing techniques in general chemistry.
Research on Problem Solving in Chemistry The most up-to-date review on research on problem solving in chemistry, by Gabel and Bunce, covers several hundred research articles (8). It focuses on several main areas of problem-solving research, including the nature of the problem to be solved, understanding of concepts needed for solving the problem, characteristics of the learner, the nature of the learning environment, and the effects of various instructional strategies on problem-solving performance. One part of Gabel and Bunce’s review pertinent to the work reported here was the idea of different levels of representing chemical phenomena (e.g., symbolically through equations, at the particulate level through diagrams, or at the macroscopic level through laboratories and demonstrations). Research in this area ranges from observing general chemistry students solving stoichiometry problems to work focusing on graduate students solving problems in organic synthesis (9). This work reiterates the importance of successful problem solvers’ being able to switch between ways of representing a chemical problem across several domains covering the symbolic, particulate, and macroscopic levels. Research on how people solve chemical problems shows that several factors are involved. One factor that can be particularly important is that the problem solver be able to switch between various ways of representing chemical phenomena. This area of research supports the use of demonstrations (and laboratory experiences) as an instructional tool for assisting student learning. This approach to instruction can help students to make links between symbolic representations usually presented in lecture formats and macroscopic chemical phenomena that chemists experience in
Vol. 74 No. 6 June 1997 • Journal of Chemical Education
715
In the Classroom Table 1. Settings in Which Demonstration-Based Assessment Has Been Used Institution
Course
Florida State University
1st-semester science and engineering majors course
Purdue University
1st-semester nonmajors course typically taken by agriculturerelated majors
60
Course supervisor and 2 TAs
University of Louisville
1st- and 2nd-semester science and engineering majors course First semester nonmajors course
150
2 TAs
150
None
University of Northern Iowa
1st- and 2nd-semester science majors courses
75
None
University of Southern Mississippi
1st-semester science and engineering majors course
85
None
day-to-day work. Using demonstrations as an assessment tool has at least two consequences. First, it can orient students’ attention toward learning from demonstrations because they know it will be assessed. It also provides a measure of the problem-solving capabilities of the students because it requires them to be able to switch forms of representing problems dealing with chemical phenomena.
Research on Cooperative Learning Johnson, Johnson, and Smith provide a brief review of research on cooperative learning in college classrooms (10). Although there has been interest in college-level chemistry classes (as evidenced by a recent symposium organized by William R. Robinson at the Fall 1994 ACS national meeting on cooperative learning), the work of Johnson et al. provides a greater overview of how cooperative learning may help chemistry learning at the college level. Their review focused on two broad areas: factors involved in cooperative learning, and relations of cooperative learning to learning outcomes. Factors involved in cooperative learning include giving and receiving assistance, exchanging information and approaches to thinking, providing peer feedback, and engaging in challenge and controversy. Johnson et al. reported that these factors are related to learning outcomes. For example, experimental studies across numerous college disciplines showed that achievement was higher in situations that utilized cooperative learning. In addition, the authors reported that studies showed that cooperative learning strategies helped students to develop more positive attitudes toward the subject area. According to Bowen, as of the fall of 1994 only one databased, theory-guided study had been done to examine the effects cooperative learning has on college-level chemistry students (11). In 1991 Basili and Sanford conducted a study to assess the effects of using small groups to elicit misconceptions prior to instruction (12). A total of 62 community college students enrolled in a preparatory course for general chemistry participated in the study. Two groups of two classes each were formed. Students in the experimental group were instructed in how to make concept maps, assigned to heterogeneous groups after every five days of lecture, asked to explain in their group their responses to study questions given out the previous week, told to talk about their concept maps and develop one to turn in, asked to grade others in the group in terms of helping behaviors, and received bonus points if all groups members scored above 70 on the class exams. In contrast, the control-group students were shown a demonstration after every five days of lecture, wrote observations about the demonstration for credit, and could turn in a practice test for bonus points.
716
Class Size 220
Support Staff 3 TAs
Chi-square analysis of the number of pretreatment misconceptions showed no detectable differences between the treatment and control groups. However, after instruction there was a significant difference in the number of misconceptions held by the treatment and control students (with the treatment students having fewer misconceptions). In addition, percentages of students with correct conceptions was higher for the treatment across all concepts. In summary, research on cooperative learning in chemistry and other areas shows that such instruction can assist student learning. This provides a foundation for including group-based experiences in our instruction and testing for chemistry learning. With work on problem solving in chemistry showing that representing chemical phenomena in different ways is important (symbolically, macroscopically, and at the particulate level) and that cooperative learning can enhance achievement, we believe that these two areas support the strategy of using demonstration-based cooperative testing techniques. Through the group activity we hope to promote ongoing cooperative learning outside the classroom that assists students to achieve to a higher level by asking them to relate macroscopic phenomena to symbolic representations discussed in class. Below we give an overview of the kinds of classes we have worked with, and some of the assessment activities and questions we have used.
Description of the Classes We have used a number of demonstration-based assessment techniques with a variety of classes. Table 1 summarizes some of these courses. It can be seen that the techniques described below have been used in a variety of classes in terms of student population, size and course support. Description of the Approaches to Demonstration-Based Group Testing These demonstration-based activities are used in a number of ways, including:
Journal of Chemical Education • Vol. 74 No. 6 June 1997
•
As a separate quiz in which students see a demonstration and work together in small groups before handing in their own sheet answering questions about the demonstration after 30 minutes.
•
As part of an exam in which the demonstration is shown at the beginning of the test and students answer questions about the demonstration that are recorded along with the other test items.
•
As a set of questions on an exam that is done 48 hours before the written exam. Students see the demonstration and can work in groups outside of class before turning in the questions as part of the exam.
In the Classroom One way of using this approach is described below, in terms of what happens in a typical 50-minute period. The demonstration exam questions are administered during lecture 48 hours before each written exam. While students may work together on the questions in or out of class, each student is responsible for turning in his or her own work. The format on demonstration day takes place as follows.
Example 3: ∆V of Mixing. One hundred milliliters of ethanol is mixed with 100 mL of water. Students observe that the volume is less than 200 mL. They are asked the question shown in Figure 3.
1. Announcements of the day (2 min) 2. Presentation of new material (15–20 min) 3. Distribution of question (2 min) 4. Conducting demonstration/recording data (5–8 min) 5. Students working in self-selected groups to answer questions (15–20 min)
Example Demonstrations and Questions We do not want to belabor the point of using good demonstrations. There are numerous sources for demonstration ideas ranging from this Journal to other sources. What we would like to do in this section is to emphasize the kinds of demonstrations that can be used as assessment activities. The kinds of demonstration activities used are based primarily on the content being covered and the two kinds of questions typically asked. One question type asks students to explain a chemical or physical phenomenon, and the other asks them to identify something based on chemical or physical properties. Examples of demonstrations and questions of both types are provided below. Example 1: Crushing Can. The demonstration involves boiling a small amount of water in a pop can until steam is emitted from the can. The can is inverted quickly up-sidedown into a container of ice water. The students observe that the can is crushed. Sample questions to ask are given below. 1. Explain what happens in this demonstration with regard to the gas molecules involved. Use drawings to help clarify your explanation if you want to. Start with the cold can sitting on the bench top and take it through the crushing. 2. Would it make a difference in the demonstration if the can were not inverted in the ice water? Explain your reasoning. Bonus: Imagine we do the same demonstration, only inverting the can into liquid nitrogen instead of ice water. (Temperature of liquid nitrogen is 77 K). Explain what you would observe here.
Example 2: Conductivity of Substances. The demonstration involves examining the electrical conductivity of various substances. The first set of questions are based on using an electrical conductivity apparatus to test the conductivity of two white solid substances (salt and sugar). Then the substances are each dissolved in water and the resulting solution is tested for conductivity. Students are then asked: In the front of the room are two compounds, compound A and compound B, both of which are solids at room temperature. Observe how each responds to the conductivity tester; first in their pure forms and then when water is added. Answer the following questions based on your observations. a. What kind of atoms do you think make up the two compounds A & B? b. How would you explain the difference in their behavior when added to water?
Figure 3. Demonstration-based testing item dealing with ∆V of mixing.
There are several points to note about these types of demonstration-based questions. First, students should be asked to record observations as part of their work. This helps them focus attention on what they observed rather than on “what’s supposed to happen”. The observations also allow the person grading the work to see if students made reasonable conclusions from the available data. A second point seen in these examples is that some of the questions ask students to link the observed phenomenon to a particulate-level representation in which they describe what the particles do during the phenomenon. Finally, students are asked questions that extend the idea of the demonstration. For instance, in example 1 students are asked to extend the idea of pressure of steam in the can to what would happen if the same can were inverted into liquid nitrogen. The second major type of question asked through demonstration-based assessment has to do with identifying or quantifying various aspects of substances. These demonstrations can be fairly simple (e.g., based on density) or more complex (e.g., based on a titration). Examples of these types of questions and demonstrations are given below. Example 4: Identification of Unknown Metals/Metalloids. Students are shown two pieces of material labeled A and B (Si and Sn) and given the masses of each. Next, the pieces are added to different graduated cylinders containing water. The change in water volume is recorded. The students are asked to do the following: This portion of the exam is based on a demonstration that will be performed in class. The purpose of this portion of the exam is to give you an opportunity to apply some of the concepts we have been studying to solve a particular chemical problem. You may use your notes, book, and talk with other students in the class. However, on Friday, March 10, you will turn this sheet in with the rest of your exam. Useful data are contained below in the table. Element Density (g/cm 3) C 2.25 Si 2.33 Ge 5.32 Sn 7.31 Pb 11.35
Vol. 74 No. 6 June 1997 • Journal of Chemical Education
717
In the Classroom 1. On the back of this sheet plot the data contained in the table above. What periodic trend does there appear to be for density? 2. How would you explain the trend you summarized in the question above? (Hint—consider the relative size and mass of atoms of each of these elements). 3. Using the data collected in class, as well as data from the table, determine the identity of elements A and B (they can be C, Si, Ge, Sn, or Pb). Substance A Substance B Mass ______ Initial volume water ____ Final volume water _____ Identity _____
Mass ______ Initial volume water ____ Final volume water _____ Identity _____
Example 5: Identification of Acids and Their Concentrations. Students are given the concentration of a sodium hydroxide solution. Next, phenolphthalein is added to two flasks marked A and B. Flask A contains a solution of nitric acid and flask B contains sulfuric acid. Each flask of acid is titrated to the endpoint with the sodium hydroxide solution and the students record the data in their table (shown below). After the titrations a small amount of barium nitrate solution is added to each titration flask. In flask B a white precipitate forms, whereas there is no visible reaction in flask A. The students are asked to answer the following questions: Collected Data 1.00 Concentration of sodium hydroxide solution ______ Amount of sodium hydroxide solution needed to neutralize the acid in beaker A _____________ Amount of sodium hydroxide solution needed to neutralize the acid in beaker B _____________ Result of reaction with barium nitrate and remains of neutralization in beaker A _____________ Result of reaction with barium nitrate and remains of neutralization in beaker B _____________ 1. One beaker contains sulfuric acid and the other contains nitric acid. The identity of the acid in beaker A is ________. Explain how you arrived at your response. The identity of the acid in beaker B is ________. Explain how you arrived at your response. 2. Using balanced chemical equations, determine the concentrations of the acids. The concentration of the acid in beaker A is ____ based on the following calculations. The concentration of the acid in beaker B is _____ based on the following calculations. 3. The identity of the solid formed in beaker B is _____. Explain how you arrived at your response. There should be ____ grams of the solid substance formed in beaker B based on the following calculations.
The examples above show the kinds of ways that identification questions can be asked based on demonstrations. In example 4 the students were asked to make observations of volume displacement so they could determine (based on the physical property of density) the identity of two substances. They were also asked to explain the periodic trend the data supported. In example 5 students had to use their knowledge about chemical reactivity (i.e., solubility of products) and combine it with their knowledge about stoichiometry to identify the concentrations of the unknown solutions.
Student Outcomes During the Spring semester some quantitative data were collected at the University of Southern Mississippi to examine the impact of including demonstration-based test items in the assessment program. Two sections of first-semester general chemistry were taught by two faculty with more than five years teaching experience. The classes met at the same time, used the same text, and had the same labs. During the semester both groups of students had five 50-minute exams. As part of four of their five exams, treatment students also had demonstration-based questions. At the end of the term the final exams used in the treatment and control sections had eight items in common. Four items from content areas covered by demonstration-based assessment activities and four items from other content areas were chosen from a pool of draft items in a test of conceptual understanding being developed by the ACS (13). Table 2 shows that there was no detectable difference between treatment and control students for content not covered by demonstration-based assessment tasks during the term. (This serves as an internal standard to demonstrate that the two groups did not differ in areas where treatment was not applied.) In contrast, treatment students outperformed control students on items for which they had been tested during the term. These results demonstrate that use of demonstration-based assessment can affect future performance on standardized tests emphasizing conceptual (rather than algorithmic) understanding. In addition, a survey instrument concerning assessment practices was administered to 4 laboratory sections of first-semester general chemistry containing both treatment and control students. (Some of the students were from the lecture using demonstration-based assessment items, and some were from the control class.) Although there were no detectable differences between the two groups as to preference or perceived accuracy of several assessment formats, a difference was detected between the treatment and control groups in how much time they spent studying with other students. Table 3 shows that treatment students studied with other students twice as much as their control counterparts. These results support the notion that demonstration-based cooperative assessment can enhance student interaction outside of class.
Table 2. Student Performance on Common Examination Items Correct Answers (% of students) Group
NondemonstrationRelated Items
Demonstration-Related Items
Control class
26
29
Treatment class
29 χ2
38
= 0.21; df = 1; p >.05
χ2
= 5.89 df = 1; p
