Incrementally Approaching an Inquiry Lab Curriculum: Can Changing

Research: Science and Education edited by

Chemical Education Research

Diane M. Bunce The Catholic University of America Washington, DC 20064

Incrementally Approaching an Inquiry Lab Curriculum: Can Changing a Single Laboratory Experiment Improve Student Performance in General Chemistry?

Renée S. Cole University of Central Missouri Warrensburg, MO 64093

Kristen L. Cacciatore Department of Chemistry, University of Massachusetts–Boston, Boston, MA 02125 Hannah Sevian* Departments of Chemistry and Curriculum and Instruction, University of Massachusetts–Boston, Boston, MA 02125; *[email protected]

Current research shows that wholesale change in general chemistry lab curricula from traditional, directive student experiments to discovery-based student experiments has a significant, positive impact on student learning (1–3). A recent research synthesis from the National Research Council reported that traditional laboratory experiences are not the most effective way to teach science (4), and the National Science Education Standards emphasize the critical role of student inquiry in science courses (5). However, many barriers to wholesale change exist, notably cost and institutional capacities, and as a result many general chemistry laboratory curricula remain largely traditional in pedagogy (6). One alternative to wholesale change is incremental change, the gradual modification of the curriculum to incorporate research about how students learn science best. To date, the impact of incremental change on student learning and performance in the general chemistry laboratory has not been fully explored. Descriptive reports (7–9) describe positive results, and one controlled research study reported a significant effect when students used a structured writing tool that promotes inquiry and reflection instead of a standard laboratory report format to complete an experiment on physical equilibrium in a general chemistry course (10). In our pilot studies of newly designed research-based inquiry experiments at our university, laboratory instructors reported that students were more engaged and exhibited a greater understanding of the goals and content of the experiments than these instructors have observed of students during traditional experiments. Students’ written work in the pilot studies supported those observations. We endeavor to extend and expand this line of research through a series of studies of incremental change in the general chemistry laboratory. These studies are examining the impact on student performance when the number of research-based inquiry experiments in the curriculum is increased from one, to two, to three, out of ten total experiments, and when the chemistry content and procedural complexity of the experiments varies. The goal of this series is to ascertain whether incremental change produces incremental improvement and is therefore a reasonable alternative to immediate wholesale change, or whether instead sweeping curriculum overhaul is necessary to improve student 498

learning in the general chemistry laboratory. America’s Lab Report (4)—a 2005 report from the National Research Council that summarized research findings about student learning in science laboratories—concluded that traditional, cookbook-type laboratories are not the most effective way to teach science. The NRC found that high-quality lab experiences are integrated with other components of the course, address prior student conceptions, promote student inquiry, and provide opportunities for metacognitive processing. This report identified seven learning goals for lab experiences:

1. Master subject matter

2. Develop scientific reasoning

3. Understand complexity and ambiguity of empirical work

4. Develop practical skills

5. Understand the nature of science

6. Cultivate interest in science and in learning science

7. Develop teamwork abilities

These goals have been studied to varying degrees. In our study of the impact of incremental change on the general chemistry curriculum we are focusing on the first three goals identified in America’s Lab Report. The university setting in which our research was conducted is a public, urban, commuter school with high-minority, older, nontraditional student population. The general chemistry lecture and lab curricula are traditional in terms of pedagogy. A First Step in Incorporating Incremental Change In this study, two-thirds of the students—the treatment group—in a general chemistry course completed an inquiry stoichiometry experiment (11) with a research-based design (referred to subsequently in this report as the “alternative experiment”), while the other third of the class, representing the control group, completed a traditional stoichiometry experiment that has been in use at the university for many years.

Journal of Chemical Education • Vol. 86 No. 4 April 2009 • www.JCE.DivCHED.org • © Division of Chemical Education

Research: Science and Education

Table 1 offers a comparison of the two experiments. In both experiments students heat an unknown solid to constant mass and apply stoichiometry to the data they collect in order to characterize the unknown solid. The experiment conducted by the treatment group utilizes a novel format in which students are provided with an incomplete lab report—which is missing either a detailed procedure, calculations to support the results, or a discussion—instead of a standard lab procedure and are asked to confirm or refute the report’s findings. Another difference between the two experiments is that the alternative experiment has a green chemistry approach, while the traditional experiment does not (11, 12). By contrast, in the traditional experiment students follow a very detailed step-by-step procedure to complete the experiment and calculations. Several days prior to the experiment students in both groups are given a prelab reading assignment containing stoichiometry problems related to the laboratory experiment. Hypotheses If incremental change in the chemistry laboratory curriculum as we have defined it does lead to measurable improvement in student content mastery, development of scientific reasoning, and understanding of the complexity and ambiguity of experimental science, then the following three differences between the treatment group and the control group may be observed: the treatment group will develop stronger experimental design skills and data analysis capabilities than the control group (Hypothesis 1); the treatment group will exhibit greater content understanding on problems asking for direct application of stoichiometry principles (Hypothesis 2); and the treatment group will exhibit greater content understanding on problems asking for indirect application of stoichiometry principles (Hypothesis 3). Further, if incremental change is effective, in our future work we expect that as the number of research-based inquiry labs completed by the treatment group increases, the performance differences between the treatment group and the control group will become larger and more generalized.

Study Structure and Particulars This study was carried out during Spring 2006 in a first semester general chemistry course for science majors. All students attended the same lectures (coauthor HS was the lecturer) and took the same exams. Each student enrolled in one of three lab sections, except for five students who had previously completed an equivalent laboratory course and were exempted. For this study two of the lab sections were randomly chosen to be treatment group (N = 45 total), while the third was the control group (N = 25). Each lab section had a different faculty instructor; HS did not teach any of the three lab sections. The treatment and control groups did nine identical laboratory experiments over the course of the semester. The only curriculum component that was different for the treatment and control groups was the stoichiometry experiment, which occurred during the fourth week of the ten-week laboratory course. Of 86 students who completed the course, our sample included only 70 of those students because we excluded students who were exempt from the lab, did not take one of the exams, or missed the lab of interest. Because inquiry-oriented experiments are not the norm at our institution, we provided several structures to support the implementation of the alternative experiment and we observed the actual laboratory sessions to determine whether students were engaging in the experiment in the manner in which it was intended. To this end, we taught an inquiry pedagogy training session that was attended by all faculty instructors and teaching assistants who taught the experiment during this study to prepare them to effectively facilitate the inquiry-based experiment. This session included: (i) discussion of the rationale for student inquiry in the laboratory; (ii) a description of the alternative experiment; and (iii) tips for implementing the laboratory successfully. We also provided a written instructor’s guide delineating explicit prompts to promote inquiry that instructors and TAs could use with students. Care was taken to have the new guide parallel as closely as possible the format of the instructor’s guide that is provided by the university for the

Table 1. Comparison of the Two Laboratory Experiments Used Lab Elements

Traditional Lab Experiment Properties

Alternative Lab Experiment Properties

Stoichiometry

Stoichiometry; green chemistry

Transferring solids; heating a solid to constant mass

Transferring solids; heating a solid to constant mass

Bunsen burner, crucible, analytical balance

Bunsen burner, crucible, analytical balance

Time on task

3 hours

3 hours

Pedagogical features

Step-by step laboratory procedure and calculations

Incomplete lab report serves as a template for procedure and calculations; explicit discussion of green chemistry

Toxic solid

None

Prelab questions

Mass A → mass B stoichiometry problems involving compounds to be experimented with

Mass A → mass B stoichiometry problems involving compounds to be experimented with

Calculations required during experiment

Grams → moles and moles A → moles B conversions

Grams → moles and moles A → moles B conversions

Calculations only

Calculations; critique of experiment template; green chemistry application

Chemistry content covered Laboratory techniques Equipment used

Hazards

Postlab questions

© Division of Chemical Education • www.JCE.DivCHED.org • Vol. 86 No. 4 April 2009 • Journal of Chemical Education

499


event: purpose:

pretest

Exam 1

stoichiometry lab

Exam 2

Exam 3

ACS final

interviews

demographics; prior knowledge

covariate

treatment

short-term

not used

longer-term

longer-term

Figure 1. Timeline of events and measures in the research study.

traditional experiment, which contains sample data, a grading scheme detailing the number of points awarded for each step of the required calculations, and tips about common procedural errors that students may make. One or the other of the authors attended each of the three laboratory sessions during which students completed either the alternative or the traditional experiment. Attending the labs was for the sole purpose of observing students at work in order to make qualitative comparisons about students’ interactions with instructors and each other, their degree of engagement, and other aspects of their behavior in the lab. The author–observers played no formal or informal instructional role during the sessions. Following the three laboratory sessions the first author (KLC) informally interviewed each of the instructors and TAs individually about their perceptions of the students’ behavior and learning during the sessions. At the beginning of the semester, all participants completed a precourse demographic survey and Chemistry Concepts Inventory (13). Demographic parameters included students’ gender, English language learner status, college major, and history of prior chemistry courses; no significant differences on any of these parameters existed between the lab sections. Quantitative assessments used in this study include three midterm exams and the final exam. The midterms were a mix of American Chemical Society (ACS) multiple-choice questions and open-response questions drawn from several sources, including instructor-

designed questions and questions modeled after advanced placement (AP) chemistry exams. The first midterm occurred before the stoichiometry laboratory. The second midterm was two weeks following the lab and one-third of it was stoichiometry or stoichiometry-related. The final exam was the 2005 ACS First Term General Chemistry exam. These events and their chronology are illustrated in the timeline shown in Figure 1. In addition to the primary quantitative assessments used above, seven students were interviewed about their experiences in the laboratory. This occurred during a two-week period after the final exam and involved four students who had completed the alternative lab, and three students who had completed the traditional experiment. Interview volunteers were solicited from the chemistry lecture course and were interviewed once in semi-structured sessions that ranged from 20 to 45 minutes by an undergraduate research associate trained by an experienced qualitative researcher on the university faculty. Transcripts of the audiotaped interviews were analyzed through a phenomenological, qualitative approach that utilized open coding (14) to determine the nature of relationships that existed between students’ impressions of their laboratory experiences, their understanding of stoichiometry content, and their ability to design a laboratory procedure. Students’ written work on a stoichiometry problem posed during the interview was also coded and analyzed. Textbox 1 shows the interview protocol and reasoning for including these questions in the interviews.

Interview Protocol

Question Rationale

1 Which labs do you remember best?

Helps students recall their experiences in lab, in order to prepare them for later questions that ask them to recall details about those experiences.

2 Which labs were most helpful in learning the lecture material? Why?

Assesses student’s beliefs about the connections between lecture and laboratory portions of the chemistry course

3 Earlier this semester you did a lab that involved heating a solid to a constant mass. Why might someone want to heat a solid to a constant mass?

Probes whether students understand and recall the procedures’ purpose in both stoichiometry experiments

4 Describe how you would heat this sample (point to bottle of white solid) to a constant mass, including what equipment you would use (show laboratory equipment including balance, crucible, graduated cylinder, Bunsen burner, wash bottle, stirring rod, spatula, ringstand) how you would use it, and what measurements you would take.

Assesses students’ recall and understanding of the laboratory procedure and equipment used in both stoichiometry experiments

5 This question asks you to do some calculations like those you did in lab this semester. (Interviewee is given paper, a periodic table, and a pencil.)

Assesses students’ stoichiometry problem-solving abilities and strategies on a task very similar to the one they did using the data they collected in both stoichiometry experiments

“A sample of an unknown solid material originally has a mass of 5.403 g. After heating to a constant mass its mass is 3.453 g. If all of the mass lost is water, how many moles of water were lost? The unknown solid was either CuSO4·5H2O or CoCl2·6H2O. Which one was it? How do you know?” Textbox 1. Interview protocol and rationale.

500



Statistical Analysis Typically, comparisons between treatment and control groups are carried out using a t-test comparison of means on each performance measure. Because of our study design, in which the treatment group consisted of two distinct lab sections, we had to be concerned about the chance variability between those lab groups. This issue prevented us from simply lumping data from the two treatment sections into one large pool and conducting a t-test to compare the treatment and control groups. Instead we used a linear contrast (15) to rigorously test whether any differences between the treatment group and the control group were larger than random differences that occur between the two treatment lab sections. Linear contrast is a general linear model, similar to regression analysis, and is mathematically equivalent to a hierarchical ANOVA or a nested ANOVA. Because the students were not randomly assigned to these three lab sections from some larger population, lab section is considered as a fixed effect. We confirmed that the assumptions of ANOVA were not violated in this analysis, meaning that the variances within the treatment and control groups on each measure are statistically equivalent. In addition, all measures used can be considered valid because the range of student performance in both the treatment and control groups was in the middle of the scale used to assess each measure (see Table 2 for mean scores). A second potential issue that our analysis accounts for is differences between the treatment and control groups that existed prior to the study event. The treatment group performed better than the control group on both performance measures that occurred before the stoichiometry experiment, the CCI pretest and the first course exam. Although these preexisting differences between the groups were not statistically significant ( p = 0.20 on pretest and p = 0.08 on Exam 1), we nonetheless were concerned that these differences could lead to the erroneous conclusion that differences in student performance after

the stoichiometry experiment were due to a positive treatment effect when they were in fact at least partly due to preexisting differences between the treatment and control groups. This concern led to the use of students’ Exam 1 score as a covariate in our linear contrast analysis. The use of the covariate controls for any preexisting differences between the groups. Exam 1 is an appropriate covariate for two reasons. It was similar in structure to the later course exams used in our analysis of treatment effects, and it contained several problems that are related to stoichiometry (e.g., mole–mass conversions). Therefore students’ Exam 1 scores provide a measure both of students’ general likelihood of success on exams of the type used in the course and of students’ knowledge and skills that would affect their future success in solving stoichiometry problems in particular. In summary, our statistical analysis differs from the generally used t-test comparison of means through the use of linear contrast and the inclusion of an appropriate covariate in all analyses. Both of these variations make the analysis statistically more stringent than a simple t-test, thereby increasing our confidence in the robustness of the findings reported here. Quantitative Findings Analyzing Hypothesis 1 Our first hypothesis is that the treatment group will develop stronger experimental design skills, reasoning skills, and data analysis capabilities than the control group. Measures used to test this hypothesis are parts of the Exam 2 stoichiometry lab open-response problem (Textbox 2), and three basic lab skills multiple-choice problems on the ACS final (Table 3). The Exam 2 open-response question contained two parts; the first part (Part A) asked students to design a procedure for and analyze sample data from an experiment very similar to the stoichiometry experiments completed by both the treatment and control groups. The second part (Part B) of this problem required

Table 2. Summary of Results, Effect Sizes, and p Values Hypothesis

Control Group Mean

Treatment Group Mean

p Value

Effect Size

Open response question on Exam 2, part (a)

3.84

5.44

0.010

1.4 of 7 points

Basic lab skills questions on final exam

1.12

1.44

0.395

not applicable

Five multiple-choice questions on Exam 2 and final exam

2.32

3.33

0.018

0.70 of 5 questions

Open response question on Exam 2, part (b)

0.72

1.53

0.065

not applicable

Both 1 and 2 from above

Open response question on Exam 2, total score

4.56

6.98

0.005

1.9 of 11 points

3 Problem solving on questions indirectly related to two-substance stoichiometry

Fourteen multiple-choice questions on Exam 2 and final exam

7.12

7.84

0.88

not applicable

1 Lab skills

2 Problem solving on questions directly related to two-substance stoichiometry

Measure


501

Research: Science and Education Table 3. ACS Questions on Lab Skills Tested on Final Exam Question

Description

1

Characterize accuracy and precision of a data set

2

Compare precision of different laboratory measurements

3

Determine mass of solid transferred by subtraction

students to perform stoichiometry calculations with the data; student performance on this section is considered as part of the analysis of Hypothesis 2 below. This question was designed by the instructor based on similar problems from past AP chemistry exams. The problem was graded using a rubric (Textbox 3) by the first author; grading was anonymous as no names or other identifying information were attached to the problems. The problem was independently re-graded by two experienced general chemistry instructors at our university not involved in this study to verify its reliability. Cronbach’s alpha (α) values for

An impure sample containing aluminum carbonate, an antacid, is to be analyzed in the laboratory. The aluminum carbonate undergoes the following reaction when heated: Al2(CO3)3(s)

each of the two parts of the question and the question as a whole are greater than 0.98 (Part A α = 0.987, Part B α = 0.998, total score α = 0.987). Two other chemistry faculty at our university with extensive general chemistry teaching experience independently evaluated the rubric and exam question prior to their use in this study; both faculty considered the question and rubric fair, clearly stated, and valid as a measure of stoichiometry problem-solving skills and basic experimental design and data analysis skills. The treatment group significantly outperformed the control group on Part A, the experimental design and data analysis portion, of the open-response problem ( p = 0.010). The treatment effect is estimated to be 1.4 points out of 7, a large improvement. The multiple-choice laboratory questions on the ACS final were not specifically related to stoichiometry; rather they addressed general lab skills such as measuring volume in a graduated cylinder. There was not a statistically significant difference between the treatment and control groups on these questions as a group ( p = 0.395).

Part A (7 points) Either: • Measure mass of empty crucible (1 point) • Measure mass of crucible with sample (aluminum carbonate) in it (1 point)

→ Al2O3(s) + 3CO2(g)

It is assumed that the rest of the impure sample does not react when heated (i.e., it is inert). Equipment: mass balance, 250-mL beaker, crucible and lid, tongs, Bunsen burner, wash bottle, scoopula, graduated cylinder Part A, Section 1 Using any of the above lab equipment and materials (you do not need to use all the equipment), briefly outline the procedure for obtaining the measurements needed to determine the mass percentage of aluminum carbonate present in the impure sample. Clearly identify in your procedure outline which equipment is needed. Part A, Section 2 A student collected the following data. Determine the mass of carbon dioxide gas released. Mass of empty crucible

10.014 g

Mass of crucible with sample

13.712 g

Mass of crucible after final heating 11.769 g Part B Using your answer from the calculation above and the values given below, determine the mass of the aluminum carbonate that reacted. Molar Masses Al2(CO3)3

233.99 g/mol

Al2O3

101.96 g/mol

Or:

• Measure mass of sample alone (2 points)

• Place crucible in clay triangle on ringstand over Bunsen burner and heat crucible for a period of time (1 point) • Measure mass (1 point) • Repeat heating and cooling until mass no longer changes (1 point) • Set up subtraction calculation (1 point) • Answer is 1.943 g (1 point)

Part B (4 points, 1 point for each bullet) Note: A student’s answer from Part A may be wrong, leading to wrong results in Part B. Such a student can still earn all 4 points for Part B if the steps are done correctly. • Start with answer from Part A (1 point) • Convert mass CO2 from Part A to moles (1 mol = 44.01 g) (1 point) • Use 1:3 mole ratio to convert moles CO2 to moles Al2(CO3)3 (1 point) • Convert moles Al2(CO3)3 to mass Al2(CO3)3 (1 mol = 233.99 g) (1 point)

CO2 44.01 g/mol Textbox 2. Open-response stoichiometry question on Exam 2, which followed the stoichiometry experiment.

502

Textbox 3. Rubric used to assess open-response stoichiometry question on Exam 2. (The whole question is worth 11 points total.)



Analyzing Hypothesis 2 Our second hypothesis is that the treatment group should exhibit greater content understanding on problems asking for direct application of stoichiometry principles. These problems ask students to convert from an amount of substance A to an amount of substance B based on a reaction involving A and B. We excluded solution stoichiometry problems from this category because the laboratory experiments did not include solutions, and we focused our analysis on problems very similar to those encountered in the experiments we are comparing. The two measures used to test this hypothesis are the total number of correct answers on the five multiple-choice stoichiometry problems contained on the Exam 2 and the final exam (see Table 4 for a summary) and Part B of Exam 2 open-response question (see Textbox 2), which was a mass–mass stoichiometry calculation. Our analysis shows that the treatment group outperformed the control group on the five multiple choice questions combined ( p = 0.018). The treatment effect is estimated to be 0.70 questions out of five (95% confidence interval 0.12 to 1.28 questions), which represents a substantial improvement. The difference between the treatment and control groups on the Exam 2 open-response Part B was not statistically significant at the p = 0.05 level ( p = 0.065), yet this suggests that the students in the treatment group performed better than their control group counterparts. In addition, when the Exam 2 open-response question total score (the sum of Parts A and B) is considered, the difference between the two groups is highly significant ( p = 0.005), with the treatment group performing an estimated 1.9 points better on an 11-point scale (95% confidence interval 0.6 to 3.3 points). Analyzing Hypothesis 3 Our third hypothesis is that the treatment group may exhibit greater content understanding on problems asking for indirect application of stoichiometry principles. These problems do not involve A → B conversions of the type addressed in Hypothesis 2, but do ask students to use some of the same knowledge and mathematical steps as stoichiometry problems. Examples include mass–mole conversions and empirical formula determinations. The measure used to test this hypothesis is the total number of correct answers on the 14 multiple-choice problems on Exam 2 and the final exam (Table 5). We found no statistically significant difference between the treatment and control groups on this measure ( p = 0.88). The results for each of the hypotheses are compared in Table 2.

engagement in the laboratory, and student comments about the experiment. The questions posed by students during the traditional laboratory session were fewer in number and they focused almost exclusively on procedural details and formulas to use for calculations, for example, “Is this flame hot enough?” and “Does the molar mass of barium chloride hydrate need to include the mass of water?” By contrast, the students in the alternative experiment posed more than three times as many questions, and the nature of most of these questions was qualitatively different in that they reflected higher-order thinking about both the procedure and how to analyze and understand the resulting data, for example, “Do we have to weigh the dish (crucible) empty before we put in the chemical…otherwise how can we tell how much the weight changes?” Also, several students in the alternative laboratory made comments that reflected the Table 4. ACS Questions on Two-Substance Stoichiometry Problems Tested on Exam 2 and Final Exam Question

Exam

1

2

Description

2

Final

Mass–mass stoichiometry with limiting reagent

3

Final

Mass–mass stoichiometry

4

Final

Mole–mole stoichiometry with limiting reagent

5

Final

Mass–mass stoichiometry with percent yield

Determine % composition using mass–mass stoichiometry with limiting reagent

Table 5. ACS Questions Indirectly Related to Stoichiometry Tested on Exam 2 and Final Exam Question

Exam

Description

1

2

Determine stoichiometric coefficients to balance given equation

2

2

Use stoichiometry to determine acidity

3

2

Determine molarity from mass and volume

4

2

Determine molarity of solution produced by mixing two solutions

5

Final

Determine empirical formula from percent composition data

6

Final

Determine stoichiometric coefficients to balance given equation

7

Final

Convert mass to moles

Qualitative Findings

8

Final

Convert mass to number of particles

This study is primarily quantitative in nature; the purpose of the qualitative aspects of the study—including observations of the laboratory sessions, informal interviews of the TAs and instructors, and interviews of a small sample of student participants—was to support and refine quantitative findings as well as to inform future work. Our quantitative findings show significant differences in student learning between the alternative and traditional laboratory groups and the results of our qualitative analysis, as summarized below, support this conclusion. Our observations during the traditional and alternative laboratory experiments revealed marked differences in student– instructor and student–student interactions, students’ apparent

9

Final

Determine mass percent of an element in a compound

10

Final

Determine molarity of ions in a solution containing two sources of the ion

11

Final

Determine dilution volume of stock solution to obtain given molarity

12

Final

Use given molarity and volume of solution to determine mass in solution

13

Final

Determine volume needed to reach equivalence point

14

Final

Solution stoichiometry


503


challenging nature of the task, for example, “My brain hurts from thinking for three hours!” and no such comments were observed during the traditional lab. Both author–observers and the laboratory instructors reported more student–student interaction in the alternative laboratory than was typical during the more traditional experiments that preceded and followed it during the semester. Analysis of the seven student interview transcripts and the interviewees’ written work in response to question 5 in the interview indicate some trends supportive of the quantitative findings as well. Open coding of transcripts was used to identify twelve different steps in the laboratory procedure, to distinguish between correct and incorrect sequencing of steps, to identify eight pieces of laboratory apparatus used in the experiment, and to identify seven different calculation steps as well as correct results at each step. Compared to the three interviewees who completed the traditional experiment, the four interviewees who completed the alternative experiment gave more detailed and accurate laboratory procedures in response to question 4 of the interview protocol (see Textbox 1) and persisted longer and tried more strategies to solve both parts of the stoichiometry problem in question 5. More significantly, the four alternative lab interviewees performed better on the stoichiometry problem, with one student answering the entire problem correctly and two students answering parts of the problem correctly. Of the three traditional lab interviewees, none answered the problem entirely correctly and only one answered any parts correctly. This suggests a real difference between what the two groups learned because the problem posed during the interview was more closely related to the calculations done in the traditional experiment than those required for the alternative experiment. Discussion We found that the treatment group did significantly better than the control group on a complex experimental design and data analysis open-response problem that was directly related to stoichiometry. Conversely, the treatment group did not perform better than the control group on multiple-choice questions requiring general laboratory knowledge and skills not related to stoichiometry. These findings indicate that there was a positive treatment effect on student development of reasoning skills needed for success in experimental work. Only the treatment group had direct experience with experimental design in this study, so this finding may seem obvious at first. However, we argue that it is meaningful because it shows a measurable difference in skills as a result of changing one single experiment out of ten, where in the one revised experiment students must design their own procedure and data analysis scheme instead of a similar experiment in which the procedure and calculations are prescribed. A very small intervention had a real effect that persisted at least two weeks. Nevertheless, this effect extended only to experimental work closely related to the content matter and laboratory techniques in the inquiry laboratory experiment. Our analysis suggests that students may need to conduct a number of inquiry experiments that address a wide range of chemistry content and laboratory techniques in order to acquire generally strong experimental skills. We found that students in the treatment group did significantly better than those in the control group on problems requiring direct application of stoichiometry principles, but did not 504

perform better on problems indirectly related to stoichiometry. Our findings indicate that there was a positive treatment effect on student learning of chemistry content, although this effect did not extend beyond the chemistry content addressed in the inquiry experiment. This analysis suggests that student content understanding and performance in the general chemistry course may be improved through the use of inquiry experiments in the laboratory. However, improvements in student performance may only occur in content areas addressed in the inquiry experiments. Two features, green chemistry and inquiry, make the alternative lab experiment different from the traditional lab. We were not explicitly studying whether a focus on green chemistry influences student learning, although it is possible that it does. Thus, causality cannot be concluded as to whether the difference in performance by students on stoichiometry content resulted from the inquiry nature of the treatment or from possible increased motivation by students in the treatment group to study stoichiometry because the alternative lab used a green chemistry approach, and this is an unfortunate design flaw in the study. However, we do not believe that green chemistry features in the alternative experiment are the source of the observed differences in student learning in the treatment and control groups, for the following reasons. First, our observations in the laboratory indicate that the primary difference in student response was related to the inquiry nature of the task. Students’ interactions and questions in the lab revolved around how to carry out the experiment and how to use the data to answer the question of interest. Also, the green chemistry part of the postlab analysis represented a much less significant portion of the time students spent in the laboratory than did the inquiry tasks of designing and carrying out the procedure, and postlab tasks of figuring out calculations and critiquing the lab report used as a template. Finally, there is no evidence in the chemistry education literature that studying green chemistry aids student learning of core chemistry content. In order to provide stronger evidence that the inquiry nature of the laboratory was the cause of the improved student performance, and not the green chemistry orientation of the activity, we have undertaken a subsequent study. In this second study the study design was essentially identical to the one reported here except that the inquiry-based experiment was not explicitly a green chemistry approach, so the only difference between the experiments was whether they were inquiry-based or not. Preliminary analysis of student learning in this later study indicates that students who completed the alternative (inquiry) experiment outperformed students who did the traditional experiment, thus supporting our contention that the differences described in this study can be ascribed to the inquiry features of the alternative experiment. The complete findings from the second study will be reported in a subsequent paper. These conclusions must be considered tentative, as they have been formulated based on a small study involving only a single experiment. However, these results are also promising, because a small intervention—substituting a single researchbased inquiry experiment for a traditional experiment on the same material—led to measurable improvements in student performance in their general chemistry lecture course. If confirmed through future studies, these results would support the value of gradual change from traditional to research-based inquiry-oriented general chemistry laboratory curricula for those institutions that cannot implement immediate and dramatic curriculum reform programs.



Future Work The results and analysis presented here suggest several additional avenues for further exploration of the impact of incremental change on student learning in general chemistry. We are in the process of carrying out studies to examine student learning following inquiry experiments on topics other than stoichiometry in order to ensure that these findings apply to other general chemistry content. These studies will also measure the impact of larger curricular change increments by studying student learning after changing out two or three inquiry experiments instead of one as in this study. The aim of our series of studies is to examine whether larger and more generalized improvements in student performance occur as laboratory activities are shifted toward more inquiry, and whether there is a point at which the return on the investment begins to diminish. Acknowledgments The authors thank Eugene Gallagher for his invaluable guidance on statistical analysis and Carol Smith for sharing her expertise with qualitative methods. We also thank Jose Amado for his assistance with interviewing and data entry, and all of the faculty, graduate teaching assistants, and general chemistry students who participated in this study. This work was funded in part by NSF EHR-0412390.

3. Bodner , G.; Hunter, W.; Lamba, S. Chem. Educator 1998, 3 (3), 1. 4. National Research Council. America’s Lab Report: Investigations in High School Science; National Academy Press: Washington, DC, 2005. 5. National Research Council. National Science Education Standards; National Academy Press: Washington, DC, 1996. 6. Hofstein, A.; Lunetta, V. Sci. Ed. 2004, 88, 28. 7. Green, W. J.; Elliott, C.; Cummins, R. H. J. Chem. Educ. 2004, 81, 239. 8. Mills, P. A.; Sweeney, W. V.; Marino, R.; Clarkson, S. J. Chem. Educ. 2000, 77, 1161. 9. DeMeo, S. J. Chem. Educ. 2001, 78, 201. 10. Rudd, J. A., II; Greenbowe, T. J.; Hand, B. M.; Legg, M. J. J. Chem. Educ. 2001, 78, 1680. 11. Cacciatore, K. L.; Sevian, H. J. Chem. Educ. 2006, 83, 1039. 12. Anastas, P. T.; Warner, J. C. Green Chemistry: Theory and Practice; Oxford University Press: New York, 1998. 13. Robinson, W. R.; Mulford, D. R. J. Chem. Educ. 2002, 79, 739. 14. Creswell, J. W. Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research; Pearson Education: Upper Saddle River, NJ, 2005. 15. Kutner, M. H.; Nachtsheim, C. J.; Neter, J. Applied Linear Statistical Models, 5th ed.; McGraw-Hill: New York, 2005.

Literature Cited

Supporting JCE Online Material

1. Oliver-Hoyo, M. T.; Allen, D.; Hunt, W. F.; Hutson, J.; Pitts, A. J. Chem. Educ. 2004, 81, 441. 2. Farrell, J. J.; Moog, R. S.; Spencer, J. N. J. Chem. Educ. 1999, 76, 570.

Abstract and keywords

http://www.jce.divched.org/Journal/Issues/2009/Apr/abs498.html

Full text (PDF) with links to cited JCE articles


505

Incrementally Approaching an Inquiry Lab Curriculum: Can Changing

Recommend Documents