Article pubs.acs.org/jchemeduc
Assessing High School Student Learning on Science Outreach Lab Activities Courtney L. Thomas* Department of Chemistry, Susquehanna University, Selinsgrove, Pennsylvania 17870, United States ABSTRACT: The effect of hands-on laboratory activities on secondary student learning was examined. Assessment was conducted over a two-year period, with 262 students participating the first year and 264 students the second year. Students took a prequiz, performed a laboratory activity (gas chromatography of alcohols, or photosynthesis and respiration), and followed up with a postquiz. The percentage of correct answers on the pre- and postquiz was tallied for each student and each question. A paired t-test was performed to detect differences in scores. Student scores increased significantly after both laboratory activities both years (p < 0.001). For each gas chromatography of alcohols quiz question, a significant increase was observed the second year (p = 0.041), but not the first year (p = 0.11). For each photosynthesis and respiration quiz question, a significant increase was observed the first year (p = 0.038), but not the second year (p = 0.066). However, when one specific photosynthesis and respiration quiz question was omitted, both years showed a significant increase (first year, p = 0.0064; second year, p = 0.039). Overall, these results indicate laboratory activities provided by science outreach programs can positively contribute to student learning. KEYWORDS: High School/Introductory Chemistry, Public Understanding/Outreach, Hands-On Learning/Manipulatives, Gas Chromatography, Photosynthesis
I
equipment and instrumentation in the hands of students to advance the use and comprehension of technology. Laboratory activities for biology, chemistry, and physics classrooms are available through Science in Motion. Pennsylvania high school science teachers interested in requesting an activity contact the nearest Science in Motion site to arrange a visit with a mobile educator. Mobile educators are certified teachers with expertise in a specific area (biology, chemistry, or physics) who work with teachers to arrange times for Science in Motion laboratory activities to be brought to the teacher’s classroom. The current Susquehanna University Science in Motion (SU−SIM) mobile educator is a retired teacher with over 30 years of secondary science education experience. Teachers using Science in Motion retain control of their curriculum by choosing the activities for their classroom. Partnered with colleges, Science in Motion sites provide laboratory activities at no cost to schools. The Commonwealth of Pennsylvania provides funding for salaries and supplies; the colleges provide free housing and access to university resources, including laboratory equipment and faculty. As the Susquehanna University Science in Motion program director for six years, I witnessed participation by schools and teachers increase each year. With current state and federal budget cuts, outreach programs are being drastically cut or even eliminated. At SU−SIM, the program has scaled back from three full-time mobile educators to just one for the 2010−2011 school year because of decreased funding. During the 2011−2012 school year, the program only received funding to operate from January to June 2012 with one mobile educator. To make a case for continued funding, outreach programs must demonstrate
n January 2011, the United States Department of Education unveiled the results of the 2009 National Assessment of Educational Progress. These results show that only 21% of 12th-grade students perform at a proficient level (solid academic performance) in science.1 In addition to these results, the Programme for International Student Assessment reported the results of their 2009 assessment, which found that American students continue to be outperformed by their international peers in science.2 These troubling findings raise the question: What can be done to improve science education in America? Active learning is important, especially in science education.3 Recent research indicates the concept of “deliberate practice”, in which students practice science reasoning and problem solving during class with instructor feedback, results in higher learning gains compared with a traditional lecture approach.4 Students learn by doing. Teachers are encouraged to incorporate activities in the classroom to help their students develop a deeper understanding of concepts.5 In science courses, those activities can take the form of laboratory experiments. While many teachers try to incorporate such activities in the high school science classroom, budget or background knowledge may limit the availability of experiments for students. In an effort to improve secondary education science instruction, many organizations have developed science outreach programs, including the Science in Motion program originating at Juniata College.6 Precollege science outreach has been shown to increase science content knowledge and student motivation to study science.7 Currently, 12 colleges across the Commonwealth of Pennsylvania participate in Science in Motion outreach. The Science in Motion program at Susquehanna University, similar to other sites, provides hands-on science laboratory activities for high school students. The program specifically aims to put current scientific © 2012 American Chemical Society and Division of Chemical Education, Inc.
Published: June 21, 2012 1259
dx.doi.org/10.1021/ed200320g | J. Chem. Educ. 2012, 89, 1259−1263
Journal of Chemical Education
Article
their effectiveness through assessment. This study examines student learning through a hands-on gas chromatography (GC) of alcohols or photosynthesis and respiration laboratory activity. Student learning is assessed through performance on a quiz taken before and after the activity.
■
METHODS
Gas Chromatography of Alcohols Laboratory Activity
The laboratory activity Using Gas Chromatography: Separation of Alcohols was adapted from the Juniata College Science in Motion program.8 The objective of this activity is for students to perform qualitative and quantitative analysis of samples using the GC. Pennsylvania State Academic Standards 3.7.12.A (apply advanced tools, materials, and techniques to answer complex questions), 3.4.10.A (apply knowledge of mixtures to appropriate separation techniques), and 2.1.8.D (apply ratio and proportion to mathematical problems) are also targeted. This activity was chosen for its popularity with teachers as well as its hands-on approach to teach an important concept in chemistry, gas chromatography. The GC instrument is connected to a laptop computer running PeakSimple software for data collection. In this activity, students run a standard solution of equal volume proportions (1:1:1) ethanol, methanol, and 2-propanol. Students record retention time and calculate area under the curve for each alcohol. After the standard, students run two to three unknown samples, which contain the same alcohols, but in different proportions (e.g., 5:2:3, methanol, ethanol, 2-propanol) or only two of the three alcohols are present (e.g., 1:1 methanol, ethanol). If time allows, some teachers also request students analyze alcohol content in other samples, such as mouthwash, windshield washer fluid, and rubbing alcohol. Enough portable GC instruments are provided for students to work in small groups (3−5) and students take turns using the syringe to load samples. A four-question, multiple-choice quiz was developed and given to students before and after the laboratory activity (Figure 1). Secondary chemistry students (juniors and seniors) taught by the same teacher in one school served by SU−SIM participated in the study. During the 2007−2008 school year, 80 students participated; the number decreased to 59 for the 2008−2009 school year (one less class period). The percentage of correct answers on the pre- and postquiz was tallied for each student as well as each question. A paired t-test (two-tailed) was performed to identify a difference between pre- and postquiz scores.
Figure 1. Quiz taken before and after the gas chromatography of alcohols activity.
covered with aluminum foil) and 5 min from a sample in light conditions (aluminum foil removed, water bottle used as heat sink between a lamp and the container). Three minutes pass before collecting data in the light to allow photosynthesis to ramp up after the “dark” data collection period. Students work in small groups (2−4 students) to ensure hands-on participation during the experiment. The teacher and Science in Motion mobile educator circulate around the room to answer questions and assist when needed. Students continue to work in groups to analyze their own data using Vernier Logger Pro software. The collected data are displayed as a graph: students highlight areas where CO2 level readings increase (during respiration, in the dark) or decrease (during photosynthesis, in the light), perform a linear regression to obtain a best fit line, and record the slope of the line as the rate of respiration or photosynthesis (ppt/min). After cleaning up, students compare the rates and answer questions on the activity. Owing to time constraints, this analysis is often postponed until the next class meeting. A four-question, multiple-choice quiz was developed and given to students before and after the laboratory activity (Figure 2). High school biology students (juniors and seniors) served by SU−SIM participated in the study. During the 2007− 2008 school year, 182 students in two schools participated. The number of participants increased to 205 during the 2008−2009 school year when a third school was added. The percentage of correct answers on the pre- and postquiz was tallied for each student as well as each question. A paired t-test (two-tailed) was performed to identify a difference between pre- and postquiz scores.
Photosynthesis and Respiration Laboratory Activity
Photosynthesis and Respiration (biology experiment 31B) from Vernier9 was used for this laboratory activity, in which the objective is for students to determine the rate of respiration and photosynthesis of a plant. Pennsylvania State Academic Standards 3.3.10.B (describe and explain the chemical and structural bases of living organisms), 3.6.10.A (describe various methods of biochemical conversion), and 3.7.12.A (apply advanced tools, materials, and techniques to answer complex questions) are also targeted. This activity was chosen for its popularity as well as its hands-on approach to teach the core concepts of photosynthesis and respiration. In this activity, students use a carbon dioxide (CO2) sensor attached to a computer through a digitizer to record the amount of this gas present in a closed container of spinach leaves. Students collect data for 5 min from a sample in dark conditions (container
■
RESULTS
Gas Chromatography of Alcohols Quizzes
To determine whether student scores on the gas chromatography of alcohols quiz changed after the laboratory activity, 1260
dx.doi.org/10.1021/ed200320g | J. Chem. Educ. 2012, 89, 1259−1263
Journal of Chemical Education
Article
Analysis of each question (1−3) was also pursued. Students answered more questions correctly on the postquiz both years (Table 1). To determine whether the change was significant, a Table 1. Gas Chromatography of Alcohols Quiz Individual Question Assessmenta Correct Answers, % Period 2007−2008b Q1 Q2 Q3 2008−2009c Q1 Q2 Q3
Before Activity
After Activity
Difference, %
80 83 58
96 98 99
+16 +15 +41
78 75 54
90 100 78
+12 +25 +24
a
Note: Because of concerns that students could not know the answer to question 4 before performing the experiment, those data were omitted from analysis here. bN = 80. cN = 59.
paired t-test (two-tailed, n = 3) was performed on the data in Table 1. For the 2007−2008 data, p = 0.11, which is not significant at a confidence level of 95%, because p > 0.05. However, the 2008−2009 data did show a significant increase with p = 0.041.
Figure 2. Quiz taken before and after the photosynthesis and respiration activity. For question 1 on the prequiz, both answer A (CO2) and answer C (O2) were accepted as correct, indicating student knowledge of what gases are involved in photosynthesis.
Photosynthesis and Respiration Quizzes
To determine whether student scores on the photosynthesis and respiration quiz changed after the laboratory activity, individual student pre- and postquiz scores were analyzed. Preand postquiz mean and SEM data are shown for both years in Figure 4. Individual student scores were subjected to a paired t-
individual student pre- and postquiz scores were analyzed. Concern was raised that students could not know the answer to question 4 before performing the experiment, so it was not a fair question to measure learning. Thus, question 4 data were omitted from analysis. Concern was also raised regarding student ability to answer question 3 prior to the experiment. Question 3 data were not omitted from analysis, however, as the wording of the question, specifically the phrase “to start data collection” should be enough of an indication to allow students with conceptual knowledge of GC to correctly answer the question before performing the activity. Pre- and postquiz mean and standard error of the mean (SEM) data are shown for both years in Figure 3. Individual student scores were subjected to a paired t-test (two-tailed). For the 2007−2008 data (n = 80), p = 4.5 × 10−11. Analysis of the 2008−2009 data (n = 59) revealed p = 3.1 × 10−5. Both results were very significant at a confidence level of 99.9% (p < 0.001).
Figure 4. Comparison of pre- and postquiz means on the photosynthesis and respiration quiz for both years. Error bars indicate values for the standard error of the mean.
test (two-tailed). For the 2007−2008 data (n = 182), p = 5.7 × 10−51. Analysis of the 2008−2009 data (n = 205) revealed p = 1.8 × 10−33. Both results were strongly significant at a confidence level of 99.9% (p < 0.001). Comparison of answers to each question was also pursued. Students answered more questions correctly on the postquiz in both years tested (Table 2). To test whether the change was statistically significant, a paired t-test (two-tailed, n = 4) was performed on the data in Table 2. For the 2007−2008 data, p = 0.038. At a confidence level of 95% (p < 0.05), this result is significant. For the 2008−2009 data, p = 0.066. At a confidence
Figure 3. Comparison of pre- and postquiz mean results for the gas chromatography of alcohols quiz both years. Error bars indicate values for the standard error of the mean. 1261
dx.doi.org/10.1021/ed200320g | J. Chem. Educ. 2012, 89, 1259−1263
Journal of Chemical Education
Article
Table 2. Photosynthesis and Respiration Quiz Individual Question Assessment
Table 3. Photosynthesis and Respiration Quiz Individual Question Assessment Omitting New School Data
Correct Answers, % Period
Before Activity
2007−2008a Q1 Q2 Q3 Q4 2008−2009b Q1 Q2 Q3 Q4 a
After Activity
Correct Answers, % Difference, %
42 67 20 27
96 76 91 93
+54 +9 +71 +66
67 69 32 33
97 73 89 95
+30 +4 +57 +62
a
2008−2009a
Before Activity
After Activity
Difference, %
Q1 Q2 Q3 Q4
70 69 34 35
97 77 90 97
+27 +8 +56 +62
N = 154.
resulted in a slightly lower probability of obtaining the same results by chance (p = 0.060). This value is still not quite significant at a 95% confidence level. Thus, addition of a new school during the 2008−2009 school year did not mask a significant result and did not need to be omitted. One trend noticed in the study was the minimal increase in percentage correct answers on question 2 of the photosynthesis and respiration quiz. The difference range was between +30 and +66 percentage points for questions 1, 3, and 4, while the range was only between +4 and +9 percentage points for question 2 (Table 2). In fact, when a paired t-test was performed on scores for questions 1, 3, and 4 (omitting question 2 scores), p = 0.0064 for the 2007−2008 data and p = 0.039 for the 2008− 2009 data. Both of these results are significant at a 95% confidence level. The minimal increase in the percentage of correct answers on question 2 indicates that the photosynthesis and respiration activity was not as successful in demonstrating that knowledge as hoped. The minimal achievement in learning of this fundamental concept will be addressed in future activities. Overall, the results of this study showed that student performance on a quiz improved after a laboratory activity. These results indicate that laboratory activities provided by science outreach programs, such as Science in Motion, can positively contribute to student learning.
N = 182. bN = 205.
level of 95%, this result does not quite reach the threshold of significance.
■
DISCUSSION Student scores increased on both the gas chromatography of alcohols and the photosynthesis and respiration postquiz when compared to the prequiz (Figures 3 and 4). These results indicate gains in student learning after the laboratory activity. During both years of the study, the changes in student scores were highly significant, as indicated by p values far below 0.001. While these results indicated student scores increased overall, changes in answers to individual questions were also of interest. To assess learning at the level of individual questions, the percentage of correct answers was tabulated for each question before and after the activity and subjected to a paired t-test. The gas chromatography of alcohols 2007−2008 data did not show a significant difference, but the 2008−2009 data did show a significant increase in score for each quiz question. The significant increase in the second year was examined. Because the same teacher, in the same school (multiple class periods), participated in this study both years, an assumption could be made that the teacher prepared the students similarly both years before the activity. However, Table 1 shows that students in the second year did not score as well on the prequiz as students in the first year. Postquiz scores were similar for both years. This suggests the students participating the second year were not as well prepared for the activity as students during the first year of the study. As would be expected, student preparation can affect prequiz scores. The high postquiz scores in both years indicate that regardless of level of preparation, students who performed the activity demonstrated acquisition of knowledge on the postquiz. The results of the paired t-test of the 2007−2008 photosynthesis and respiration data showed a significant difference between the pre- and postquiz scores (p = 0.038). However, the same analysis of the 2008−2009 data did not quite show a significant difference (p = 0.066). The source of this difference between the two years of data was examined. During the first year of the study (2007−2008), students from two schools participated in the photosynthesis and respiration activity. During the second year of the study (2008−2009), a third school was added. To determine whether the new school’s data accounted for the change, that data set was removed and only data sets from schools participating both years were analyzed (Table 3). A paired t-test of the 2008− 2009 data omitting the new school (154 students participated)
■
AUTHOR INFORMATION
Corresponding Author
*E-mail:
[email protected]. Notes
The authors declare no competing financial interest.
■
ACKNOWLEDGMENTS Funding for this study was provided by The Commonwealth of Pennsylvania through a grant to fund Science in Motion programs. Support was also provided by Susquehanna University. The author also wishes to acknowledge Wade Johnson for critical reading of the manuscript.
■
REFERENCES
(1) National Center for Education Statistics. The Nation’s Report Card: Science 2009 (NCES 2011-451); Institute of Education Sciences, U.S. Department of Education: Washington, DC, 2011. (2) PISA 2009 Results: What Students Know and Can DoStudent Performance in Reading, Mathematics and Science (Volume I); OECD, Programme for International Student Assessment: Paris, France, 2010. http://dx.doi.org/10.1787/9789264091450-en (accessed Jun 2012). (3) Spencer, J. N. J. Chem. Educ. 1999, 76 (4), 566−569. (4) Deslauriers, L.; Schelew, E.; Wieman, C. Science 2011, 332 (6031), 862−864. (5) National Committee on Science Education Standards and Assessment, and National Research Council. National Science
1262
dx.doi.org/10.1021/ed200320g | J. Chem. Educ. 2012, 89, 1259−1263
Journal of Chemical Education
Article
Education Standards; National Academies Press: Washington, DC, 1995. (6) Hermens, R. A.; Mitchell, D. J. J. Chem. Educ. 1995, 72 (2), 166− 167. (7) Felix, D. A.; Hertle, M. D.; Conley, J. G.; Washington, L. B.; Bruns, P. J. Cell Biol. Educ. 2004, 3, 189−195. (8) Juniata College Science in Motion Lab Activity Using Gas Chromatography: Separation of Alcohols. http://www.juniata.edu/ services/ScienceInMotion/chem/gas%20chromatography.html (accessed Jun 2012). (9) Masterman, D.; Redding, K. Biology with Vernier: Biology Experiments Using Vernier Sensors; Vernier Software and Technology: Beaverton, OR, 2007.
1263
dx.doi.org/10.1021/ed200320g | J. Chem. Educ. 2012, 89, 1259−1263
