Article pubs.acs.org/jchemeduc
Positive Impacts Using POGIL in Organic Chemistry Sara M. Hein* Department of Chemistry, Winona State University, Winona, Minnesota 55987, United States ABSTRACT: A student-centered learning technique, process-oriented, guided-inquiry learning (POGIL), has been developed as a pedagogical technique that facilitates collaborative and cooperative learning in the chemistry classroom. With the use of this technique, students enhance their higher-order thinking skills and process skills synergistically. In addition, they develop positive relationships with other students in the course. POGIL was recently implemented at a mid-sized, comprehensive public institution and used in the organic chemistry sequence. Comparisons of the ACS exam percentile rankings and incoming proficiency (ACS scores and grade point averages) data were made to determine the extent of the effect that POGIL had on student learning when compared to students who had been taught using traditional methods. Overall, the data provide evidence to suggest that students learning by the POGIL method have a greater grasp of content knowledge than students who learned by the traditional lecture approach, as evidenced by higher final exam scores for POGIL students. The POGIL experience positively impacted students of all levels of proficiency. Difficulties associated with the implementation and perceptions of reform-based learning methods are addressed. KEYWORDS: Second-Year Undergraduate, Organic Chemistry, Collaborative/Cooperative Learning, Inquiry-Based/Discovery Learning FEATURE: Chemical Education Research POGIL method performed at least as well on the final exam as students taught using a traditional lecture method.
N
umerous articles refer to the use of collaborative and cooperative learning methods for teaching chemistry (e.g., ref 1a−c). Discussions and studies that focus on developing student-centered learning experiences have led to transformations of teaching styles away from the traditional lecture format. Most transformation efforts have been carried out in general chemistry.2 Few discussions address the use of collaborative learning in the organic chemistry classroom.3 Process-oriented, guided-inquiry learning (POGIL),4 is a student-centered learning method in which student teams discover content to assemble concepts in the classroom. Through this method, students work on activities in groups to explore concepts by first examining data, or examples, to develop mental constructs that can later be applied to problems. This constructivist approach enables students to develop problem-solving skills, and subsequently, higher-order thinking skills.5 Additionally, it is believed that students who learn by this method develop process skills that foster their maturity in communication, written expression, and problemsolving, which are highly desirable traits for careers. Methods of discovery have been shown to develop confidence and deeper levels of understanding.6 At Winona State University (WSU), a midsized, comprehensive public institution located in southeast Minnesota, the transformation from a traditional lecture format to a studentcentered learning approach was undertaken to study the impact such a method would have on student assessments in a full-year organic chemistry sequence. The POGIL method was chosen based on recent studies of initial results and student perceptions.7 Although this guided-inquiry method has shown promise, no definitive evidence has been presented to indicate a positive effect on assessments carried out in organic courses. This study presents evidence that students who learn by the © 2012 American Chemical Society and Division of Chemical Education, Inc.
■
METHODS Approximately 11% of the student population at WSU are declared science majors. Over the last several years, enrollment in the yearlong sequence of organic chemistry has grown from three sections (72 students) to more than five sections (124 students), necessitating two instructors for teaching them. The average ACT score for incoming students at WSU is 22.5; approximately 50% of the students enrolled in the first semester of organic chemistry, Principles of Organic Chemistry I, are female. The majority of the students enrolled in this course have taken both semesters of general chemistry at WSU. All general chemistry sections were taught using traditional lecture methods. Three or four instructors have taught the general chemistry sequence each year of this study using the latest edition of Brown, LeMay, and Bursten’s Chemistry, The Central Science.8 Chapters 1−20 are covered consistently among the instructors. To evaluate the effectiveness of POGIL on concept retention and subsequently, cumulative organic chemistry knowledge, student final exam scores based on ACS national standardized exam percentiles in organic chemistry were studied. A traditional lecture-type group (years 1−3) and a second group taught using the POGIL method (years 4−6) were used in comparisons. Both the traditional and POGIL methods were employed for two semesters to include the topics generally taught in a typical second-year organic chemistry course. Students met for three 50-min periods over a span of 15 weeks for each of two semesters after which the ACS Published: April 17, 2012 860
dx.doi.org/10.1021/ed100217v | J. Chem. Educ. 2012, 89, 860−864
Journal of Chemical Education
Article
Standardized Organic Exam was taken during the final exam week. The 2002 ACS form was used for the first four years of the study; after the 2002 form was retired, the 2008 ACS form was used. The same instructor taught each of the courses from which data are presented for yearlong analyses. The instructor taught the course for three years (2003−2005) using a traditional lecture-style format. Then, the instructor implemented the POGIL method for three years (2008−2010). The distribution of scores for each of the six years was compared. Exam scores for 2006 and 2007 were not examined because the instructor was not assigned to the two-semester organic chemistry sequence. For each year of instruction, the same textbook was used for both semesters and students took the second semester of organic chemistry immediately after taking the first semester. During the first week of classes, the instructor communicated to the students the teaching methodology to be used for the courses. The traditional lecture group and the POGIL group were assessed on the same criteria: online and written homework, laboratory reports, midsemester exams, and the ACS final exam. To encourage students’ participation, a minor (5%) component was included in student grades in the treatment group. Developers of the method indicate that accountability within a group translates to better whole-group learning effects.9 In the traditional lecture groups, lecture periods were interjected with short, 1−2 min problem-solving sessions. Three to four problems were presented to the students each class period to work on individually. Then students were encouraged to compare answers and discuss differences. Students were provided with sets of incomplete lecture slides before the class periods so that students had them available to take notes on. Students enrolled in the POGIL sections met for the same amount of time and frequency as students who were taught in traditional lectures. In the POGIL sections, a successful former student, trained in tutoring and supplemental instruction (SI), was present during the class periods and available for guidance. In POGIL classes, students worked through activities from a published POGIL textbook.10 During the activities, students worked in groups of four. Answers, including how they were solved, were transcribed on a sheet that was collected at the end of the period. Students were observed and assisted by both the instructor and SI student. Points of confusion within the groups were clarified through the use of leading questions and verification of record taking during the class period. Activities were paused intermittently to gain a sense of class understanding at various points through the use of questioning and presenter examples to the entire population. The instructor also halted work when several groups were having difficulty with the material to ask questions and clarify the material presented. The following class period commenced with clarification of difficult or misunderstood critical thinking questions determined by the instructor found after reviewing the record sheets from the previous activity. Activities were chosen to accompany the sequence of chapters that were presented from a typical secondyear organic textbook. The chapters and topics covered in both traditional lecture sections and POGIL sections were the same except that fewer details were explored in the POGIL sections. The populations for each of these lecture periods included between 30 and 88 students; they were dispersed based on the number of lab sections (24/section) that were available. Data concerning the aptitude of incoming students were compared
using ACT scores and grade point averages (GPAs), as available, provided by the Office of Assessment and Institutional Research. These data are presented in Table 1. GPA Table 1. Comparison of Average ACT Scores and Mean Entering Grade Point Averagesa Class Method Traditional, 2003 2004 2005 POGIL, 2008 2009 2010 a
Average ACT Total Score (n) 24.7 24.8 23.6 24.1 24.7 23.1
(56) (65) (31) (50) (28) (34)
Mean Incoming Students’ GPA (n) 3.25 3.21 3.32 3.21 3.36 2.99
(75) (71) (36) (35) (24) (27)
Values include only students taking final after both semesters.
values were considered to gain a sense of the students’ capabilities in college courses. Because students entering organic chemistry courses had generally completed a minimum of 24 credits before entering the sequence, it was presumed that an analysis of cumulative GPAs would reveal students’ adjustment to college in addition to their proficiency. The difference in n reflects that not all data points were available for every student. During both first and second semesters, students in the treatment group and control group took four exams created by the instructor. The ACS standardized organic chemistry exam was given at the end of the second semester. Students were allowed to drop their lowest exam score each of the semesters. Because the midterm exams given each year were different, the scores could not be compared. However, national percentile rankings on the ACS final exam were compared. Course grades were not compared because of the integrated nature of the laboratory and lecture components of the course. Table 2 provides a comparison of the mean ACS scores and percentiles, including students for whom no ACT or GPA data were available. Table 2. Comparison of ACS Standardized Organic Exam Scores and Class Method Used Instruction Method Traditional Lecture POGIL
Exam Years
ACS Exam Form
Mean Score (SD)
Mean Percentile
2003−2005
2002
35.1 (10.2)
29.0
2008−2010
2002 2008
35.3 (10.4) 37.0 (11.6)
30.5 44.1
Only data for students who completed both semesters of organic chemistry in the same school year were used in the analysis. Of the students that did complete the course, 158 were in the traditional lecture style comparison group; 103 students were included in the POGIL experience for both semesters.
■
RESULTS AND DISCUSSION To test the effect of teaching method on ACS Organic Exam percentile scores, tests for homogeneity of variance were carried out using both incoming student GPAs and ACT scores across all years of study. No significant difference was found (Table 1) in average ACT scores (p = 0.6 using a Wilcoxon’s rank sum test) or incoming student GPA scores (p = 0.9) for any of the years. Given that the groups were not statistically different, further investigations were made using both parametric and 861
dx.doi.org/10.1021/ed100217v | J. Chem. Educ. 2012, 89, 860−864
Journal of Chemical Education
Article
regression analysis with the aforementioned adjustments. Although these latter findings are not significant, they do trend toward better student performance on the final exam. Comparison of the national percentile rankings on the ACS Organic Chemistry Exam between the treatment group and the control group provides a different perspective about the effectiveness of the POGIL method in the classroom. As seen in Figure 1, the key difference between the two teaching styles
nonparametric methods to determine whether a correlation exists between ranking and the teaching method. It was expected that students’ incoming aptitude would impact student percentile ranking, as well as the teaching method experienced. To investigate, a Pearson’s correlation analysis was carried out. ACS percentile rankings significantly correlated with incoming ACT scores (p = 0.005) and GPA values (p < 0.0001). Therefore, this information alone does not support the theory that students taught by the POGIL method performed better than their traditional lecture counterparts. These findings are consistent with what has been previously reported.11 To focus on the effect of the teaching method (traditional lecture or POGIL) alone on student percentile rankings, a multiple linear regression analysis was carried out after controlling for the variance in percentile rankings owing to students’ GPAs and ACT scores. Even though both GPA and ACT scores were considered, it was noted that GPA scores were more highly correlated with the ACS percentile ranking. Also, GPA and ACT scores are significantly correlated (p < 0.0001), which indicates that these variables may contribute overlapping information. Therefore, only GPA scores were considered as a covariate in the model. The treatment was statistically significant (b = 10.82, t(160) = 2.973, p = 0.003). These results suggest that students in the treatment (POGIL) group would be expected to score 10.82 percentage points higher than students in the control (traditional) group. An effect size of 0.42 was observed. A nonparametric analysis (Mann−Whitney test) was also used. Significance of the same magnitude (p = 0.003) was observed. However, this investigation did include a caveat: the change in instrument (transition from the ACS Organic Exam 2002 to ACS Organic Exam 2008) over the study period presented a challenge to interpreting the extent of the impact of teaching method on student percentile rankings. The change in instrument made it impossible to isolate the effect of teaching method alone. This confounding factor is not accounted for the aforementioned analysis. Therefore, further statistical analysis, which included incorporation of the change in instrument, was carried out. Table 2 includes three pools of data (Traditional 2002, POGIL 2002, and POGIL 2008) that could be considered. These data were used separately to carry out a second multiple linear regression analysis to predict the ACS percentile rankings based on teaching method and year while adjusting for the quadratic relationship between students’ GPA and the percentile rankings. The treatment was statistically significant (p = 0.0005). The results indicate that after adjusting for GPA, the average ACS percentile ranking for students in the POGIL 2008 group was 15.29 percentage points higher than that of students in the POGIL 2002 group and 14.01 percentage points higher than that of students in the Traditional 2002 group. There was no evidence for a difference in the average ACS percentile ranking between Traditional 2002 and POGIL 2002. An additional analysis of the teaching method variable with exam form was carried out using only the third year of control (Traditional 2002) with the first year of treatment (POGIL 2002) as it was assumed that a closer time period would provide the most consistent data in terms of teaching maturity and incoming student proficiency. During these years, ACS national percentile means (incoming students’ GPAs) were 26.6 (3.32) and 30.5 (3.21), respectively. This treatment was not statistically significant when analyzed using a multiple linear
Figure 1. ACS Exam distribution for students taught by the same instructor for both semesters.
(2003, 2004, and 2005 vs 2008, 2009, and 2010) is that the number of students ranking in the 25th percentile and below showed a meaningful decrease over each year the POGIL method was used. A greater number of students were able to score in the 26th−100th percentile when learning by the POGIL method than when they were taught by using traditional lecture methods. The impact of the method is magnified by the upper-quartile increase, especially after the 2008 Form was introduced. Further, a larger number of students scored in the upper quartile, even in the third year when the treatment group had a lower incoming average GPA. These data provide evidence to suggest that students learning by the POGIL method have a greater grasp of the content knowledge than students who learned by the traditional lecture approach and subsequently were able to perform better on the year-end assessment. A more pronounced effect can be observed when comparing the average percentile rankings of students in the control group (traditional lecture) and the treatment group (POGIL). Table 3 shows both the mean and median data of both teaching methods over the entire statistical period. (The n value includes students for whom no ACT or GPA data was available in Table 1.) This information shows that POGIL students ranked higher in both median and mean. Further, analysis indicates that 72% Table 3. Distribution of ACS National Percentiles
862
Parameters
Traditional
POGIL
Mean national percentile ranking Median national percentile ranking n n above traditional method mean Percentage of students above traditional method mean Top 10% mean
29.0% 20.0% 158 82%
39.2% 36.0% 103 74 72% 89%
dx.doi.org/10.1021/ed100217v | J. Chem. Educ. 2012, 89, 860−864
Journal of Chemical Education
Article
Perhaps an area of concern in the implementation of POGIL has been that the method caters to lower-achieving students. Findings show that fewer students earn lower grades in chemistry courses; more students are able to earn aboveaverage or average grades.13 This makes sense as lowerachieving students stand to gain the most from such an approach. Further, little evidence suggests that reform-based learning methods help good students as it can be presumed that excellent students will find ways to perform well no matter which method they are taught by. However, the data in Table 3 refute this argument. The highest achieving 10% of the students in the treatment group outperformed the top 10% in the control group. These data suggest that POGIL helps all types of students do better on the ACS Organic Chemistry Exam.
of students taught by the POGIL method achieved higher than the median achieved by students in the control group. This means that more than half of the students in the treatment group were able to score higher on the nationally standardized ACS organic chemistry exam than the students who were in the control group. Crucially, greater than 20% (total 72%) more students taught by the POGIL method achieved better than the students taught by the traditional lecture method. It should be noted that, when a similar analysis was carried out between the third year of lecture and the first year of POGIL, the results were only slightly higher (51% achieved higher). Overall, the data indicate that students performed very well on the 2008 ACS Organic Chemistry Exam, which could be attributed to the success of the POGIL method. The pronounced difference between the first and second years using the POGIL method could be attributed to any combination of change in test form, self-selection by students, or maturity of teaching using the method by the instructor. Additionally, the comfort level of the transition may have also played a role in the success of students in subsequent years.
■
CONCLUSION The implementation of the POGIL method in the organic chemistry classroom has been shown to positively impact student proficiency on nationally standardized ACS organic chemistry exams. The extent to which the positive impact has been made is quantified by the number of students who were able to score in the three predetermined categories of percentile rankings, as well as the average scores on the national ACS Organic Chemistry Exam. A significant p-value in the Wilcoxon Rank Sum Test indicates a direct relationship between the type of teaching method and student achievement. Paired correlations between ACT scores and GPAs were controlled to determine the extent of the effect of the teaching method used. An effect size of 0.42 was observed. As a result, POGIL students are likely to gain percentage points on the final exam when compared to traditionally taught counterparts. The extent of the gain is best described by the overall analysis of the ACS Organic Chemistry Exam percentile rankings over the entire period of the study.
■
TRADITIONAL LECTURE TO POGIL TRANSITION The decision to transition to a student-centered learning method using the POGIL method was made to offer a better learning environment for organic chemistry students. Moreover, it was implemented by fully immersing students: they were expected to learn by this method on a regular basis. As a result, this kind of reform-based instruction was met with resistance by some students. Anecdotally, unsolicited comments from some of the students during the first year indicated that they did not find the POGIL experience as valuable as other traditional methods. They expressed frustration with having to construct knowledge on their own rather than be “told” a professor’s understanding of the material. For example, one student indicated that he was required to “learn the material on his own” rather than have it taught to him. However, as time progressed, fewer negative remarks were observed. To better understand the relationship between the POGIL experience and student performance, attrition levels were explored between the comparison groups (Table 4). The data
■
*E-mail:
[email protected].
■
ACKNOWLEDGMENTS The author thanks WSU’s Chemistry Department for invaluable assistance in suggestions for approaches to dissemination of these data. Statistical analysis guidance by Tisha Hooks is also appreciated.
Table 4. Attrition Levels for Comparison Groups Semester
Attrition for Traditional Lecture, %
Attrition for POGIL, %
Fall Spring
15.2 3.7
10.8 4.9
AUTHOR INFORMATION
Corresponding Author
■
REFERENCES
(1) (a) Bowen, C. W. J. Chem. Educ. 2000, 77, 116−119. (b) Cooper, M. M.; Cox, C. T., Jr.; Namouz, M.; Case, E. J. Chem. Educ. 2008, 85, 866−872. (c) Hake, R. Am. J. Phys. 1998, 66 (1), 64−74. (2) (a) Lewis, J. E.; Lewis, S. E. J. Chem. Educ. 2005, 82, 135−139. (b) Doymus, K. J. Chem. Educ. 2007, 84, 1857−1860. (c) Lyon, D. C.; Lagowski, J. J. J. Chem. Educ. 2008, 85, 1571−1576. (3) (a) Gosser, D. K., Jr.; Roth, V. J. Chem. Educ. 1998, 75, 185−187. (b) Gosser, D.; Roth, V.; Gafney, L.; Kampmeier, J.; Strozak, V.; Varma-Nelson, P.; Radel, S.; Weiner, M. Chem. Educator [Online] 1996, 1, 1−17; DOI: 10.1333/s00897960002a. (c) Bradley, A. Z.; Ulrich, S. M.; Jones, M. J., Jr.; Jones, S. M. J. Chem. Educ. 2002, 79, 514−519. (d) Dinan, F. J.; Frydrychowski, V. A. J. Chem. Educ. 1995, 72, 429−431. (e) Dougherty, R. C. J. Chem. Educ. 1997, 74, 722−726. (4) (a) Farrell, J. J.; Moog, R. S.; Spencer, J. N. J. Chem. Educ. 1999, 75, 570−574. (b) Moog, R. S.; Spencer, J. N. Process Oriented Guided Inquiry Learning (POGIL); American Chemical Society: Washington DC, 2008.
indicate that the student drop rate was not dependent on the type of teaching method used. In fact, the attrition rates remained fairly constant throughout the six years that were compared. Therefore, the initial dissension communicated by the students was determined to be illustrative of the reactions of students who experience reformed teaching methods for the first time. The implementation of such efforts takes time and there can be struggles with student perceptions. McGinnis12 describes the effects of student attitudes and beliefs toward reformed teaching methods in science and mathematics courses based on students’ initial disposition toward the subject taught. Once students understand their role in the classroom, they accept the reformed method more easily. The implementation of the POGIL method in the organic chemistry sequence at WSU followed this progression. 863
dx.doi.org/10.1021/ed100217v | J. Chem. Educ. 2012, 89, 860−864
Journal of Chemical Education
Article
(5) Bloom, B. S.; Engelhart, M. D.; Furst, E. J.; Hill, W. H.; Krathwohl, D. R. Taxonomy of Educational Objectives: Classification of Educational Goals, I. Cognitive Domain; David McKay Company: New York, 1956. (6) Abraham, M. R. Inquiry and the Learning Cycle Approach. In Chemists’ Guide to Effective Teaching; Pienta, N. J., Cooper, M. M., Greenbowe, T. J., Eds.; Pearson-Prentice Hall: Upper Saddle River, NJ, 2005; pp 41−52. (7) (a) Moog, R. S. The POGIL Project: Process-Oriented, GuidedInquiry Learning through the Chemistry Curriculum. Presented at the 18th Biennial Conference on Chemical Education, Ames, IA, 2004. (b) Ruder, S. M.; Hunnicutt, S. S. POGIL in Chemistry Courses at a Large Urban University: A Case Study. In Process Oriented Guided Inquiry Learning (POGIL); Moog, R. S., Spencer, J. N., Eds.; American Chemical Society: Washington, DC, 2008; pp 133−147; (c) Lewis, S. E.; Lewis, J. E. J. Chem. Educ. 2005, 82, 135−139. (8) Brown, T. L.; LeMay, H. E.; Bursten, B. E.; Murphy, C. J.; Woodward, P. M. Chemistry, The Central Science, 11th ed.; Prentice Hall: Upper Saddle River, NJ, 2006. (9) Hanson, D. Instructor’s Guide to Process-Oriented Guided-Inquiry Learning; Pacific Crest: Lisle, IL, 2006. (10) Straumanis, A. R. Organic Chemistry: A Guided Inquiry; Houghton Mifflin: Boston, MA, 2004 and 2009. (11) Straumanis, A. R. College of Charleston, Charleston, South Carolina. Unpublished work, 2009. (12) McGinnis, J. R.; Kramer, S.; Shama, G.; Graeber, A. O.; Parker, C. A.; Watanabe, T. J. Res. Sci. Teach. 2002, 39, 713−737. (13) Spencer, J. N. J. Chem. Educ. 1999, 76 (4), 566−569.
■
NOTE ADDED AFTER ASAP PUBLICATION This paper was published to the Web with a mistake in Figure 1 on April 17, 2012. This was corrected in the version published on May 15, 2012.
864
dx.doi.org/10.1021/ed100217v | J. Chem. Educ. 2012, 89, 860−864
