Research: Science and Education Chemical Education Research
A Quantitative Literature Review of Cooperative Learning Effects on High School and College Chemistry Achievement Craig W. Bowen Department of Chemistry, Clemson University, Hunter Laboratories, Clemson, SC 29634-1905;
[email protected] Several reviews are available on the effects of cooperative learning on student learning at the K–12 level in general and in science and mathematics (1–3). In addition, there are reviews about effects of cooperative learning in college populations (4). However, no quantitative review has been done of cooperative learning in high school and college chemistry courses. The purpose of this paper is twofold. First, a brief overview of metaanalysis (a quantitative approach to conducting literature reviews) is given. The second purpose is to illustrate the power of this technique by reporting quantitative effects of cooperative learning on chemistry achievement in high school and college classes. Background on Cooperative Learning Before considering meta-analysis and how to conduct quantitative summaries of articles on data-based research and evaluation, a short overview is needed to orient readers unfamiliar with components of cooperative learning strategies because this will be the focal area for the meta-analyses reported later. This Journal has published several articles on how to incorporate cooperative learning into chemistry lecture and laboratory classrooms (5–11) and there are other available resources as well, such as those compiled by Nurrenbern and Krupp (12) and a brief review provided by Nurrenbern and Robinson (13). However, most of these cite a key reference that highlights components of cooperative learning in its different forms. Johnson, Johnson, and Smith (4 ) point to five important components of cooperative learning that should be included in activities: 1. POSITIVE INTERDEPENDENCE Students are given tasks that they perceive being able to complete only if all group members contribute to the effort. This can be achieved through several design approaches: (i) providing a group reward for successful interdependence; (ii) having activities in which resources are shared; or (iii) providing a task that is too difficult for students to do individually. 2. FACE-TO-FACE INTERACTION Students are given time and space as part of the activity for meeting with group members and providing assistance with the learning task at hand. 3. INDIVIDUAL ACCOUNTABILITY Students are required to learn the material at hand and demonstrate that they have mastered it. The group should facilitate the learning of all group members, but each group member needs to be responsible for demonstrating his or her own learning. 4. INTERPERSONAL SKILLS Students are given opportunities to practice various groups skills such as developing trust, communicating
116
effectively, and handling conflicts. Feedback should be provided so students can enhance these skills. 5. GROUP PROCESSING Students are given time and space to reflect on the processes that took place in their group that facilitated (and stymied) achieving the goals of the group. The focus is to learn about group dynamics for future situations.
This set of components is useful to consider when reading the articles cited in this paper, in which cooperative learning strategies are used in chemistry classes. Background on Meta-Analysis A meta-analysis is a quantitative approach to reviewing research literature in a specific area. In educational research, the many factors that vary from one teaching context to another make it difficult to design definitive experiments to determine the extent to which a given instructional approach affects a given student outcome. A meta-analysis combines a number of studies (usually conducted by different researchers in a variety of educational contexts) to quantify the effect an instructional approach has on a given outcome. By broadening the pool of data to include different contexts and to increase sample sizes, a better quantitative estimate can be made of how an instructional practice affects student outcomes. Technical details on conducting meta-analyses can be found elsewhere (14–16 ). However, the process usually includes these general steps: 1. Describing the independent variables (e.g., cooperative learning as a teaching method) and outcome variables (e.g., academic achievement) of interest. 2. Identifying quantitative research studies that address relations between independent and outcome variables. 3. Tabulating quantitative information from each study that indicates the effect the independent variable has on the outcome variable. 4. Determining the effect size for the data reported in the study so the data are “normalized”. The effect size is the difference between the means of the outcome scores of the experimental and control groups divided by the standard deviation of the scores of the control group (although sometimes a pooled standard deviation from all scores is used). A positive effect of the instructional strategy on the outcome variable is indicated by a mean effect size across the studies that is greater than zero.
Effect size =
Meantreatment group – Meancontrol group Standard deviationcontrol group
Journal of Chemical Education • Vol. 77 No. 1 January 2000 • JChemEd.chem.wisc.edu
Research: Science and Education
One caveat when estimating effect size of an instructional treatment is that studies that show no effect are less likely to be published in the literature. In addition, many studies fail to report all the information needed to calculate effect size (e.g., only reporting mean scores). Different formulas from the one shown here may be used if different quantitative information is reported in a study (e.g., χ2 values, t-test results, or F-test results from analyses of variance). Because an effect size shows how far the mean outcome variable of one treatment group is above or below the mean outcome variable of another group in terms of number of standard deviations, the percentile difference in performance in the two groups can be determined by looking at standard normal scores (or Z scores) found in most applied statistics texts. An effect size of zero is the same as a Z-score of zero. Table 1. Effect Sizes and This means that a student Percentile Changes between performing at the 50th perTreatment and Control Groups centile in one group is pera Effect Size Percentile Change forming at the same per0.00 50 centile in the other group. 0.20 58 With an effect size of 0.20, 0.40 66 a student at the 50th per0.60 73 centile in the treatment group is performing 0.20 0.80 79 standard deviations higher 1.00 84 than a student at the 50th 1.20 88 percentile in the control 1.40 92 group. By looking up a Z1.60 95 score of 0.20, we see that 1.80 96 the student in the treataA student performing at the 50th ment group is performing percentile of the treatment group is at the 58th percentile of the performing at this percentile of the control group. Table 1 gives control group. effect size and percentile change information for selected values; more extensive values can be found in most applied statistics books. Because the standard normal distribution is symmetric, negative effect sizes are of the same absolute size change in percentiles, but in the opposite direction. Using effect sizes is a way to “normalize” different outcome measures so quantitative comparisons can be made. For example, consider two different studies examining the effects of cooperative learning on different aspects of chemistry achievement. Lonning (27 ) studied high school students (n = 36) taking a physical science course who were assigned to either a control group or a treatment group being taught the particulate nature of matter over a four-week period. Both groups of students were encouraged to work in groups as they learned various chemical concepts regarding the particulate nature of matter. However, students in the experimental group were also taught collaborative skills and had both group and individual components to their grade. Outcomes were measured by a test of conceptual understanding about the particulate nature of matter. These data are summarized in Table 2. In a second study examining how cooperative learning affects chemistry achievement, Dinan and Frydrychowski (25) worked with students in organic chemistry. College students (n = 36) in a first-term organic chemistry course were assigned to team-learning groups for the semester. The teams were re-
quired to read material before lecture and were then given an individual minitest followed by a group minitest. Responses were graded immediately and a lecture was given on material not well understood. Comparison of the students’ achievement on the final exam with previous cohort performance (traditional lecture classes) showed that students achieved more in the team group. Their data are also summarized in Table 2. The mean scores from these two studies cannot be directly compared across the treatment groups because different achievement measurements were used and they had a different range of possible scores. However, by calculating the effect size from each study, a quantitative summary of the overall effect can be estimated even though different ways of measuring the outcome were used. In both studies, cooperative learning had a positive effect on chemistry achievement. Summary of Meta-Analysis of Cooperative Learning Effects on Outcomes in College-Level SMET Courses A review and meta-analysis was conducted to answer the question “Is cooperative learning more effective than traditional instruction in promoting academic achievement, persistence, and attitudes among undergraduates in science, mathematics, engineering, and technology [SMET]courses?” (17 ). The authors used these criteria for including research studies in the analysis: 1. The target instructional population was undergraduates in SMET courses. 2. The instructional treatment took place in a classroom, incorporated one or more components of cooperative learning (described earlier), and included individual accountability. 3. Experimental and control groups were used in the study, and sufficient statistical information was provided to generate the data needed for the meta-analysis (e.g., means and standard deviations are reported). 4. The research was reported in English between 1980 and 1996 (journal articles, conference proceedings, dissertations, and unpublished manuscripts).
Using various abstracting services (e.g., in education and psychology), a total of 37 research studies were identified that met these criteria for examining how cooperative learning impacts achievement. Of these studies, four involved chemistry courses (18–21). Nine studies contained persistence data (none in chemistry), and 11 studies contained attitudinal data (one in chemistry).
Achievement Outcomes A summary of the analysis of the studies, which involved almost 3,500 students, indicates that cooperative learning (in its varied forms) has a significant and positive effect on Table 2. Statistics on Chemistr y Achievement (End-of-Term Exam) from Two Studies Effect Size
Study
Treatment Group
Mean (SD)
Lonning (27 )
Cooperative learning Individual learning
2.29 (1.21) 1.42 (1.22)
0.71
Dinan and Cooperative learning Frydrychowski (25) Individual learning
70.8 (10.2) 67.1 (14.2)
0.26
JChemEd.chem.wisc.edu • Vol. 77 No. 1 January 2000 • Journal of Chemical Education
117
Research: Science and Education
Figure 1. Frequency distribution of 49 effect sizes from 37 research studies on cooperative learning in college-level SMET courses.
Figure 2. Frequency distribution of 30 effect sizes from 15 research studies on cooperative learning in college and high school chemistry courses.
achievement-related outcomes of college students in SMET courses. From these 37 studies a total of 49 effect sizes were calculated (some studies measured multiple outcomes) for achievement outcomes. Figure 1 shows the distribution of these achievement effect sizes. The mean effect size is 0.51 with a standard deviation of 0.35. Because it shows a positive effect, the data support the assertion that cooperative learning can significantly enhance student achievement in undergraduate SMET courses. In practical terms, an effect size of 0.51 means that median student performance is increased from the 50th percentile in the group taught by traditional methods to the 70th percentile in the cooperative learning group.
chemistry in these journals: Journal of Chemical Education, Science Education, International Journal of Science Education, and Journal of Research in Science Teaching. This was done because these journals tend to publish data-based articles that can be used in meta-analysis and because abstracts do not always indicate the educational treatment (in this case cooperative learning) that is applied in a study. A total of 437 high school and almost 1,100 college students participated in the 15 chemistry studies that were identified (11 more chemistry studies than were included in the previous meta-analysis). Table 3 shows the effect sizes of cooperative learning on chemistry achievement (as defined by each study) associated with these chemistry-related studies, as well as the grade level, content area, and number of students. Figure 2 shows the effect-size outcomes in a graphic form. These research reports on cooperative learning in chemistry courses show a mean effect size of 0.37 (and a standard deviation of 0.39) across the 30 effect sizes reported in the 15 identified chemistry studies. This indicates that while median student performance in a traditional course is at the 50th percentile, the median student performance in a cooperative learning environment is 14 percentile points higher.
Persistence and Attitude Outcomes A summary of the 9 studies that collected persistence data and 11 studies that reported attitudinal data indicates that cooperative learning also has a significant and positive effect on student attitudes towards SMET courses. The authors report that persistence for continued study in SMET courses of students taught with cooperative learning approaches was 22% greater than persistence of students taught by traditional approaches. Students in cooperative learning classes also had more positive attitudes toward their classes. Meta-Analysis of Cooperative Learning Effects on Achievement in High School– and College-Level Chemistry Courses Although the meta-analysis reported in the previous section involved some college-level chemistry students, an additional meta-analysis is reported here. It focuses on high school and college chemistry courses using cooperative learning. In addition to the four chemistry studies identified in the SMET meta-analysis, other college-level studies were found, and studies involving high school students were also included. Identification of these studies was not limited to abstracting services; the author located additional articles by scanning articles that might involve cooperative learning in 118
Conclusion This paper has briefly introduced a quantitative approach to summarizing research results. Meta-analysis can provide quantitative estimates of the effects of an instructional treatment on an outcome variable. It can be used as a starting point to examine influences of our instructional practices on chemistry learning. For example, additional meta-analyses could address questions like these: What effects does computer-based instruction have on student attitudes toward and learning of chemistry? To what extent does the use of demonstrations help students learn or enjoy chemistry?
Journal of Chemical Education • Vol. 77 No. 1 January 2000 • JChemEd.chem.wisc.edu
Research: Science and Education Table 3. Achievement Effect Sizes in Chemistr y Cooperative Learning Studies Author (Ref)
Level
Course
Students (No.)
Effect Size ᎑0.63 (test 1) 0.30 (test 2) 0.34 (test 3)
Banerjee & Vidyapati (22)
College
General
68
Basili & Sanford (20)
College
General
62
0.78 (matter) 0.67 (energy) 0.49 (gases) 0.76 (liquids) 0.93 (solids)
Bowen & Phelps (23)
College
General
67
0.95
Burron, James, & Ambrosio (24)
College
General
51
0.11 (achieve.) 0.12 (lab final)
Dinan & Frydrychowski (25) College
Organic
103
0.26
Dougherty (26)
College
Organic
260
0.28
Lonning (27)
High school General
36
0.71
Lundeberg (18)
College
General
148
0.61
Metz (28)
College
General
125
Niaz (29)
College
General
72
Okebukola (30)
High school General
Ross & Fulton (31)
College
Smith, Hinckley, & Volk (19) College Springer (21) Tingle & Good (32)
0.18 (test 1) 0.05 (test 2) ᎑0.24 (test 3) ᎑0.16 (test 4) ᎑0.22 (final) 0.43 (prob. 1) 0.64 (prob. 2) 1.01 (prob. 3) 0.03 (prob. 4) 0.45 (prob. 5)
223
0.59
Analytical
65
0.26
General
52
0.72
Not available for review High school General
178
0.51 0.07
Once general questions such as these have been answered, additional research can probe aspects of computer-based instruction, or demonstration use, that have the greatest impact on student learning. Of course, people conducting primary research or evaluation studies should report sufficient statistical data so that their work might be included in possible future meta-analyses. To illustrate how meta-analyses are conducted, a topic of interest to the chemical education community was selected. The meta-analysis reported here shows that, on average, using aspects of cooperative learning can enhance chemistry achievement for high school and college students. On the basis of the results of this meta-analysis, it is strongly recommended that chemistry instructors continue incorporating cooperative learning practices into their classes. Results such as these can be used to support efforts at curriculum change that instructors might make in their teaching situations. Literature Cited 1. Slavin, R. E. Psychol. Bull. 1983, 94, 429–445. 2. Johnson, D. W.; Johnson, R. T.; Maruyama, G. Rev. Educ. Res. 1983, 53, 5–54. 3. Slavin, R. E. Cooperative Learning: Theory, Research, and Practice, 2nd ed.; Allyn & Bacon: Boston, 1995.
4. Johnson, D. W.; Johnson, R. T.; Smith, K. Active Learning: Cooperation in the College Classroom; Interaction Book Company: Edina, MN, 1991. 5. Cooper, M. M. J. Chem. Educ. 1995, 72, 162. 6. Coppola, B. P.; Lawton, R. G. J. Chem. Educ. 1995, 72, 1120. 7. Amenta, D. S.; Mosbo, J. A. J. Chem. Educ. 1994, 71, 661. 8. Anderson, J. S.; Hayes, D. M.; Werner, T. C. J. Chem. Educ. 1995, 72, 653–655. 9. Fleming, F. F. J. Chem. Educ. 1995, 72, 719–720. 10. Kogut, L. S. J. Chem. Educ. 1997, 74, 720–722. 11. Towns, M. H. J. Chem. Educ. 1998, 75, 67–69. 12. Nurrenbern, S. C. Experiences in Cooperative Learning: A Collection for Chemistry Teachers; Institute for Chemical Education: Madison, WI, 1995. 13. Nurrenbern, S. C.; Robinson, W. R. J. Chem. Educ. 1997, 74, 623–624. 14. Cooper, H. M. Integrating Research: A Guide for Literature Reviews, 2nd ed.; Sage: Newbury Park, CA, 1989. 15. Hedges, L. V.; Olkin, I. Statistical Methods for Meta-Analysis; Academic: Orlando, FL, 1985. 16. Glass, G. V.; McGaw, B.; Smith, M. L. Meta-Analysis in Social Research; Sage: Beverly Hills, CA, 1981. 17. Springer, L.; Stanne, M. E.; Donovan, S. Effects of Cooperative Learning on Undergraduates in Science, Mathematics, Engineering, and Technology: A Meta-Analysis; National Institute for Science Education: Madison, WI, 1997. This document can be obtained from L. Springer, National Institute for Science Education, 1025 W. Johnson St., Madison, WI, 53706. 18. Lundeberg, M. A. J. Res. Sci. Teach. 1990, 27, 145–155. 19. Smith, M. E.; Hinckley, C. C.; Volk, G. L. J. Chem. Educ. 1991, 68, 413–415. 20. Basili, P. A.; Sanford, J. P. J. Res. Sci. Teach. 1991, 28, 293– 304. 21. Springer, L. Relating Concepts and Applications through Structured Active Learning; Presented at the annual meeting of the American Educational Research Association; Chicago, April 1997. 22. Banerjee, A. C.; Vidyapati, T. F. Int. J. Sci. Educ. 1997, 19, 903–910. 23. Bowen, C. W.; Phelps, A. J. J. Chem. Educ. 1997, 74, 715– 719. 24. Burron, B.; James, M. L.; Ambrosio, A. L. J. Res. Sci. Teach. 1993, 30, 697–707. 25. Dinan, F. J.; Frydrychowski, V. A. J. Chem. Educ. 1995, 72, 429–431. 26. Dougherty, R. C. J. Chem. Educ. 1997, 74, 722–726. 27. Lonning, R. A. J. Res. Sci. Teach. 1993, 30, 1087–1101. 28. Metz, P. A. The Effect of Interactive Instruction and Lectures on the Achievement and Attitudes of Chemistry Students; Ph.D. Dissertation, Purdue University, 1987; Dissertation Abstr. Int. 1998, 49, 474A. 29. Niaz, M. J. Res. Sci. Teach. 1995, 32, 959–970. 30. Okebukola, P. A. Int. J. Sci. Educ. 1986, 8, 73–77. 31. Ross, M. R.; Fulton, R. B. J. Chem. Educ. 1994, 71, 141– 143. 32. Tingle, J. B.; Good, R. J. Res. Sci. Teach. 1990, 27, 671–683.
JChemEd.chem.wisc.edu • Vol. 77 No. 1 January 2000 • Journal of Chemical Education
119
