Research: Science and Education edited by
Chemical Education Research
Diane M. Bunce
Elementary to High School Students’ Growth over an Academic Year in Understanding the Concept of Matter
The Catholic University of America Washington, DC 20064
W
Xiufeng Liu Department of Learning and Instruction, Graduate School of Education, State University of New York at Buffalo, Buffalo, NY 14260-1000;
[email protected] The development of the concept and theories of matter is one of the greatest achievements in chemistry. The importance of matter in elementary and secondary school science curriculum is also apparent, because it is common to organize learning outcomes around unifying themes such as matter. Given the complexity of the matter concept and its theories, appropriate and clear expectations of students’ understanding of matter are necessary. The National Science Education Standards (1) state that K–12 students are expected to develop an understanding of forms and processes of matter change (such as state change, solution processes, as well as chemical and physical changes), and be able to explain how the forms and processes of matter change take place using the particulate model. The above key understandings can be categorized into four aspects: conservation of matter, physical property and change, chemical property and change, and composition and structure of matter (2). A full understanding of matter involves a reasoning that is both descriptive and explanatory, macro–observable and micro–particulate, and qualitative and quantitative (3–6). Developing Conceptions of Matter Educators have conducted extensive research including comprehensive syntheses of research findings in the past on students’ alternative conceptions of matter (7–10). Most previous studies on students’ alternative conceptions on matter were qualitative, and involved only one or a few grade levels. Because matter is a unified theme in organizing K–12 science learning outcomes according to the National Science Education Standards (1), appropriate understanding of matter can have a significant impact on student mastery of other science concepts. It is necessary to study students’ overall progression in developing understandings of matter from elementary through high school. Using the 1995 Third International Math and Science Study (TIMSS) U.S. national data sets (11), Liu and Lesniak recently found no cognitive hierarchy among four aspects of matter (conservation; composition and structure; physical properties and changes; and chemical properties and changes) (2). Based on the findings, Liu and Lesniak hypothesized that matter concept development in children from elementary through high school takes place in multiple stages. Each stage involves integrated understanding of all four matter aspects. Five stages may be identified. The first, preparatory stage, involves recognizing changes involving everyday matter such as water and air. This stage may occur before the 3rd or 4th grade. The second stage occurs by the 7th grade, and involves describing and representing www.JCE.DivCHED.org
•
matter and change in a variety of substances. The third stage involves differentiating physical and chemical properties and changes by grades 8 and 12 nonphysical science students. It requires specialized courses in high school chemistry in order for students to explain matter and change using the particulate theory—the fourth stage. Explaining and predicting matter and changes using bonding theories represents the last stage, the highest level of matter concept development, and this level may not be achieved without university chemistry courses. In a follow-up qualitative interview study (12) involving students in grades 3 through high school chemistry, Liu and Lesniak confirmed the developmental pattern above. Although this progression as described presents a general trend on how students develop in-depth understandings of matter, quantifying the trend is desirable to further test the hypothesis and to monitor students’ learning progression of matter over time. A valid and reliable measurement instrument specifically developed for matter is also needed for these purposes. In this study, a new measurement approach— Rasch modeling (13)—was used to develop an instrument that consists of three linked forms, one for elementary, one for junior high, and one for high school students. This paper briefly describes the instrument, details the data sources and analysis methods, and presents findings on students’ conceptual progression concerning the concept of matter. The research questions for this study are: 1. How does an understanding of matter grow for students from elementary through high school? 2. How does an understanding of matter grow for students over an academic year?
Methods
Measurement Instrument The instrument used, called the Progression of Understanding Matter (PUM), consists of three forms: The elementary form (grades 3–6), junior high school form (grades 7–9), and high school form (grades 10–12). Questions in different forms have different difficulty levels and relate to different aspects of matter. Details about the validity and reliability of the instrument are available in the Supplemental Material.W Sample Demographics The instrument was administered to 536 students in December 2004, and 445 students in May 2005. All the students were from an Atlantic province of Canada. Because of the province’s small population and geographic area, no clear
Vol. 84 No. 11 November 2007
•
Journal of Chemical Education
1853
Research: Science and Education
Table 1. Distribution of Descriptive Statistics of Students’ Conceptual Understanding of Matter Grade Level 3
Test (N)
Minimum Scores
Maximum Scores
Scores’ Mean
SD of Scores
1.0
1.0
1.0
0.0
—
—
—
—
(35)
1.0
2.0
1.1
0.2
M (44)
1.0
4.0
2.0
1.2
D
(32)
1.0
4.0
1.2
0.6
M (48)
1.0
4.0
1.5
0.9
D
(29)
1.0
4.0
2.2
1.2
M (46)
1.0
5.0
2.7
1.4
D
(27)
1.0
4.0
2.1
1.1
M (31)
1.0
5.0
2.5
1.4
D
(30)
1.0
5.0
2.9
1.2
M (30)
1.0
5.0
3.0
1.4
D
(30)
1.0
5.0
3.7
1.3
M (30)
1.0
5.0
3.3
1.3
D (130)
1.0
5.0
3.1
1.3
M (90)
1.0
5.0
3.3
1.3
D (101)
1.0
5.0
3.3
1.2
M (62)
1.0
5.0
3.5
1.1
D
(89)
1.0
5.0
3.9
1.1
M (63)
1.0
5.0
4.0
1.1
D
(29)
M 4
5
6
7
8
9
10
11
12
D
Note: D represents the December test; M represents the May test.
distinction exists among urban, suburban, and rural schools. The cultural heritage of the province’s residents is relatively homogenous—primarily Irish and Scottish. All schools from K–12 follow the provincial science curriculum guides and use the same provincially adopted textbooks. Matter is included in the science curricula every year from grade 3 through grade 12, following a spiral approach to the development of the concept. The distribution of teacher quality and resources are also similar from school to school, again because of the province’s small population and geographic area. Like other schools in Canada and the U.S., science in elementary schools (K–6) is typically taught by nonspecialists, although in junior and high school, science is taught by science specialists. Students in the sample were from two elementary schools (grades 3–6), one junior high school (grades 7–9), and two high schools (grades 10–12). Because students in grades 3– 10 in those schools were all taking an integrated science course, all students in grades 3–10 in those schools were invited to participate. For students in grades 11 and 12, only those who were taking chemistry were invited to participate. Consent forms were distributed to all the students. Only students whose parents signed the consent forms were asked to take the test. The test was given twice, once during the first week of De1854
Journal of Chemical Education
•
Figure 1. Charting the progression of students’ understandings of concepts of matter from grades 3–12 over a school year.
cember and again during the first week of May. Altogether, 536 students completed the test in December and 445 students completed the test in May. Because of the nature of cross-sectional surveys, the students taking the December test and the students taking the May test were considered as two independent samples from two different subpopulations.
Analyses Students’ responses to the questions in the PUM were scored using scoring keys. The total scores that students earned on the instrument (i.e., the raw scores), were then converted into Rasch-scale scores according to a conversion table described in the Supplemental Material.W Rasch-scale scores were obtained through a Rasch modeling process, which is a mathematical approach to model students’ responses to test questions to map out the correspondence between student responses and their true abilities (13, 14). Rasch-scale scores provide more accurate measurement of students’ true abilities than raw scores; only Rasch-scale scores on different forms of the matter instrument are directly comparable. The Raschscale scores were then further transformed into corresponding developmental stages of understanding according to a conversion chart also described in the Supplemental Material.W Using students’ stages of understanding as the dependent variable, and grade levels and times of testing (December and May) as the independent variables, analysis of variance (ANOVA) was conducted to test for the existence of statistically significant effects on the growth of students’ matter understanding of concepts of matter. Findings
Descriptive Statistics Table 1 presents the descriptive statistics of students concerning their stages of conceptual understanding of matter. With the exception of 3rd and 4th graders on the December test, students’ understandings vary greatly within each grade ranging from stage 1 to stage 4 or 5. Standard deviation values >1 for those grade levels also indicate wide divergence. Figure 1 shows the overall growth patterns in students’ understanding of the matter concept from grade 3 through
Vol. 84 No. 11 November 2007
•
www.JCE.DivCHED.org
Research: Science and Education
grade 12, and within each grade from December testing to May testing. It can be seen that overall student understanding of matter increases from lower grades to higher grades and from December to May within each grade. However, a few anomalies are observed. For the December test, 7th graders achieved below 6th graders, and 10th and 11th graders achieved lower than 9th graders. For the May test, 5th graders achieved lower than 4th graders, and 7th graders achieved lower than 6th graders. In addition, 9th graders achieved lower in May than in December. In order to test the statistical significance of the growth by grade and by time of testing (from December to May) using analysis of variance (ANOVA), an examination of the distribution of reached stages among students within each grade at each testing showed that the assumption of normal distribution was not universally met. Also, from Table 1 we see that the standard deviations for grades 5–12 are close to each other, indicating that there is homogeneity of variance among groups. Given that the F-test is robust against moderate violation of normality and homogeneity assumptions (14), an ANOVA after excluding grades 3 and 4 was conducted to test the significance of growth in understanding by grade and by time of testing. The ANOVA source table is presented in Table 2. From Table 2, we see that a statistically significant effect of grade emerges from the data (F ⫽ 45.89, p < .01, η ⫽ .27, power ⫽ 1.0): that is, as students progress by grade, their understanding of matter increases significantly. However, there is no statistically significant effect of testing time, nor is there a statistically significant interaction effect between grade and time of testing. Table 3 shows the results of the Tukey post-hoc comparison. The data indicate that although there is an overall significant difference among the group means across the grades, a few homogenous groups can be identified: (a) grade 5, (b) grades 6 and 7, (c) grades 6 and 8, (d) grades 8, 10, and 11, (e) grades 10, 11, 9, and (f ) grade 12. Group means within a homogenous group are not statistically significantly different, while group means between two homogeneous groups are statistically significantly different. Please note that the means for grades 6, 8, 10, and 11 appear in two groups, indicating overlaps between groups. It is interesting that the grades within and across groups are not in a naturally increasing order. This means that there is a fluctuation in students’ understanding of matter as the grade increases, although the fluctuation does not overcome the effect of overall gradual increase in students’ understanding from lower grades to higher grades. Discussion Two interrelated research questions informed this study, as described above. We have seen from the results presented that, overall, students in the sample develop their conceptual understanding of matter gradually from a lower grade to a higher grade, and over the span of an academic year. The growth in students’ conceptual understanding of matter does not contain any sudden spurt. This may be due to the complexity and the unified nature of the concept. It shows that developing students’ conceptual understanding of matter is a long-term effort. Thus, we need to introduce the matter concept from an early grade, such as grade 3, and continue www.JCE.DivCHED.org
•
Table 2. Effects of Grade Level and Timing of the Test on Students’ Conceptual Understanding of Matter Sourcea
DF
F Values
η Values
p Values
Power b
Grade
7
45.89
.27
.00
1.00
Testing
1
3.00
.00
.08
0.41
Grade X
7
0.78
.01
.61
0.34
852
(1.43)
Testing MError a
Results for grades 3 and 4 are excluded from these data. power analysis, α ⫽ .05.
bFor
Table 3. Homogenous Groups in the Progression of Students’ Conceptual Understanding of Matter Homogenous Group Means at α ⫽ 0.05 Grade
n
a
b
c
d
e
05
080
1.3
07
058
2.3
06
075
2.9
08
060
10
220
3.2
3.2
11
163
3.4
3.4
09
060
12
152
f
2.5 2.9
2.9
3.5 4.0
Note: Means were calculated using scores with ranges of 1–5.
developing students’ understanding of matter in all subsequent grades. In addition, we need to teach the matter concept holistically by involving all the aspects of conservation, physical properties and change, chemical properties and change, and composition and structure of matter. An interesting research question for future efforts would be to find out whether any intervention programs could accelerate students’ conceptual understanding of matter. In the findings reported here, we noted some fluctuation, that is, regress, in students’ conceptual understanding. Since this study was a cross-sectional survey, not a longitudinal study, the fluctuation may be due to random effects of sampling. This also demonstrates the importance of closely monitoring students’ conceptual progression from lower grades to higher grades. Any abnormal patterns need to be investigated and measures need to be taken to address them. The study reported in this paper shows one application of the PUM (described more fully in the Supplemental MaterialW). With this instrument, further research might be conducted to quantify students’ progression rate of conceptual understandings of matter. This quantification would be useful in many ways. For example, quantifying the progression of conceptual understandings of matter may be used to monitor
Vol. 84 No. 11 November 2007
•
Journal of Chemical Education
1855
Research: Science and Education
students’ learning progress. Preferably, a national norm of students’ conceptual understandings of matter at grade levels would be established so that a norm is available for comparison of a particular group of students’ understandings of matter. A normalized standard could inform accountability and instructional improvement decisions because unacceptable progression in students’ understandings of matter should be identified and addressed to develop clearer understanding in subsequent grades, such as in high school chemistry. Establishing a national benchmark for students’ conceptual understanding of matter is particularly important in the U.S. given the National Science Education Standards (1) and the No Child Left Behind Act (16). Because matter is a unified concept, teachers can expect that students’ understanding of matter is closely correlated with students’ understanding of other chemistry concepts, such as ionization energy. Thus, the PUM instrument might also be used as a measurement of covariates for studies on other topics. Once again, because matter is a unified concept, teachers could also use the instrument as a diagnostic tool to evaluate students’ potential for future chemistry learning. Acknowledgments This study was supported by the Social Sciences and Humanities Research Council of Canada, Council identification number 64043. The data presented, the statements made, and the views expressed are solely the responsibility of the author. W
Supplemental Material
Details about the validation process of the PUM instrument as well as the actual PUM instrument are available in this issue of JCE Online.
1856
Journal of Chemical Education
•
Literature Cited 1. National Research Council. National Science Education Standards; National Academy Press: Washington, DC, 1996. 2. Liu, X.; Lesniak, K. Sci. Educ. 2005, 89, 433–450. 3. Johnstone, A. H. Sch. Sci. Rev. 1982, 64, 377–379. 4. Johnstone, A. H. J. Chem. Educ. 1993, 70, 701–705. 5. Gabel, D. L.; Bunce, D. M. Research on Problem Solving: Chemistry. In Handbook of Research on Science Teaching and Learning, Gabel, D. L., Eds.; Macmillan: New York, 1994; pp 301–326. 6. Kozma, R. B.; Russell, J. J. Res. Sci. Teach. 1997, 34, 949– 968. 7. Andersson, B. Stud. Sci. Educ. 1990, 18, 53–885. 8. Krnel, D.; Watson, R.; Glazar, S. A. Int. J. Sci. Educ. 1988, 20, 257–289. 9. Liu, X. Int. J. Sci. Educ. 2001, 23, 55–81. 10. Kind, V. Beyond Appearances: Students’ Misconceptions about Basic Chemical Ideas, 2nd ed.; Royal Society of Chemistry: London, 2004. http://www.chemsoc.org/pdf/LearnNet/rsc/ miscon.pdf (accessed Jul 2007). 11. TIMMS 1995 Study Instruments and Procedures Web Page. http://isc.bc.edu/timss1995i/t95_study.html (accessed Jul 2007). 12. Liu, X.; Lesniak, K. J. Res. Sci. Teach. 2006, 43, 320–347. 13. Rasch, G. Probabilistic Models for Some Intelligence and Attainment Tests; Danmarks Paedogogiske Institut: Copenhagen, 1960. (Expanded edition with foreword and afterword by B. D. Wright; The University of Chicago Press: Chicago, 1980.) 14. Applications of Rasch Measurement in Science Education, Liu, X., Boone, W., Eds.; JAM Press: Maple Grove, MN, 2006. 15. Glass, G. V.; Hopkins, K. D. Statistical Methods in Education and Psychology; Prentice-Hall Inc.: Englewood Cliffs, NJ, 1984. 16. No Child Left Behind Home Page. http://www.ed.gov/nclb/ landing.jhtml (accessed Jul 2007).
Vol. 84 No. 11 November 2007
•
www.JCE.DivCHED.org
