J o h n F. Schaff Syracuse Univers~ty Syracuse, New York 13210 and Paul H. Westmeyer The Florida State University Tallahassee. Florida 32306
Comparison of Students in Modern and Traditional High School Courses
Scientists and educators concerned with chemistry teaching assert that the modern secondary school chemistry courses, "Chemical Systems" (CBA) and "Chemistry: An Experimental Science" (CHEM Study), are developing complex intellectual student abilities that are not developed by their conventional counterparts. Several attempts to determine these higher abilities have been made by comparing the performance of students taught modern and conventional high school chemistry courses. Investigators have used achievement tests, critical thinking tests, and a cognitive preference test for the comparisons (1-4). Even though these instruments have shown differences between the two types of curricula, discrepancies exist in the results reported, and few, if any, conclusions can be drawn with respect to the different intellectual abilities developed in students. Recently, studies using chemistry tests based on the behavioral descriptions contained in the "Taxonomy of Educational Objectives, Handbook I: Cognitive Domain" (5) have produced several distinct insights in determining the intellectual abilities developed by the modern chemistry courses (0,7). This report describes an investigation in which students taught modern and conventional high school chemistry curricula were compared using a test designed to measure the most complex intellectual process, Eualualion, represent,ed in the "Taxonomy." Questions were designed to measure this intellectual process within a single unit of content, Rinetic-Molecular Theory, taught in both types of courses. The courses representing the conventional and modern curricula with their associated laboratory programs were the following. Conventional Currieule H. C., WILLIAMS,J. E., "Modern Chemistry" (METCALFIS, AND CAST=, J. F.), Holt, Rinehart, and Winston, Inc., 1966. "Chemistry and You" ( B A K ~P., S., Bnnosunu, G. RI., EICHINGER, J. W., A N D E. A. SIGLER), L y o n ~and Carnahan, Inc., 1966. Modern Curricula "Chemical Systems" (CHEMICAL BONDA~monCrrPROJECT), McGraw-Hill Book Company, 1964. Table 1. "Modern Chemistry" (Holt)
TI
c, T
C,
=
C,
c, c, c,
Teacher; C
=
T,
c, c,
Class
82 / Journal of Chemical Education
C,
1963.
It was inferred that if chemistry curricula differ in their goals related to the development of intellectual processes of students, the extent to which the process, Evaluation, is developed in the students by different curricula should vary. Experimental Design
The statistical design employed was a nested hierarchical factor experimental design represented schematically in Table 1. The hypothesis tested in the null form was There is no diference between mean scores on the Evaluat,ion test of subjects given instruction in modern chemistry curricula and subjects given instruction in conventional chemistry curricula. Selection of Classes
Twenty-four classes, twelve for each treatment group, were selected from seven schools in the northeastern section of Illinois. The classes were divided equally among twelve teachers with no more than two teachers represented by one school. Each teacher had a minimum of two classes of the chemistry course desired for the investigation and was not making major revisions of the program used. Each school maintained ample facilities to include a good quality laboratory program with its chemistry course. The Criterion Instrument
The instrument used in this investigation was the "Assessment of Evaluation Ability in Chemistry." Two equivalent forms, J and S, of this instrument were prepared for purposes of estimating its reliability. Each form was based on the "Taxonomy" and consisted of two parts, Part I representing the lowest intellectual This research, conducted at the Florida State University, was supported by ihe Small Grants Program of the Division of Chemical Education, American Chemical Society.
Nested Hierarchical Factor Experimental Design
Conventional "Chemistry and You" (Lyons & Cmn)
Ta
Ts
"Chemistry: An Experimental Science" (CHEMICAL EDUCATION MATF;RI.~L STUDY), W. H. Freeman and Company,
Te
"Chemical Systems" (CB.4)
Te
c,, c,, c,*
T, C,8
c , 4
Ts C,
C,s
Tr
Modern
c,, c,,
C,
"Chemistry: An Exper. Science" (CHEM Study)
TI,
C,,
Cn
Tn
C*2
TI, Cs, c,,
process, Knowledge, and Part I1 representing thc highest or most complex intellectual process, Evaluation. The Knowledge test was incorporated to serve as a control by comparing the knowledge of related subject matter between students in the two treatment groups. No attempt was made to control for, or to measure the intervening categories of the "Taxonomy." The Knowledge test consisted of thirty-five multiplechoice questions related to Kinetic-Molecular Theory as it applies to gases, liquids, and solids. Item 4 from Form J, Part I of the instrument, reads as follows 4. The process of molecules scattering on their own when they are placed in an open environment is called: 1. osmosis 2. diffusion 3. condensation 4. dissociation
In the Evaluation test, the student was asked to evaluate nine different situations, each on the basis of his knowledge of Kinetic-Molecular Theory as it applies to gases, liquids, and solids. Each situation was broken down into five specific judgments producing a total of forty-five items. Item VII from Form J, Part I1 of the instrument, reads as follows VII According t o Kinetic-Molecular Theory, the molecules of gases are in constant motion. However, let's consider a. model for gases in which the molecules of gases are not in motion. I n this model for gases, molecules are still considered as small particles with van der Wads forces existing between them; the particles are separated by large distances. I t is claimed that this model for gases in which the molecules are stationary is just as useful in explaining the properties of gases as the model in which the molecules are in motion. 31. Would this model far gases, in which the molecules are ststionary, he useful to explain the mixing of two gases a t standard temperature and pressure (STP)? YesNoExplain your answer: 32. Would this model for gases be useful to explain the eondensation of a gas into a liquid? Yes_ NoExplain your answer: 33. Would it be possible to explain a change in the kinetic energy of gases by using this model for gases? YesNo Explain your answer: 34. Would the model for gases in which the molecules are stationary be useful to explain the gas law (Charles) relating temperature and gas volume a t a constant pressure? Yes- NoExplain your answer: 35. Would this modelof gases heuseful to explain Avogadros's Law, "Equal volumes of all gases under the same conditions of temperature and pressure contain the same numher of molecules?" Yes- NoExplain your answer:
The instrument was constructed so that the Knowledge test was administered prior to the Evaluation test. Both forms of the instrument were designed and developed by the investigator for this specific study. Validation of t h e lnslrument
The instrument was examined both for content validity and for process validity. Separate but identical validation procedures were used for the validation processes. One group of judges, possessing appropri-
Table 2.
Summary of Analysis of Variance, Knowledge Test Part Form J
I;
Sum of squares
Soufce of variance
Curricula (M), Courses (S) wlth M Teachers (T)with (M and S) Classes (C) with (M, S, and T ) Experimental .Error Total
Degrees of freedom
Mean square
F ratio
225. 10 227.71
1 2
225.10 113.85
2.17 1.10
826.19
8
103.27
3.16"
391.62 6338.62 8009.24
12 360 383
32.64 17.61
1.85"
Significant a t 0.05 level. Table 3.
Summary of Analysis of Variance, Evaluation Test Part II, Form J
Curricula (M) Courses (S) and M Teachers (T) with (M and S) Classes (C) with (%I,.& and T ) Experimental Error Total a
Degrees of freedom
F
Mean square
ratio
1
3319.38
16.38'
548.94
2
274.47
1.35
1620.46
8
202.56
2.W
866.53
12
72.21
1.92'
13502.19 19857.50
360 383
37.51
Sum of squares
Soufce of var~ance
3319.38
Significant rtt 0.01 level. Significant a t 0.05 level. A value of 2.85 would have been significant a t the 0.05 level.
ate backgrounds in chemistry content and experience with high school chemistry programs, determined the content validity of the instrument. The process validity of the instrument was determined by a second group of judges possessing a working understanding of the "Taxonomy." The judges were asked to verify the classification of the items constructed specifically for the Knowledge category and the Evaluation category. A very high level of consistency was demonstrated by the judges in the entire validation process. Identical or related items received the same classification verification on both forms of the instrument. Results
An analysis of variance of class means was used to examine differences between treatments utilizing the BMD08V "Analysis of Variance" program (8). The results shown in Table 2 were obtained from testing the control for Ihowledge between the two treatment groups. The null hypothesis tested was There is no difference between mean scores on the Knowledge test of subjects given instruction in modern chemistry curricula. and subjects given instruction in conventional chemistry curricula.
The F-ratio related to this hypothesis was not significant a t the 0.05 level. Since this was chosen as the decision level the null hypothesis was not rejected for the Knowledge test. Testing for a difference between the two treatment groups on the Evaluation test was the basis for this investigation. The results are shown in Table 3 and were obtained from testing the following null hypothesis Volume
47, Number I, January 1970 / 83
There is no difference between mean scores on the Evaluation test of subjects given instruction in modern chemistry curricula and subjects given instruction in conventional chemistry curricula.
This hypothesis was rejected a t the 0.01 level of significance. The reliability of both the Knowledge and Evaluation tests was determined by computing a correlation coefficient between mean class scores on Forms J and S of the instrument. The Pearson product-moment correlations calculated were 0.85 for the Knowledge test and 0.75 for the Evaluation test. This investigation disclosed that students taught the modern curricula demonstrated a higher performance on the Evaluation test than students taught the conventional curricula, while concurrently demonstrating an equivalent performance on the Knowledge
84 / Journal o f Chemical Education
test when measured by the instrument "Assessment of Evaluation Ability in Chemistry." These results substantiate the hypothesis that there are differences in the complex intellectual process, Evaluation, developed in students by the two curricula. This suggests that there may also be differences in other intellectual processes of the "Taxonomy." Literature Cited ( 1 ) HemH, R. W., A N D STICKEL=, D. W., Sotence Teochei, 3 0 , 45 (Sept. 1963). (2) PYE,E. L., ~ w A o w o ~ n s o a K. , H., Science Teochei, 3 4 , 3 0 (Feb. 1967). , L., J. C H Enuc., ~ 4 4 , 4 7 1 (1967). ( 3 ) M ~ n n s R. ( 4 ) RAINBY,R. G . , School Science ond Mothematicr, 6 4 , 539 (1964). (5) R m o a , H. S. (Editor) "Taxonomy of Educational Obiectives. Handbook I: Cognitive Domain," David McKay, New York, 1956. ( 6 ) HEnnolr. J. D . , J . Res. Sci. Teaching, 4 , 159 (1966). ( 7 ) ANoEnson, J. S., Unpublished doctoral dissertstion. Florida State University, 1964. (8) D ~ x o n ,W. 1.. (Editor) " B M D Biomedioal Computer Programs.'. Health Service Computing Facility, U.C.L.A.. Los Anqeles 1967.
