A NOTE ON THE RESULTS OBTAINED FROM IOWA CHEMISTRY TESTS C. S. SWCOMBE, LINCOLN SCHOOL OP TEACHERS' COLLEGE, NEWYORKCITY
In an earlier publication Cornog and Stoddardl present results of the use of their tests in various colleges throughout the country. Two conclusions in their paper appear to be open to question, and require some notice. Part I. Variability in Aptitude In Table IV they show mean scores from different colleges in the Aptitude Test. In discussing this table they say, "A noticeable feature of this table is the rather narrow limits within which the mean scores fluctuate. East, west, country or city, large institution or small, the student in the middle of the'class most often makes a score not varying widely from 60. This indicates that freshman chemical aptitude is much the same everywhere, and that no college represented has a notable initial handicap or advantage in the quality of its starting material." In Table V they give corresponding figures for the Training Test, and discuss the wide range of mean scores there shown. They conclude "there are small differences in the average ability of freshmen to learn chemistry, but great differences in the capacity of different high schools to teach chemistry.'' An examination of their tables does not seem to warrant such a statement. Por purposes of just comparison the mean scores of oolleges presenting results of both tests have been isolated. (The finding discussed below holds if all colleges in both tables are taken, but the selected com~arisonis more valid.) Aptitude Test Mean score
College
Case School of Applied Science Earlham College Greensboro College University of Iowa Kansas Agricultural College University of Minnesota Montana State College University of North Carolina Rutgers University Santa Barbara Utah Agricultural College University of Washington Possible Score ~
~
The ranges of these means are, for: Aptitude Test, from 34.88 to 77.31 (diff. 42.42) Training Test, from 31.50 to 90.32 (diff. 58.82)
-
'THIS JOURNAL, 3,1408-15 (Dec., 1926).
73.74 52.66 34.88 47.46 59.78 67.42 64.48 63.00 77.31 69.38 44.92 70.28 (115)
Training Test Mean =ore
89.28 63.20 31.50 54.73 44.39 65.45 47.05 66.50 90.32 70.13 67.15 69.71 (188)
VOL.4, No. 7
RESULTSOBTAINED FROM IOWACHEMISTRY TESTS
895
Thus the Aptitude Test scores show a narrower range of variation. But the possihle scores, which give an indication of the range within which the mean scores can vary, are not the same, and the raw comparison is not admissible. It may be said that when the possihle range as indicated by the possible score equals 115,the actual range of mean scores equals 42.43, and when it equals 188, the actual range of mean scores equals 58.82. For each test, the only legitimate comparison of variability is the ratio, rangelpossible range. This ratio is in the case of the Aptitude Test Training Test
0.369 0.314
Thus the true variability of the Aptitude Test scores is actually greater than that of the Training Test. The difference, however, is small and may be neglected. It may be said, then, that Cornog and Stoddard's tables show that the differences in aptitude of students is just as large as are the differences in ability, when this aptitude is trained. Purthermore, if the correlation between the rank orders of the colleges, on the basis of the two tests, is calculated it is found to he 0.78. That is to say, a college which receives well-trained students in chemistry also receives untrained students with a high degree of chemical aptitude. What this indicates as to the "capacity of different high schools to teach chemistry" is left to the speculative insight of the reader of this note. Part 11. The Kind of Chemistry That Is Retained by Freshmen In Part I11 of their paper, Cornog and Stoddard give figures illustrating, and discuss, "how much and what kind of chemistry is retained by freshmen who have had high-school chemistry." On p. 1414they show the percentage of students answering certain specific questions, and use these cases in drasticallycriticizing high-school chemistry teaching, and the aptitude of students. In the Glenn and Welton tests, certain questions were deliberately included more than once, usually in different tests. The tests, numbering some 36 on various topics, are supposed to have been taken a t an interval of about a week, in sequence. The following results of these duplicated items are ill~minating.~ wn
of err+
12 14 14 15 12 12
Test ouestinn
+
Complete NaOH HC1 = Complete NaOH HC1 = Name an add used by plumbers Name an add used by plumbers The formula for ammonium hydroxide is ? The formula for ammonium hydroxide is ?
+
NO.of item
percentage
in test
correct
8 39 40 38 5 37
85 78 60 32 70 60
1 These results were isolated from unpublished (but shortly to be published) figures prepared by Mr. Glenn.
The percentage of pupils correctly answering a given question seems to dbPend not only on their absolute ability to do so, but partly upon the general setting and partly upon the position of the question in that setting. In general, test results show that the later an item is placed in a test the smaller is the percentage of children correctly answering it.8 The reason usually given for this is that tests are built so that questions become more difficult toward the end. This may be one reason, but the illustrative cases given above show that other factors operate; for 70% of pupils knew the formula for ammonium hydroxide when placed early in a test, but only 60% knew it 15 minutes later, when they were near the end of the same test. It would appear that, even if ample time is given, some children do not get through so far as to attempt some questions which they can answer. This must have happened to lOy0of the children who took Glenn's test No. 12. These results illustrate the point made, namely, that one cannot justly say that a certain percentage of children cannot answer a particular question without stating the conditions under which that question was given. The fact that 31y0 of students fail to complete and balance the equation KOH HC1 = in a specific circumstance (in the Iowa Test) does not warrant the general assumption that "about a third of the group learned nothing of equation writing while in high school."
+
Conclusion Criticisms of present results and practice in the teaching of high-school chemistry are becoming increasingly numerous. Test methods and statistical treatment of results are increasingly used as means for obtaining grounds for criticism. The above analysis of presented results of such usage appears timely a t this stage, for if testing procedures are not to defeat their own ends and fall into disrepute, they must be rigidly exact and scrupulously fair to present teachers and students. Mr. Glenn's figures show this very definitely. See Stoddard's figures in University of Iowa Studies in Education, Vol. 111, No. 2, pp. 44-52.
