The 1938-1939 COLLEGE CHEMISTRY TESTING PROGRAM F. D. MARTIN Purdue University, Lafayette. Indiana
T
HROUGH the Testing Committee of the Division of Chemical Education, the American Chemical Society cooperates with the American Council on Education in constructing, each year, a test designed to measure the degree of achievement of certain objectives common to all elementary college chemistry courses. TABLE
1
. C a e ~ r s ~ aP~RCBNIILBS, u 1939 (Booklet Marked) Par1 I Par1 I1 Poll NI rnforlnorion Problems Scicnlilic Mdhod and Vocnbulon ond E.lunlions No. eases No. colleges
8705 157
Memn . .---
29.9
Sigma MadmUm score Percentile
68
100 99
11.7
Told
8705 157 18.1 12.3 56
66 57
Collaboratina with the members of the Testincr Committee, the Cooperative Test Service, representkg the American Council on Education, and iifty instructors from twenty-nine colleges and universities gave the Committee much helpful criticism and advice. This year two tests were prepared, the first covering the usual freshman course in general chemistry, the second, qualitative ana@sis. These tests were printed and distributed through the Cooperative Test Service. A total of two hundred eighty-seven colleges participated in the general chemistry testing program as compared with two hundred seventy-five last year. Of this number, one hundred sixty-nine schools sent in the results for 10,237 students in time to be included in this report. Sixty-seven colleges used the Cooperative Chemistry Test in Qualitative Analysis. The provisional percentiles for this test are based upon eight hundred eighty-seven cases reported by the tweuty-nine differentcolleges reporting the results. This test was constructed similarly to the 1938 test, including the same three parts:' (I) Information and Vocabulary; 01) Problems and Equations; and (111) Scientific Method. Profiting by the criticisms and suggestions submitted by the users of the 1938 test, the Testing Committee revised the material included, as well as the time allotment in an effortto obtain a fairer and more serviceable examination.. The problems and equations were simplified and arranged more nearly in accordance with the order of difficulty, thus giving encouragement to the poorer students. An &art was made to eliminate some of the ambiguity from the statements used as test items in Part 111. Nevertheless, some instructors probably still feel that this part should have undergone further editinx. Table 1 gives the national based upon the results of 8705 students from one hundred fifty-seven
'
are bared on retvrns from an eatlegen who* report sheets were received bcfore J U I5th. ~ he reores listed are the raw scores of student. ~ h ~ r e c o r d their e d annwers in the test boo*lefo.
The values in each d u m n are the score at or below which fall the Paeentages of the eases indicated in the columns a t the extreme right and left of the page. Par example, the bottom entry in the Total column shows that n score of 15 or below wan -de by on. per of the studentp included; or that the lowest one per 01 =hemistry achieve total ncoresfrom 16 to 19, and f,. 16 m less, the seeand lowest one per aofonh. aod the hiahest one oer cent. achieve scores above 137.
Two hundred eighty-seven colleges participated in the program. Of these, one hundred sixty-nine returned reports on the Report Form No. 2 in time to be included in the basic tabulations and computations. There wereone hundred fifty-five in this latter mono which used booklet-recorded data. and fourteen which used &&er sheets. Due to an error in the tati is tical Department of the Cooperative Test Service two of the answer sheet distributions slipped into the final tabulations with the test booklet data. The condensed percentiles for colleges using separate answer sheets are therefore based on results from only twelve colleges. q-he two answer sheet colleges will not have disturbed the na. tional booklet percentiles t o any appreciable degree, since the number of cases in these groups are only fifty-four and thirty, respectively, and both colleges have averages within two points of the national 50th percentile.
Since the 1939 test was quite similar to the 1938 test, it is of interest to compare the relative achievement on the various parts, as measured, roughly, by the ratio of
mine the best high-jumper in a group of college athletes by keeping the bar set a t five feet; i t must be moved up in successive steps until even the best jumper cannot clear it. Viewed from this standpoint i t would aonear that the test was not undulv di5cnlt. *. Total since a student made a ~ e r f e c score t on Part I1 Indi- College lndi- College Indi- College Indi- College and other students made very nearly perfect vidual A v e r vidual Avervidual Avervidual AverScorer aaes Scoru aaer Scores axes scores axes scores on Parts I and 111. Table 3 gives the percentiles based upon the results of 1632 students from twelve colleges using separate answer sheets instead of writing the answers directly in the test booklet. While these "answer sheet percentiles" are only a point or two lower than the corresponding "booklet percentiles" it is highly probable that this close agreement is due to the superior ability of the students from these twelve colleges, as on several other tests i t has been found that the use of separate answer sheets on a test involving the time factor handicaps the average student by as much as from five to ten percentile points. In a FIGURE1.-VAR~ILITY AMONG INDIVIDUAL STUDENTS ~ L N D~ O N G study2 of two carefully matched sections, one COLLEGE AVERAGES.TABWIDEPORTION OP EACE BARREPRESENTS THE RWGEOP =R MIDDLEF ~ PER Y CENT. OF SCORES OR COLLEGE AVER- using the separate answer sheets, the other the AGES. %SHORT HORIZONTALLINE IS AT TEE MBDIAN. THENARROW booklet method, it was found that, on the 1938 PARTS OF EACH BAREXTEND TO THE 1 6 AND ~ 8~ 4 PERCENTILES. ~ ~ THE Cooperative Chemistry Test, the difference in the VERTICALLINES AT THE ENDSOP THE BARSREACH TO THE LOTH AND 9 h H PERCENTILES. THECROSSES ABOVE AND BEWW TEE SHORT BARSREP- means of the raw scores amounted to a difference RESENT TAE HIGHEST AND LOWEST COLLEGE AVERAOES RETURNED of about 10 in the percentile ranking, that is, students usinp separate answer sheets would have ranked ;bout ten points higher on the the Mean Score to the Maximum Score on each part. percentile scale if they had taken the same test Table 2 gives the results of this comparison. and placed the answers directly in the test booklet. Unfortunately, no such data are available for this year's TABLE 2 test; i t is hoped $at in future yeais the comparative RsGmrvs A c ~ l s v a a t e ~ON r TRB 1039 TBSTAS COWPAPBD TO m~ 1038 scores of matched sections will be available f u r making C o 6 s s a ~ r r v aCmwrsmu Tssr the proper correction for the use of the separate answer Mcaaseore Sigma
Maximumamre
Mean X l w % Mar.
Pnrl I 1938 1939
I938
Porl I1
38.829.9 14.8 11.7 91 68 42.7 43.9
17.2 18.1 12.4 12.3 70 56 24.6 32.3
Porl N I
1939 I938
Told
1939
1938
1939
17.9 16.6 10.8 8.6 65 50 27.5 33.1.
78.9 32.3 226 32.7
64.6 28.1 174 37.1
TABLE 3 CONDENSED P B R C B N ~ B SYOB COLLBDBS U S ~ NSB~ARATB O ANSWERS ~ B B T S Port I
Par1 II
Par1 III
Told
No. eases
No. eolleger Percentile
100 It will be noted that the achievement on Part I 08 93 showed no significant change, whereas on Parts I1 and 00 I11 the relative achievement was appreciably higher in 84 80 1939. It is possible that this improvement was due to 75 the simplification of those sections, as already discussed, 70 . 60 rather than to any effort on the part of college chemistry teachers to mold their courses to fit the topics covered in this examination. To someone not acquainted with the methods of educational measurement i t would appear, from the bottom row of figures, that college chemistry teachers are not giving their pupils adequate instruction or else this examination was too rigorous for the average college freshman. It must be remembered, however, that a test of this sort is constructed with the sheets. It may be noted that while the use of separate idea of measuring not only the achievement of the aver- answer sheets may involve a change in the national age student, but the capabilities of extraordinary stu- percentile ranking, i t will make no appreciable change dents whose achievement cannot be measured by any a MARTINAND ALLEN, "The 'clerical facility' factor for students test in which a pupil slightly above the average can taking objectively scored tests by direct answer o n the test sheet secure a practically perfect score. One cannot deter- versus separate answer sheets," J. CHEM.EDUC.. 1 7 , 7 5 6 (1940).
Pa*: I Bookkl
Barn Score
Answer Shad
1 0 0 0 1 0 1 0 1 1 1 2 3 3 2 0
2 0 0 0 1 0 2 1 2 6 1 4 3 6 4 13 17 14 8 11 9 11 7 8 8 8 6 1 2 0 1
6
5 6 6 7 6 7 10 10 10 5 13 7 7 7 7 7 5 4 0 0 1 1 2
-
Total
155
TOW
Effect of High-S~hool Chrmirtry
No
Scr Difference All Mcn
NO.cares
Booklrl
Booklrl
1
4646.99 45-45.99 4444.99 43-43.99 4242.99 41-41.99 40-40.99 3939.99 3838.99 3737.99 36-36.99 3535.99 3434.99 33-33.99 3232.99 31-31.99 3030.99 29-29.99 28-28.99 27-27.99 26-26.99 2C25.99 24-24.99 23-23.99 22-22.99 21-21.99 20-20.99 19-19.99 18-18.99 17-17.99 16-16.99 15-15.99 14-14.99 13-13.99 12-12.99 11-11.99 10-10.99 9- 9.99 8- 8.99 7- 7.99 6- 6.99 5- 6.99
6182
All
Womn 2523
H i a h - S c h ~ IHi&School Chcmirlry Chcmirlry
5894
2602
-
155
TABLE 5
Mr.'? Libcrol Arlr
1437
Percentile 100 98 93 90 84 80 75 70 60 50 40 30 25 20 16 10 7 2 0
in the relatiwe standing of students in a given class, as the good students will still demonstrate their superiority, no matter what method is used to record their answers. Table 4 and Figure 1 show that the wide variability of achievement, both for colleges as well as individuals,
Wornm'r Libcrnl Arlr 1033
141 108 90 85 78 73 68 64 67
-
Ty9a of C o W e ~ ~ ~ i c ~ .l l ~ & i and Tcaclrarr Tednical EnpiCollege Collcgc neering 3381 351 341 724
Cocducoliond Libcrd AvC
158 130 112 106 96 91 84 80 70
144 110 90 85 75 71 68 63 56
144 106 90 82 74 70 66 62 56
160 138 126 119 111 106 102 97 87
Jmiw Coke
1438 152 126 109 104 94 89 81 76 67
Percentile 100 98 93 90 84 80 76 70 60
noted in previous reports3.' is also true for the 1939 test.
students will be discouraged becduse the standards for passing may be far above their capabilities. The recognition of the rights of the superior student has been acknowledged in many colleges by the establishment of special classes for those studenk6 In one state university three different freshman courses are offered to suit the special requirements of the two TABLEG . thousand or more freshman taking General Chemistry. Students entering the schools of Agriculture, Home TBSI SEORBS BY B N ~ CHBX~STRY CLASSBS IN AC O ~ ~ ~ . M*DB ~ ~ DIY ~F B R~ m e S m e Uwrvsasm Economics, or Science are assigned to a three credit Thrca Crsdil Course Four C ~ c d i l Adaonccd R o w in descriptive suited to their needs and At& Hornc Course j o y Cndil Course for ..nur. s&.u ~ronornicr ~ n ~ superim i ~ Sludrnir ~ ~ ~ abilities, s the engineering students are assigned to a four credit course in which special emphasis is laid upon the M~~~ raw aeore, partl 23.8 22.3 19 37.7 49.6 more exact and scientific aspects of chemistry, while ~erecoti~e superior students intending to specialize in chemistry Ranking, part I 31 28 20 74 95 or chemical engineering may elect a still more advanced Mean r a n Score, course. In such cases the Cooperative Chemistry Test Part 11 15.6 15.5 13.4 30.6 43.7 is of great value in comparing the standards of achievepereenue ~~aking. ment in the various courses. As it would obviously be Part I1 48 47 41 83 96.5 unfair to compare these sections with one another, they M~~~ raw score. must be compared with similar sections in other iustituPart111 11.5 11.9' 11.3 21.4 29.6 ti^^ tions made up of students having the same professional Ranking, part 111 a1 32 31 72 $2 goal. Table 6 gives the results of a comparison of this M~~~ total 50.4 49.7 43.8 89.7 122.9 sort carried out with students in the three different Percentile Ranking. courses mentioned above. The distribution range of T O ~ ~ I 36 35 26 80.5 96.5 each section is shown by the bar graphs in Figure 7. By comparing the results for the various sections with In spite of this leveling influence, colleges do exhibit the college averages shown in Figure 1 and with the some variation, as may be seen from Figure 1, wherethe norms for the various professional groups as drawn in crosses indicating the highest and lowest college aver- Figure 3 it may be seen that the "advanced" section ages on each part of the test as well as on the total score are rather widely separated. . . Table 5, along with Figures 2, 3, 4, 5, and part 1 part 2w t 3 TOM 69 6, shows the effect of various factors upon the scores made on this examination. Apparently 66 - . . . 16 the typical high-ranking chemistry student, as .-." . - - ...
