THE RESULTS OF CERTAIN EXPERIMENTS IN THE TEACHING OF CHEMISTRY TO COLLEGE STUDENTS* VICTOR
H.NOLL,~ N I V B R S I T YOP MINNESOTA. MINNEAPOLIS, MIN~SOTA
The evaluation of laboratory work in the natural sciences is a problem which is receiving considerable attention a t the present time. It is a difficult one because of the variety of outcomes claimed a t one time or another for this type of instruction. The logical procedure in attacking the problem seems to be that of measuring these outcomes, one a t a time, with conditions so controlled' that the only variable is the amount of laboratory work. It is the purpose here to report the results of such a study made with beginning courses in general inorganic chemistry a t the University of Minnesota, during the fall and winter quarters, 1926-1927.' One of the most important outcomes of any course, it is generally agreed, is the information or general achievement of the student gained through tke course. It seems that the first step in justifying any teaching procedure would be to measure its effectupon achievement. If the particular method can be shown to increase this outcome, well and good. If not, we may still be able to justify it on other bases. The problem, then, was to measure
DATA*ORTA& GROUPSTESTED AND COMPARED Group
Chem. s. A.
I. 11.
111. IV. V.
VI.
No. of students
College
\ )
* Paper read
,Hrs. lect.
H n . quiz
per wk.
per ak.
uidte
Hrs. lab. Credits per wk. per qtr.
C
102
Ew. 71 Eng. 69 Eng. Ew. 1 60 Mines ( Dent., ~ e d . Pharm. 154 Phys. Ed. Agric.
Prereq-
124
H. S. Chem.
3
1 (written)
5
5
H. S. Chem. H. S. Chem.
3 3
1 (written) 1 (written)
3 3
4 4
None
3
I (written)
6
5
H. S. Chem.
2.5
3.5
4
H. S. Chem.
3
1 (alternately oral and written) 1 (3 written, 8 oral, per quarter)
5
5
a t Sesdon on Educational Research in Higher Institutions, Section
Q of the American Association for the Advancement of Science, New York City, De-
cember 27, 1928. * The writer desires to take this opportunity to express his appreciation of the hearty codperation and helpful criticisms of Prof. M. C. Sneed, Chief of the Division of General Inorganic Chemistry a t the University of Minnesota, and of the members of his Division. Without this generaus assistance, the study would have heen impossible.
VOL.6, No. 10
E x ~ s n m ~ IN T sBACHING OW CBRMISWY
1741
the effect of varying amounts of individual laboratory work upon achievement, keeping other factors constant when possible: Six sections, made up largely of university freshmen, were the subjects. The essential data for these six sections are shown in Table I. It will be seen that all but Section IV had had chemistry in high school. It should also he noted that Section Vhad an average of 2.5 hours of lecture and 3.5 hours of laboratory per week. This is due to the fact that the weekly quiz hour for this group was taken alternately from lecture and from laboratory periods. All courses were two quarters in length. No two sections were lectured to by the same instructor, but all used the same text, Sneed's "General Inorganic Chemist~y."~ Sections 11, 111, and IV used a briefer, more condensed laboratory manua13 than that used by the other section^.^ However, a careful comparison of these manuals showed them to be practically alike as far as subject matter is concerned. The manual for engineering students contains fewer but more comprehensive experiments. The Sneed and Kirk manual includ& a greater variety of experiments, many more illustrations, and more detailed instructions to the student. The briefer manual is desirable for engineering students because they spend fewer hours in the laboratory. Two measures of achievement were used in the experiment. Sections I and I1 were given the Iowa Placement Examination for measurement of previous training in chemistry, a t the beginning of the course. At the end of the winter quarter they were given the same examination to measure the improvement which had taken place.' Since high-school courses in chemistry have been shown by Koos6to differ vem little in either content or method from beginning courses in college, and since none of the students in Sections I and I1 approached a perfect score even in taking the Iowa Placement Examination a second time, it would.seem entirely legitimate to use this as a measure of improvement in these groups. The second measure of achievement was an objective test of 135 items devised for this experiment and based on the textbook and the common elements of the laboratdry manuals. This test (hereafter designated as the Special Test) contained multiple choice and matching types of questions, names for formulas, formulas for names, tests on valence, writing, and balancing of equations and problems. This test had a reliability coefficient of 0.86 as obtained by 'Sneed, M. C., "General Inorganic Chemistry," Ginn & Co.. New York, 674 pp., 1926. Heisig, G . B., and Ktrk, R . E., "Experiments in Chemistry for Engineering Students," Burgess, Brooke, Inc., Minneapolis, 56 pp., 1926. 4 Sneed, M. C., and Kirk, R. E., "Laboratory Manual of General Inorganic Chemistry," Burgess, Brooke, Inc., Minneapolis, 126 pp., 1926. Re5 Koos, Leonard V., "Overlapping in Chemistry." The Junior College. search Publications of the University of Minnesota. Educalion Series, No. 5, The University of Minnesota, Minneapolis, pp. 474-93, May, 1924. J
the alternate items method of self-correlation and use of the SpearmanBrown formula. The scores of 118 students in this test when correlated with their second scores in the Iowa Placement Examination yielded a validity coefficient of 0.71. This test was given'to all of the six sections in March, 1927, at the end of the winter quarter. The data obtained by use of these two tests provided two lines of investigation. First, the six sections could be compared as groups on the basis of final achievement. Second, Sections 1 and I1 could be compared by matching individuals in Section I with individuals in Section I1 on the basis of available criteria. These were, percentile rank in intelligence tests, grades received in the fall and winter quarters of university work in subiects other than chemistry, and initial ability and training in chemistry as measured by the first score in the Iowa Placement Examination. The intelligence tests were the regular batte~yof tests (unpublished) taken by all freshmen when they enter the University of Minnesota. The grades used in matching were weighted in accordance with the values recommended by W ~ o d . ~ TABLE I1 SPECIAL TESTRESULTS Gmup
I I1 I11 IV V VI
NO. of Students
Range
102 71 69 60 154 124
78133 73-123 78122 68-129 ' 58130 53-131
580
53-133
108.04 + 106.32 * 106.01 + 104.50 * $01.17 * 96.61 +=
All groups
combined
Standard deviation
Mean
102.96
0.82 0.95 0.86 1.12 0.79 0.91
* 0.58
12.40 + 0.58 12.00 * 0 . 6 7 10.65 += 0.61 12.96 * 0.79 14.55 * 0.56 15.25 * 0.65 14.10
* 0.41
The group comparisons based on Special Test results are shown in Table 11. Inspection of these data shows some rather large differences, both among the groups and between each group and the combination of all of them. Throughout the study, differences are expressed in terms of their probable errors and the chances that the obtained difference is a true one. Any difference which is three or more times its probable error is regarded as a statistically significant difference. The significant differences obtained by group or section comparisons in the Special Test are shown in Table 111. In this and following tables, the chances that the obtained difference is a true one, are expressed in terms of "per cent," based on tables of the normal probability surface. The chances that the obtained difference is a true one are expressed in terms of per cent of the total area under the normal probability curve, *Wood, Ben D., "Measurement in Higher Education," World Book Company, New York. pp. 74-80, 1923.
VOL. 6, NO. 10
EXPBR~MENTS I N TBACH~NG OP CHEMISTRY
1743
corresponding to distances on the base line between the mean and successive points of division laid off from the mean. These distances are obtained by dividing any given difference between two means by the probable error of that difference. Then, by reference to tables of the normal probahility integral such as that given by R ~ g gthe , ~ reliability of the diierence may be obtained in terms of per cent of the total area for one-half the normal probability curve surface. Multiplying such a result by two gives the percentage based on the total surface for both sides of the curve. This may also be expressed as chances in 10,000 by moving the decimal point two places to the right. TABLE I11 STATISTICALLY SIGNIFICANT DIFPERENCBS BBTWBBN GROUPS D i f f . bet",. Groups
means
I-V I-VI I-all II-V II-VI II-all III-V III-VI III-all IV-VI V-VI VI-all
6.87 11.43 5.08 5.15 9.71 3.36 4.84 9.40 3.05 7.89 4.56 -6.35
P.
E.of diff.
Per cent
-1.13 *1.23 +0.90 *1.24 -1.1.32 -1.03 *1.16 -1.25 *0.94 *1.44 -1.20 -0.99
50.00 50.00 49.99 49.74 50.00 48.60 49.75 50.00 48.5? 49.98 49.48 50.00
Grovpo
I-VI II-VI III-V III-VI III-all
D i f f . bet-. S. D ' s
2.85 3.25 3.90 4.60 3.45
P. E. of diff.
Per cent
-0.87 +0.94 10.83 -0.89 -0.66
48.58 49.01 49.92 49.97 49.78
+
Table I11 shows significant differences between the means of Sections I, 11, 111, and those of Sections V and VI; also between the means of Sections I, 11, 111, and that of all combined, these differences being in favor of Sections I, 11, and 111; also, that the mean of Section V1 is significantly lower than that of all combined. Between standard deviations we find sta-, tistically significant differences between the S. D.'s of Sections I and I1 and that of Section V1; between the S. D. of Section I11 and those of Sections V, VI, and of all combined. Section I11 is the most homogeneous and Section VI the most variable as well as the lowest in average achievement. On the basis of these comparisons it seems that there are large differences in the achievement of these groups which could scarcely be accounted for by differences in instruction, intelligence, or amount of laboratory. I t does not appear reasonable to suppose that a difference in means of 11.43 score points or 0.81 of a standard deviation could be accountedfor by diierences in the instruction or in the intelligence of these groups. It can certainly not be accounted for by differences in amounts of laboratory since &Rugg.H. O., "Statistical Methods Applied to Education,'' Houghton Mifflin Company, New York, p. 391, 1917.
Sections I and VI between which this dierence exists, had the same amount of laboratory. Again, Section I11 with only three hours of laboratory per week is almost as much superior to Section V1 as Section I is. The correlation between achievement as measured by the Special Test and percentile rank in intelligence is 0.43 * 0.04 and it does not seem probable that differences in intelligence between these respective sections would be great enough to explain such large differences in achievement. Neither is i t likely that one section was much better taught than the others. The answer to the question probably lies in the realm of factors not measured in this study, such as aptitude and interest. This seems plausible since Section I is composed largely of students who are majoring in chemistry or chemical engineering, whereas Sections 11, 111, IV, V, and VI are composed of students who take these courses chiefly because they are required to do so. In comparing Sections I and 11, an attempt was made to eliminate some of the uncontrolled variables in the above-described comparisons. Students in Section I1 were all engineers, civil, mechanical, electrical, etc. These two sections had different instructors which does not seem to have been a very important factor. They used slightly different laboratory manuals, which has already been discussed. It seems likely also that Section I1 did not cover quite as much of the text as Section I. In all other respects, their instruction was as similar as might be expected under the conditions, the chief difference being that Section I had five hours of laboratory per week and Section I1 bad three hourg In the writer's opinion all diffcrences in materials and instruction favored Section I. In Table IV are shown the data for pairs matched on the basis of the initial score in the Iowa Placement Examination. The results are expressed as actual gains, possible gains, and per cent of possible gain actually attained in the Iowa Placement Examination, and in terms of achievement as measured by the Special Test.
Section I
In. smre, la. Place Final score, Ia. Place Gain Possible gain Per cent gain Special test
90.0 * 3.39 130.1 * 2.71 40.1 * 2.52 98.0 * 3.33 40.9 * 2.09 113.6 * 1.29
Section I1
89.5 * 3.26 128.3 * 2.96 38.8 * 2.23 98.4 * 3.23 39.6 * 2.01 106.6 * 1.65
P.E.
Differences differences
0.5 1.8 1.3 -0.4 1.3 7.0
*4.70 *4.01 *3.36 *4.63 *2.89 *2.09
Per cent
2.85 11.67 10.21 2.15 11.90 48.78
The only significant difference, here, is between the means in the Special Test. This diierence is a statistically significant one and favors Section I.
VOL. 6. NO.10
EXPERIMENTS IN TEACHING OR CHEMISTRY
1715
In Table V are shown the results for pairs matched in intelligence and average weighted grades. There are no significant differences in this case. V TABLE
RESULTS POR PAIRS FORMED ON THE BASIS OP PERCENTILERANKIN INTELLIGENCE TESTSCORESAND AVERAGEWEIGHTED GRADES-NINETEEN PAIRS
P. R. intell. 63.5-4.69 Grades 5.25*0.187 Gain, la. Place 44.2 * 3.45 Possiblegain 105.4 *4.43 Per cent gain 40.9 * 2.26 Specialtest 110.0 - 1 . 4 5
P. E.
Section I1
Differences
differences
Per cent
636-4.46 5.3*0.190 43.6 '* 2.29 104.0*3.62. 42.9 * 2.54 109.0*1.67
-0.1 -0.05 0.6 1.4 -2.0 1.0
*6.47 *0.286 -4.14 15.72 *3.39 *2.21
0.27 4.82 3.76 6.43 15.21 11.93
Section I
Eighteen pairs were formed matching individuals on the basis of intelligence and initial score in the Iowa Placement Examination. The results for these pairs are shown in Table VI. TABLE VI RESULTS FOR PAIRS FORMED ON THE BASIS OR PERCENTILE RANK IN INTELLIOENCE
SCORES I N THE IOWA PLACEMENT EXAMINATION TESTSCORES AND INITIAL EIGHTEEN PAIRS Section I
P. R. intell. 67.4*4.13 In. score, la. Place 92.8 * 4.04 Gain 39.0*3.56 Possible gain 95.1*4.16 Per cent gain 40.0*2.54 Special test 106.8 * 2.24
Section I1
66.614.04 91.8 * 3.71 41.5r2.22 96.1*3.67 43.5*1.79 112.0 * 1.95
P. E. Differences differences
0.8 1.0 *2.5 -1.0 -3.5 -5.2
*5.77 +5.48 *4.19 *5.54 *3.10 *2.96
Per cent
3.49 4.82 15.46 4.82 27.09 38.11
These comparisons reveal an appreciable though not statistically sigiificant difference between the means in the Special Test, favoring Section 11. This is the only difference which has appeared in favor of Section 11. . ' In Table VII, we have twenty-four pairs matched in average weighted grades and initial scores in the Iowa Placement Examination. We find here the second significant difference between means in the Special Test, TABLE VII RESULTSPOR PAIRSFORXED ON THE BASIS OR AVERAGEWEIGATEDGWES
AND
INITIAL SCORES IN IOWAPLACEMENT EXAMINATION-TWRNTI-FOUR PAIRS Section I
In. score. Ia. Place Average grades Gain, la. Place Possible gain Per cent gain Special test
90.2 * 3.39 5.62 * 0.20 42.4 * 2.69 97.6 *3.39 42.3 *2.06 113.7 * 1.35
Section I1
89.8 + 3.14 5.65 * 0.18 35.4 * 2.19 98.3 *3.25 36.3 *2.09 107.5 * 1.77
P.E.
Differences differences P a cent
0.4 -0.03 7.0 -0.7 6.0 6.2
t4.60 *0.28 *3.47 *4.73 *2.97 *2.01
2.15 2.69 41.23 3.76 41.34 48.11
in favor of Section 1. The first one, it will be recalled, occurred with pain matched in initial scores in the Iowa Placement Examination alone. Finally, pairs were formed, using all three criteria in matching. The results for these are shown in Table VIII. No significant differences appear, although there are differences between the means in per cent gain in the Iowa Placement Examination and in the Special Test, both in favor of Section I. The differences are about as large as any others found, but lose significance because of the few cases involved, and consequent large probable errors. VIII TABLE RESULTS*ORPAIRS FORMED ON TH& BASISOP PERCENTILE RANK IN INTELLIGENCE TESTSCORES, AVERAGEW E I G ~ E GRADES, D AND INITIAL SCORES IN THE IOWA PLACEMENT EXAMINATION-NINE-PAIRS P. R. in intell. Average grades In. score, Ia. Place Gain, la. Place Possible gain Per cent gain Special test
Section I
Section I1
68.4 * 6.83 5.75'0.27 88.6 * 6.02 45.2 * 4 . 6 8 99.3 * 6.02 45.7 - 4 . 2 8 111.7 * 2 . 6
67.2 * 5.76 5.65-0.23 89.0 * 5.62 41.0 * 4 . 7 8 9 8 . 8 * 5.62 41.7 * 2 . 5 9 107.3 + 2.71
P. E. Differences differences
1.2 0.10 -0.4 4.2 0.5 4.0 4.4
*8.19 +0.35 '8.23 *6.69 *8.23 *5.00 +3.78
Per cent
3.76 7.48 1.08 16.20 1.61 20.53 28.49
All matched groups were also compared as to grades received in chemistry. Since the grades were given by two instructors, they were first equated in terms of their respective stvdard deviations. No significant differences in average weighted and equated grades were found. In all matched group comparisons, the standard deviations were compared in every case. No significant differences in variability were obtained, but the group having the higher mean achievement in the Special Test had, in every instance, the smaller standard deviation. In terms of standard scores, therefore, the differences would become more significant than the obtained raw score differences.
Conclusions I n conclusion it may therefore be said that two facts appear evident as a result of the study. First, there are significant differences between achievements of these sections, as measured by an objective test based on material included and taught in all the courses. Second, when two groups very similar in all respects excepting the amounts of time spent in the laboratory were compared as carefully as was found to be possible, the section having the greater amount of laboratory work showed consistent superiority in general achievement. The surprising and perhaps disappointing fact, a t least to advocates of extensive individual laboratory work, is that these differences are not larger or more often significant.
VOL.6, No. 10
EXPEX~IFJNT~ IN TEACHING OP CHEMISTRY
1747
Some of the defects of this study have already been pointed out. The two chief criticisms are inadequate control of variables other than the amount of laboratory work, such as previous training, different instructors, different emphasis in the various groups, etc., and failure to measure improvement as well as final achievement. Further experimentation is now being carried on, in which an attempt is being made to remedy these faults.
T H E STATUS OF CHEMISTRY IN TEXAS HIGH SCHOOLS AFFILIATED WITH THE SOUTHERN ASSOCIATION OF SECONDARY SCHOOLS1 Texas had a total of one hundred fifty high schools which were members of the Southern Association of Secondary Schools during the year 19271928, one hundred thirty-eight of which were public. In order t o determine the present status of chemistry in the public high schools of Texas, a survey was made of these one hundred thirty-eight schools because they seemed to make up a representative group. The facts were ascertained from questionnaires that were sent to the various chemistry teachers who were teaching in these public high schools d u & g the spring of 1928. The aims of the investigation were to determine the status of chemistry with respect to the number of schools offering the subject and the number of pupils studying it; the training of the teachers and their teaching load; the methods and materials used in the teaching of this science; and the personal reactions of the Texas high-school teachers as to the existing obstacles in the teaching of chemistry, and the possible changes from present subject matter and method for making the teaching of this science more effective in Texas. The following form of questionnaire was used in making the survey: School
'$exher Degree
No. of 1. 2. 3.
years teaching chemistry How many students do you have in chemistry? Is chemistry required or elective? How much science is required by your school for graduation? 4. What is the total enrolment of your school? 5 . How many periods a week do you teach chemistry? Lab. Rec. 6. What subjects do you teach besides chemistry? 7. State the author of text and laboratory manual used. 8. Underline the nature of outside material used in your course: library references, local industries, community interests, individual projects. 9. Underline the obstacles, if any, that tend t o limit the interest of chemistry students in high school: too much theory, unrelated t o experience of the child, college entrance requirements, lack of library facilities, not enough laboratory equipment. any others.. . . . 10. What changes from present subject matter and method would make chemistry meet the needs of high-school pupils more effectively? 11. Underline the source of your teaching preparation: State University, State Agricultural College, State Teachers' College, Denominational School, any others. . . . . A thesis in partial fulfilment of the Degree of Master of Arts, New York University, October, 1928.
