RETENTION in the PHYSICAL SCIENCE SURVEY COURSE' HENRIETTA FREUD
AND
NICHOLAS D. CHERONIS
Chicago City Colleges, Chicago, Illinois
D
OUBT is frequently expressed, both by students and educators, as to the permanence of the knowledge gained in a science survey course. Those whose faith is placed in the laboratory method alone feel that, since the survey students do not themselves perform experiments, their knowledge, gained from viewing demonstrations, hearing lectures, and studying books, cannot be sufficiently vivid to he permanent. The wide scope of a survey course causes many to feel that such an extensive field can be covered only superlicially and that, therefore, such learning must necessarily be lost rapidly. Frequently, also, the only criterion of a student's success or failure in the survey course is a comprehensive examination, a condition which, in the eyes of many people, must encourage "cramming" a t the expense of systematic day-by-day study, and "cramming" is believed to h e . almost valueless as a means of acquiring permanent knowledge. The experiment described in this paper was performed with the purpose of testing the accuracy of the above statements. It was hoped, also, that investigation of the retention of the different types of subject matter would yield some clues as to the selection and presentation of the material of the course. The experiment consisted in the repetition, after one year, of the comprehensive examination in physical science given in June, 1937. A group of one hundred twelve students was available for the test; after securing their cooperation by explaining the purpose of the experiment, the examination which they ha8 taken a year before was administered under conditions as similar as possible to the original, without any opportunity for study. The only condition which could not be reproduced was the nervous strain of taking an important examination; in the present case the students knew, of course, that the results of the examination would not appear on their college record. According to statements of individual students, the nervous strain stimulates some and hinders others, the former group being apparently somewhat more numerous. On the whole, the cooperation of the students was splendid. It should be noted that the examination given was not designed for this experiment, but was instead one of the regular course examinations. I t was entirely objective, consisting of four hundred thirty-nine items grouped into sixteen parts according to subject matter and type
of question. Because of the large number of items, it was thought that relatively few of them would be remembered individually, and that therefore the "practice effect" would be small. A much larger uncertainty would have been introduced by the attempt to construct a second examination testing the same knowledge as the first, but using different wording. Unfortunately, these students had taken no inventory
.\ -a -x X
X
\
X
X
.I..
'
X X
xxLx a--2I
):
.-
-
5
..
2
:, ):
t ..
o
-+
'00
c
~emory Reason
+ c
100
0
test to measure their knowledge when they started the course; the score of the 1937 examination represents the sum total of their knowledge up to that time, not merely that gained in the survey course. There is no Way of correcting this score so as to exclude previous information. To obtain such results it would have been necessary to postpone the experiment for two years. The ratio of the mean score in 1938 to that in 1937, a quantity which is ordinarily known as the retention and which will in this paper be called the apparent retention, was 75.6 per cent. This is a somewhat Presented before the Division of Chemical Education at the than was expected by even the more ninety-sixthmeeting of the A. C. S..Milwaukee, Wis, September higher kure 7. 1938. enthusiastic advocates of the survey course. In the 289
literature concerning college students two groups of in Table 1. There are three possible responses, right, results are reported; one showing retentions ranging wrong, and omitted, represented by 7, w, and o, refrom about sixty to eighty per cent.2 and the other spectively. In comparing two examinations nine comshowing retentions of one-half or even less3 Studies binations are possible: rr, nu, ro, wr, vw, wo, or, ow,oo, of retention in high school subjects usually agree more where the first letter indicates the response to the 1937 closely with the former group.4 Retention in the examination and the second that in 1938. Every item survey course compares favorably with that in other was classified in this way (a total of 98,336 responses in college subjects, even those taught by the laboratory TABLE 1 method. However, very few results on retention in the MBTRODOP ANALYSIS nhvsical ------ sciences on the colle~elevel have been ~ u b lished. In this connection it should be noted that the a~~;~;~~~~~~;;;;;g;~;~ ~t~;x;~~~;on~E= line c hin the experimental group, as judged by the results of the 1937 r = tight = wmng 0 = omitted examination, was slightly better than the whole class. "I rw "0 W, WW WO 0, OW 00 The mean 1937 score of the whole class was 62.4 per cent., while that of theexperimental group was 64.0 per 2 25 13 o 1 9 o o o o 3 0 3 cent. his was to have been expected, since i t is in and so 00 general the poorer student who drops out of college A. in the that the item war wroae ia - 1937 and tight in 1938. before completing his course. Synapse. were made of there tables, classifying the students by physical The effect of sex is noticeable but not marked; the sdence C O U T S ~ Staken. 1937 score r rr + r ( ~ apparent retention of the boys was 76.9 per cent., 1938 oeme = rr + wr + or while that of the girls was 71.4 per cent. In considerTrue retentioo = rr divided by 1937 score Lose = (rs + ro) divided bv 1937 score ing these results it should be noted that only 45.4 per Gain = (wr + 01) divided bG1937 score cent. of the girls took subsequent courses in the physical Apparent retention = 1938 score divided by 1937 $core = Irr C W? + or) divided hv 1937 score sciences, while 62.4 per cent. of the boys continued in True retention + 10s. this field. As only twenty-seven girls were in the group, possibly these figures are not typical of large all), and a table such as is shown in Table 1 was conclasses. The apparent retentions of the individual students structed for each student. The true retention is found range from 28.7 per cent. to 107.6 per cent. The by dividing those items listed in the column headed rr Pearson product-moment coefficient of correlation be- by the total 1937 score, the sum of the columns rr, nu, tween the apparent retentions and the 1937 scores is and 70. The loss is found by dividing columns nu plus 0.20, too small to indicate any significant connection ro (omitted items are considered incorrect in grading) between the two. This result confirms the investiga- by the 1937 score. The gain, on the same basis, is the tions of Cederstorm and E u r i ~ h . Johnson,' ~ however, ratio of the sum of the WY and the orrolumns to the 1937 finds a positive correlation between these two quanti- score. The apparent retention (the ratio of the scores) is seen to be actually the sum of the true retention and ties. However, consideration shows that the term "re- the gain, though in the literature i t is called simply tention" as applied to the ratio of the scores on the two "retention." There is a considerable difference beexaminations is misleading. The 1938 score is made up tween the true and apparent retentions:' jn the present of items which were answered correctly in 1938, but case. the true retention is 59.8 per cent., while the apwhich may have been either right or won% in 1937. parent retention is, as quoted above, 75.6 per cent. If an item was answered right on both examinations, it The difierence, amounting to 15.8 per cent. of the 1937 may be said to be retained, but if it was answered score, or about ten per cent. of the total number of wrong in 1937 and right in 1938 it represents not re- items, is not retention but gain. The most serious tention, but a gain during the intervening year. This consequence of using the term retention to indicate the point is illustrated by the highest individual ratio ratio of scores is that the loss is maae to appear much found, 107.6 per cent. If this were truly retention it smaller than i t actually is. For example, the loss incould not possibly be above one hundred per cent., but dicated by a "retention" of 75.6 per cent. would be the ratio of the scores is above one hundred per cent. 24.4 per cent., whereas it actually is 40.2 per cent., whenever the student has gained more than he lost since the true retention is only 59.8 per cent. Probably during the interval. To distinguish between true re- both the loss and gain items result largely, not from a tention, loss, and gain, the data were analyzed as shown changed conviction on the part of the student as to what he believes the correct response to be, but rather from a general haziness on the subject. If a student J. Ap. Psych., 18, 209 '"answers the same way on both examinations, he is ' C-a a-.a.~-,r -T. Fdur . ... apparently fairly sure of the response which he believes J. Educ. Psych.. 21, 3 7 4 7 (1930). ' FRWCAEY,Ohio Educ. Research Bul., 16, 34-7 (1937): to he correct, but changing from one answer to another T n m , ibid., 9,490 (1930); DOUGLASS, Math. Teacher, 29,287-88 might easily be the result of uncertainty. 11P.?fi> As many of the students had taken chemistry and physics courses during the intervening year, the effect
.~ 2
-
(U
-
of such subsequent training had to be evaluated. The students were grouped according to their training as shown in the left-hand column in Table 2. The apparent retentions of each group in each type of subject matter are given. In using this table and the next, caution must be observed in drawing conclusions because of the small size of the groups. It will be seen that students who have taken either chemistry or physics have an apparent retention about eight per cent. higher than that of their non-scientific classmates, while those who have taken both subjects have a retention about eighteen per cent. higher. Students who have taken chemistry do better in chemistry, and students who have taken physics do better in physics, than those who have had neither subject in college. In other subjects, however, there is not much diierence in the retention. Two exceptions should be noted: the students who have taken physics are better in chemistry also, and those who have taken both chemistry and
tion a curious observation can be made; the true retention is seen to change as does the apparent retention -better in physics for the physics student, better in chemistry for the chemistry student, with little difference elsewhere; but the gain for all groups of students is much more nearly constant, and the variations which do occur seem to be erratic. This leads to the unexpected conclusion that further training in physical science is effective in preventing a student from changTABLE 3 TRVB R
B ~ B NA~N D~CIIN N BY SValeclS AND
ems*
TABLE 2
icr, Num- 1937 chcmbe, meon ir1ry. in (per nsrron- ~ h v r - chenr-
~ o i -~ n r o n - rubO W ."my j&,
52.9 20.4
56.4 14.4
47.5 12.7
56.5 13.7
54.7 14.7
59.6 14.0
53.6 20.6
57.4 17.4
43.0 12.7
55.9 21.7
58.8 17.0
65.8
74.8 13.4
72.6 24.6
70.2 13.6
47.3 10.5
58.5 13.3
64.8 14.3
46
65.6
66.9 13.4
54.3 19.4
74.8 15.6
48.1 12.5
51.1 19.0
61.7 15.6
10
70.3
79.5 11.0
65.3 21.0
82.5 17.5
54.4 11.6
64.2 20.5
70.8 16.1
112
64.0
66.3 135
56.6 20.4
68.3 15.9
47.2 12.4
55.1 18.5
59.8 15.8
CI"~.)
OmY
16
64.5
61.4 14.8
31
58.4
physics
9
College ehemistry CollcF physics
only
Phys-
croup No science High
maon
isir~.
16
64.5
76.2
73.3
Chrmiriry
70.8
Cml- Aston- subam omy hell 60.2
70.2
69.4
31
58.4
73.6
74.2
74.8
55.7
77.6
70.8
9
65.8
88.2
97.2
83.8
57.8
71.8
79.1
46
65.6
80.3
73.7
90.4
00.6
70.1
77.3
College
ehcmistry College physics and
i
-
and
chemistry
~chool
science only College physics
College
Rdrn. lion i n all
in (per ormn- Phrrgmup ccn1.) o"W is
liol i n 011
illy
emrp
rdeoee
ics, cham-
Rdan-
High-school
APFARBNT RBIBNT~DNBY S U B J BA~NS D BY CROV?S
1937
CROUPS
Phys.
No seen=
Number
BY
Upper figure represents true retention (1937 right, 1938 right). Lower figure represents gain (1937 wrong or omitted, 1938 right).
physics are slightly better in other subjects as well. The significance of both figures is open to serious question because of the extremely small size of the groups; the second is partially explained by the higher mean on the 1937 examination, which suggests that this group has an exceptionally high scientific ability. Another anomaly is found in the high retention in astronomy shown by those students who had taken only highschool science. The order of retention of the whole group in the various subjects should also be mentioned: chemistry, physics, astronomy, and then, very low, geology. This is partially due to the fact that many more students take chemistry subsequently than physics, while it is impossible for them to take either astronomy or geology. The low retention in geology will be discussed later. As it seemed desirable to study the true retentions from the above point of view, Table 3 was constructed, in which the apparent retentions given in Table 2 are broken up into true retentions and gains. On examina-
Refention of whole gmup
ing a correct answer to an incorrect one, but much less so in causing him to change an incorrect answer to a correct one. It is true that, taken on a percentage basis, the variations in the gain appear considerably greater, but even then they are not as large as those of the true retention in most cases. This' point merits further study both from the experimental point of view, to check the validity of the results of this experiment, and also, if they are proved valid, from the theoretical, to interpret its significance in analyzing the learning process. One of the aims of this study was to see if any information could be derived as to the selection of subject matter for the course. With this in mind, Table 4 was constructed, arranging the various parts of the examination in decreasing order of the apparent retention. All questions are considered to be answered by the use of memory or reason or a combination of the two, and the parts of the examination may be roughly rated as to the percentage of items probably answered by memory. It should be recalled that the examination was not designed for this experiment; if it had been the questions would have been constructed with this in mind, to test as nearly as possible pure reason or pure memory. Column 3 gives such a rating; it is seen that most of the items in the upper half of the table, which
were well retained, require less memory than reason; while those in the lower half, which were poorly retained, require less reason than memory. The same iuformation is presented in a graphical way in the figure. The wide scattering of the points indicates that the relation is only a general trend, and is in no sense accurate in detail; but that it is real can be seen from the line joining the points which represent the averages (both a s to retention and memory) of points to the left and right, respectively, of the fifty per cent. memory line. The distinct slope indicates that there is a decided connection between the amount of reason involved and the retention. The case of geology is especially interesting. The two exercises which show much the worst reteution of any are in that field. One requires a knowledge of
.-.-
Selection; tell whether statement is true of sound or light or both. WWWW motion; larxelv facts and definitions but connected hyib&y Completion; principles of equilibrium T N C ~ B I S ~c . ti^^ of d a t e d ~ r i ~ d Applications of a~uonomieal,phpical, and chemical prindples ~ m t f a l s e ; astronomical facts, dehnitionr, principles Selection; application of principles of oddation-reduction Completion; physics problems rutf false; geological definitions, facts, some principles Completion; chemical dCfinitions grouped around c1e"tron and ionization theories; facts of metallurgy Matching; naming dismvererr of laws, the.,. ties, and so forth Selection; tell whether statement is true of nebular or planetesimal hypothesis or both false; artronomiral principles and definitions ~ r u t f a l a e ,completion; op* nnd eleeuomagnetic theory; mostly factual, some app1ieations of principles ~ ~ t ~ h i nplacing g ; geological events in eorreet eras and periods Completion; geological terminology, reeognitlon of diagrams
many geological terms: anticline, unconformity, stalactite, peneplain, and so forth. The other requires the placing of geological events, such as the Laramide revolution, or the advent of flowering plants, in the correct era and period. Both are almost pure memory questions. The other question in geology, Number 11, asks also for facts and definitions, but in that case they are grouped around principles. It is interesting to note here that, according to statistical student opinion, geology is the most interesting and the best understood of the four sciences dealt with. Returning to Table 4, a t the top of the column headed "type of exercise," note the prevalence of the word "principles." Where facts are found they are related to
principles. Toward the bottom are the questions which involve knowledge of definitions, urnelated facts, scientific terminology. This observation serves as a guide for the selection of subject matter. It is easy to tell a student to memorize a list of properties or to learn the relative abundance of the elements in the earth's crust. But unless those facts are related in such a way that one leads to another through some theory or priuciple they will be forgotten; this is the kind of material which is lost. It is much more difficult to develop the theories and principles which tie the facts together, but once a principle is understood, it is not easily lost. Principles, theories, necessary nomenclature, facts from which the principles are developed or which can be deduced from them, or which illustrate them well, should form the backbone of the survey course; they will remain. In a more advanced course in science, one is justified in presenting the factual background, much of which is unrelated by theory; this is the material with which the theory-makers of the future must deal; but in a survey course, unless such material is given for the sake of showing that the small core of theories and principles which we do know is surrounded by a much larger field of fact as yet uncorrelated, i t is much better to lay emphasis on the ideas rather than on the facts. Only those facts which are tied together by ideas remain; the others are. lost. Tyler7 has made a observation; he found that high-school students ~similar i ~ . after the lapse of a year had lost very little of their ability to apply scientific principles learned in the general science course, but much more of their knowledge of scientific nomenclature and factual iuformation. This principle is frequently violated. The ease with which definitions of terms and detailed factual information lend themselves to the construction of objective exercises leads to an over-emphasis on this type of knowledge in modern examinations. Many of these definitions are quite unnecessary for a survey student, however important they may be fdr. the specialist. Also, an attempt, usually unconscious, to teach the four sciences individually often results in the inclusion of much detail which cannot be adequately tied together by theory and practice in the limited time allotted. Anxiety to bring out the social significance of science or attempts to "motiyate" by indicating oractical uses mav cause one to rewire kuow1edg.e - of applications of scientific principles which are not themselves understood, leading in the extreme, to a degeneration of the course into a mere enumeration, with parrot-like repetition, of the more spectacular achievements of science, or to the memorizing of lists of "uses" of compounds with which the students are almost entirely unfamiliar. In physics the same thing is seen in the attempt to teach radio or television before even a moderate knowledge of the principles of electricity has been gained. It is undoubtedly important to give the student a conception of the place of science in the modern world, but that appreciation is gained perma7
T n m , J. Higher Educ.. 4, 203 (1933).
nently only when he understands how science has achieved those results. The teaching of a beginning course in science should be the development of understanding through the knitting together of principles, theories, and facts into one coherent whole, indicating threads leading away into the unknown but not following them. This core of understandable related material is what is remembered; unnecessary detail or material which has not been firmly fastened in place is lost. Even principles whose importance do not justify their detailed development or which cannot be adequately connected to applications and other principles are better left out. The general results of the study seem to be: (1) Retention is considerably higher than was expected by even the optimistic. It seems, then, that the survey course does have considerable permanent value aside from the creation of the intangible attitudes and appreciations in the development of which it is hoped and believed to be effective. (2) The fear that the use of the comprehensive aamination as the only evidence of the student's success or failure is encouraging the acquirement of purely temporary knowledge seems to be'unfounded. Either it does not encourage "cramming" to the extent feared, or else knowledge acquired by "cramming" is not as easily lost as has been thought. (3) Retention in the survey course compares favor-
ably with that in courses in laboratory sciences given in the literature. Unfortunately, such data are very scarce in the physical sciences on the college level. (4) Retention figures as given in the literature do not redly record retention, but the sum of the true retention, and the gain in the intervening interval, which is by no means negligible. Use of such figures for retention leads to a false idea of the magnitude of the loss. (5) The effect of further training is to increase the apparent and true retention in the subject studied, but according to the results of the present experiment, to leave the gain unchanged. (6) The subject matter which is best retained consists of principles, theories, their applications, and related facts. That which shows the poorest retention is that which is memorized without the exercise of reason.
The authors wish to acknowledge the assistance of Dean William H. Conley and Miss Alice Griffin, Registrar, of Wright Junior College, who made possible the performance of the experiment; of Dr. Max Engelhart, director of the Department of Examinations of the Chicago Board of Education, in making suggestions as to the method of analysis of data; of WPA Project Number 3702, which authorized assistance in tabulating data; and especially of the students whose cooperation was so willingly given.
