Frequent Quizzing, the Final Exam, and Learning Is There a Correlation? Mark B. Freillch Memphis State University. Memphis. TN 38152 Several years ago, a survey (I) of my general chemistry students indicated that they believed being quizzed freuuentlv would not be an important factor in their learnine I found this surprising, but subsequent inerevealed that manv of these students had confused "learning" with "anticipated course grades". More recently, other students also responded in terms of the course grade and were concerned about the negative effect that poor quiz grades would have (2); there was no indication that more than a few were aware that preparing for frequent quizzes could have a positive effect on their study habits. Concurrentlv with the first survev. .. several facultv members.. nartic. ularly those in the humanities and the social sciences, revealed that thev also felt that freauent auiazina had little impact upon learning (I). Since these faculty d G make the appropriate distinction between grades received and learning, I was disturbed by this lack of faith in the pedagogical value of frequent quizzing. There is evidence in the literature (3, 4, 8b) that weekly quizzing may have a beneficial effect upon achievement, but does this represent true learning? The project described herein was initiated to determine the impact of weekly quizzing upon student performance in eeneral chemistw. " ~ i f f e r e nmodels t of learning stress different aspects of the learning process (5). Behavioral models (561, as their name implies, stress learning as a behavioral response to environmental conditions and emphasize the conditioning process. Cognitive (5c,d) and information processing (5e,f) models are concerned with the mental structures that organize and process information; in harmony with the structures and processes it hypothesizes, each model seeks to modify the instructional nrocess. Information ~rocessine " models. in particular, tend to emphasize the process of problem solving; as in other cognitive models, they recognize the need to tie new knowledge into a framework of existing knowledge and are concerned with teachine novices the basic strategies for problem solvingilearning used by experts. Obviously, learning is regarded as a very active process, though the type(s) of activity varies (with significant overlap) among the models. A student who follows a good study regimen incorporates aspects of each model: repetition and practice, active recall (5d), connecting new information to old information, connecting explanations of new phenomena to explanations of phenomena already understood, and efforts to discern eeneral a ~ ~ r o a c hto e ssolvine oroblems rather than the association dfa specific approa&th a specific problem (if "titration", use "M.V"). One iustification for giving exams and quizzes is that they encourage students to study: the more students study, the more they will practice skills and digest concepts and the more readily those skills and concepts will be recalled and utilized. Not only does testing serve this noble goal, but frequent testing may encourage the student to study on a regular basis, consume information in small nackaees. and freauentlv eneaee in active recall. Then, a t ihe en2 of a semesteh st;dy, a c k n u lative final is eiven because (a) i t allows students who have done poorly on earlier tests demonstrate that they have since learned the material and (b) i t is expected to catch
he mi&^.
those students who try to memorize critical information shortly before an hourly exam (and from whose minds this knowledge sublimes soon thereafter). Though i t may not be valid (see below), many faculty believe that most students who tw to cram before a comprehensive final will be overwhelmed by thelarge amount df materialand that onlythose students who have been learn in^ the material throughout the semester will fare well on such an exam. Project Design
This studv has attemnted to discern the impact of frequent quizzing upon the learning of chemistry'iu the first half of a traditional two-semester introductorv. seauence. . The comprehensive final exam was chosen as the measuring instrument. All ~tudentaendured the "normal" complement of general chemistry exams and quizzes. In addiGon, the students were divided into three groups; one group was required to, one group was offered the opportunity to, and one group was denied the opportunity to take supplementary weekly quizzes. Final exam scores for these three groups were systematically compared in each of four semesters to determine the effectk) of freauent auizzine. The underlvine assumption was that performkce i n the Gnal exam was & accurate reflection of the amount of chemistm meaninefullv learned (5c). The results of this investiga$on bringthat assumption into question. General Chemistry 1 (Principles of Chemistry I ) at Memphis State University is tauaht in the traditional 1ectureAab format. Faculty .give.the lectures; graduate or advanced undergraduate students assist in the laboratories under the overall supervision of a faculty member. Each lecture section, consisting of 50-80 students, has only one lecturer for the duration of the semester, but several lecture sections are offered. The comprehensive, all multiple-choice final exam is eiven - to all students a t the same time. The students chosen for participation in Project Quiz were (unknowinglv on their part) self-selecting: anv student who registered fo; one of m i two lecture sections became a project volunteer. A traditional grading scheme was used throughout the project: four major (20- to %minute) quizzes (lowest scoredropped), three lecture exams, a final exam, and laboratory work. In addition, approximately one-third of the lecture students were required to take, for credit, weekly lo-minute quizzes based upon the prior week's lecture material, one-thirdof the lecture students were askedto take the quizzes, but no course credit was assigned, and onethird of the students were not given the opportunity to take the weekly quizzes. (No attempt was made to compare a learning environment in which quizzes were frequentiy given to one in which they were absent. I was concerned that the complete absence of quizzes would have a truly deleterious effect, and there was no wish to put the students a t risk. Thus. this studv has been solelv concerned with the imnact of frequent quizzing upon a "normal" semester.) I t was convenient to divide the students into experimental groups according t o laboratory sections and to give them the quizzes therein. I authored a different quiz, consisting of Volume 66 Number 3 March 1989
219
multiple-choice and long-answer/long-calculation questions. for each lab oeriod each week: the l a b o r a t o ~ instructors administered the quizzes a t the beginning of each period and were resoonsible for eradine them. Graded ouizzes were taken, an2 they were returned in iab one weec afte;being not discussed in lecture unless the students requested; they rarely did. Eight to 10 such quizzes were given each semester from Fall 1985 thkough Spring 1987, contributing approximately 9%to the overall course grade when taken for credit. All students knew whether the lab section to which they had been assigned would he taking the quizzes and whether or not the quizzes would bear credit. The students were not free to choose their exoerimental eroun. Thev knew that the quizzes would cove; only lecturemaierial akd how much of a contribution the quizzes would make, when credit-bearing, to the final course grade. All students were told that a study involving quizzing was being conducted, but detailed information about the project was kept to a minimum. Quizzing did not beain until the third week of the semest e r . - ~ r e ~ u k t l ~ , were not given during the week of a major lecture exam, the week of one of the two lah exams, Thanksgiving week (because not all lab sections met) and the last week of the semester. Lectures immediately preceding such weeks were peppered with appropriate announcements. Although no effort was expended to ensure that all students took all quizzes, make-ups were occasionally given (to the "credit" and "no credit" students alike). It wasimpossible to ensure a t the beginning of eachsemester that the active and control groups were equivalent. For example, pre-med students might have been overly represented in Monday's lab because they all had biology lab on Tuesday and the pre-engineers might have been overly represented in Tuesdav's lab because thev all had draftine on Monday. ~ h e r e f o r 4a "quiz-credit" section and a "quizkocredit" section were chosen from the same lab oeriod whenever possible. Data on the overall grade point averages and declared majors within each group were compared as the projectneared completion in order to detect any grossdifferences among the groups, and none were found. Inorder to detect the effect of frequent quizzing on performance on the final exam, grades on the final exam were compared among the three groups for each semester using a one-wav analvsis of variance (ANOVA). (6a). . . Whenever the A N O V ~indrcated differences among the three groups, a Scheffe multiple comparison test (66,7a) was undertaken to identify the groups that differed from each other and the significance of the differences. Before discussing the results aGd their implications, let us consider some questions about the above design: Were there differencesamone the students that could account for any experimentally ohsewed dkferences? Individual student grade point averages (GPA's) were used to calculate an average GPA and standard deviation for each experimental group each semester; these were compared via a Student t-test (6e) or anF-test (analysis of variance) (6d). There were no significant differences among the groups of students within any one semester or among semesters.The distribution of student majors in each of the experimental groups was considered. Though the actual numbers of students in each major varied from semester to semester (32 pre-professional (premed, pre-dental, pre-pharmacy, etc.) in Spring 1987 versus 16 preprofessional in Fall 1986, for example), those students were fairly distributed among the three experimental groups in each semester [lo, 12, and 10 pre-professional students in the three groups in Spring 1987 and six, four, and six in Fall 1986.1 Would students not taking the quizzes make greater efforts at studying because they feared that the students who were taking the quizzes would have an unfair advantage and skew the grading curve? In the final week of the Fall 1986 semester, students were asked to resoond anonvmouslvto a auestionnaire.The resnonses (2) showed that virtuallvnone o> the students were concerned ahdut , thispmsibility.'rhosrwhodidthinkthrqt~irzesrouldhaveaneffect ware more concerned with the p ~ ~ ~ l h i lthat r t v low quiz grade*w d d hurt their overall course grade than with the likelihood that consis~~~
220
~
~
~~~
~
~
~
~
~~
Journal of Chemical Education
tently preparing for quizzes would help. Could the students who were not taking the quizzes obtain copies from their friends and fine-tunetheir studying accordingly? Is it necessary to consider the results of this study in light of a factor such as "test (or testing) familiarity"? The questionnaire indicated that quiz-sharingwas not afactor. Had sharing been significant, it would have served to bring the no-quiz and quiz-no-credit groups closer together; this would not have undermined the main thrust of the study. In addition, sharing would have familiarized more of the students with the "testing style" typical of the course, a factor which would have had a positive rather than a negative impact. The aim of this project was to see whether frequent quizzing (for credit) would lead to improvements in learning, perhaps by helping students improve the quality of their study time (hy giving them more guidance as to their weaknesses) or simply by prodding them to study more regularly. Therefore, it would not have been desirable to have some students perform better than others only because they were more familiar with the testing style used. Under the conditions of this investigation, it is not likely that the three experimental groups would have differed significantly in "test familiarity", and eliminating it as a concern was one reason for selecting the finalexam as the measure of student learning. By the end of each semester, all students, including those not taking the weekly quizzes, were familiar with the way the tests were organized, the wording of questions, the material that was emphasized, etc. In addition, the final exam differed from all the other tests that the students had taken in that it was the product of a group effort: questions were chosen and worded by all of the faculty who had lecture sections in the course, with each faculty member holding limited veto power. Would frequent quizzing have as much effect upon students with good study habits as upon students who need extra guidance andlor prodding? Were experimental and control groups composed of these students in different proportions? Many of the students who take introductory chemistry at our institution have poor study habits. The results of this project reflect how these students have been affected. At manv urban. commuter institutions..~ evew lower divi~-~~~ sion class has a Large number of students who must develop their study skills. This project was concerned with the effect of frequent quizzing upon a population typical of this type of institution. It is also likely that a student's study habits and skills are reflected in his or her grade point average. As discussed above, the GPA's of the active and control groups were considered and compared. Could test "burn-out" affect this study? There was no evidence for this. I was concerned that students could become discouraged, perhaps disillusioned,by all the quizzingltesting,thereby losing the sharpness, honed by tension, that a testing situation creates. That apparently did not happen, though the survey conducted in Fall 1986 (2) revealed that the students who took the quizzes for credit were, on the whole, unhappy with their situation. (Perhaps the word "hostile" most aptly describes shout one-third of them.) Despite this, generalizingfrom the Fall of 1985,when the for-creditstudents completed better than 99% of all of the quizzes given and the nocredit students completed 85%of the total, virtually all of the students did submit to virtuallv all of the auizzine. Is the final exam an appropriate loul for assessing the amount of chemistry learned during the semester? Perhaps not. (See below.)
~. ~~
Resuns Although a total of 448 cases have been considered (see Table 1). data have beenobtained from onlv421 individuals. ~ w c n t ~ - s e v students en repeating the couise have been included twice. Table 2 summarizes the results ohtained over four semesters. Both the total data and the data remaining after the students who received A's and F's in the course were eliminated from consideration have been studied. Moderate deviations from "normal" classroom procedures may have little effect in particular upon A and F students, and inclusion of the data they generate may significantly skew theresults. This is of particular concern since there were approximately equivalent distributions of A and F students in all of the exoerimental groups except those that differed most from their peers in Spring and Fall 1986. I n Spring 1986, the quiz-forcredit group bad a statistically significant lower exam average than the other two groups, and it had no A students and five F students. In Fall 1986, the quiz-no-credit group had a statistically significant lower exam average than the other
Table 2. Projed Qulz-All Students
Table 1. Number of Students Partlclpatlng Each Semester
Final Exam Means
Cum. SMsms
Fall 1985 No Quizzes Qulzzes.No Credit Quizzes.Credit Tdal
Spring 1988 No Quizzes Q~izzes, NO Credit Quizzes,Credit Total Fail 1986 NO ~uhres QU~ZZBS.NO Credlt
QUIZZBS. Tdal
Credit
Spring 1987 No Quizzes Quizzes,NO Credit Quizzes. Credit
Total
44 36 41 121
Percent
Percent
-
36.4 29.8 33.9 100.0
-
36.4 66.1 100.0
56 24 30 110
-
50.9 21.8 27.3 100.0
-
50.9 72.7 100.0
41 34 54 129
-
31.8 28.4 41.9
100.0
31.8 58.1 100.0
27
30.7 34.1 35.2
30.7 84.8 100.0
F
Semester Number Students Ratio Prob. Fail 1985. Spring 1988* Fall 1986' Spring 1987d
121 (44-36-41)' 110 (56-24-30) 129 (41-34-54) 88 (37-30-31)
0.157 2.041 1.881 1.587
85.5% 13.5% 19.4% 21.1%
NO
NO
Quiz
Credit
Credn
57.7 126.,+ 125.4104.4
59.3 124.4 112.8104.2
59.2 111.1121.9 117.2
dlffBrenEOB m M g me m B B gIOUp8. aDlneren~ appears at 0.15 mnfldence. No hlmr dlwerentlation at 0.25 cMldence. "OIffermcaappem at 0.22 mnfidenca. 'NO dMWBnEB6 appear even at 0.25 ConfidsnEB.
'NO
30
31 88
100.0
T h e pmbabllw of obtalnlng an Fratloat less1 as large a s t h e observed one whmhe
..
weekly quizzes for credit, respectively.
..
Table 3. Proled Qulz-BCD Students Onlv Final Exam Means F
Semester
Fall 1985. Spring 1986' Fa11 19885 Spring 1987"
two groups, and it had only two A students and seven F students. Could the distribution of A a n d F students in these cases have caused the observed differences or could the distribution of A and F students have been a consequence of the frequent quizzing? The latter appears unlikely. (It has no statistical support.) Table 3 summarizes the results when only the B, C, and D students are considered. Perusal of Table 2 may lead to the conclusion that quizzing had a significant effect upon the three experimental groupsin three of the foursemesrers, theexreption being the Fall of 1985. Consideration of the data generated by H,C, and D students only, as tabulated in Table 3, diminishes the magnitude of the differences, but superficially significant differences still appear to have been generated during Spring 1986 and Spring 1987. However, the overall directional impact of the quizzing is impossible to discern. Considering all of the students (Table 2). taking auizzes for credit appeared to have no effect (Fah 1985?, negative effect ( S ~ r i n 1986). e the same effect as not taking the auizzes a t i l i (Fa1 1986) and a strongly positive effect (spring 1987). When onlv B, C, and D students are considered (Table 3), there seems to have heen no effect caused by quizzing during two semesters (Fall 1985 and Fall 1986), a negative effect in Spring 1986 and a positive effect in Spring 1987. The analysis of variancei reveals the significant differences. In the Sorine and Fall of 1986. the differences amone the experimenh Goups appear significant, but the dire; tions of these differences are inconsistent and the skewed distributions of A and F students may have affected them. When the A and F students are removed from consideration (Table R), significant differences among the three groups amear to exist only in Swing 1986 and, even then, a 1 in 11 chance remains that there ;s no real difference. In that
a
'The analysis of variance was performed using procedure SPSS, Inc.. 4 4 4 N. Michigan Avenue, Chica-
ONEWAY. SPSSIPC+. go, IL 80611.
Fa
F'
NO
NO
Number Students Ratio Prob.
Quiz
Credit
Credit
93(31-29-33)' 74 (37-17-20) 102(35-25-42) 68(18-23-25)
60.1 129.4' 123.7 108.3
58.7 131.2' 118.0 115.5
59.8 115.8119.0 118.0
0.151 86.0% 2.525 8.7% 0.586 57.0% 0.884 41.8%
spring, the students who were subjected t o the weekly quizzes for credit did more poorly than the other two groups. Data for the third (and last) lecture exam were analyzed (considerine the B., C.~ within lecture . ~ onlv ~ -and. D students) ~ ~ ~ sections to relate these grades to final exam performance. (A total of eieht sections nartici~atedin this investigation.) One lecture section from-spring 1986 was excludeh from this analysis because there were too many students in the control group (28) and too few in the two active groups (atotal of 13, with only five in the no-credit group). One lecture section from Spring 1987 was also excluded because of an unbalanced distribution of students among the three groupseight students in the control group, two students who took the testsfor credit and 13who took them without credit. The analvsis of the six remainine lecture sections showed. as m i g h be expected, strong p&itive correlations (at the 99% or better confidence level) between the grades on the third exam and the grades on the final exam. However, in four of these six sections. an analvsis of variance revealed that there was far more uniformity bf performance on the final exam across the active and control groups than there was on the third exam. [In the fifth lecture section, there was a 60% probability that the differences among the three groups on the third exam could be explained by chance alone, but only a 40% probability that the differences on the final could be explained that way. However, the analysis of variance was hampered by differences among the internal variances of the three groups (only a 25% probability that all three groups ~
- ~ . ~
Volume 66 Number 3 March 1989
221
~
~
had the same internal variances). In the sixth lecture section, the students differentiated themselves on the final exam, but not on the third exam: the quiz-for-credit group did much more poorly on the final than the other two groups.] Discussion
In the earlv 1970's. Georee Lowrv studied how nerformance i n g e n i r a l c h e n h y &affeckd by atrending'recitation sections (80). Reauired attendance did not affect exam grades hut may have affected quiz grades. (Exams and quizzes were given during lecture periods.) Lowry was forced to reject thchypothesinthat quiz grades were i n acrurate predictor of exam grades in his course. At about the same time, Martin and Srikameswaran investieated the i m ~ a r of t frequent quizzing upon exam perforGance (86). ~ i s t r i h u t e d over an academic year, 14 sets of mastery-type quizzes supplemented "normal" testing. Compared with the control, the hieher final exam scores of students assiened to take the qu&zes were statistically significant. Like h art in and Srikameswaran, we are concerned with the effect of the testingprocess. However, quizzes in this study are traditional, not mastery oriented. Although Lowry focused on the benefits a student may accrue from attending recitation sections and I focused on quizzing, our two studies led to similar statistics about testine and similar conclusions about studv habits: they also raised similar questions. The current investigation reveals that additional weekly quizzes, either required or optional, will have very little impact upon the final exam grades of students who are required to take four or five major (20-30 minute) quizzes and three full-period lecture exams during a 14 or 15-week semester. Whiie interesting, the conrlusi& is of obviously limited value. As alluded to above, its importance lies in the questions it raises and in the implications of the answers to those questions. Since the literature indicates that the quiz is a tool that acts as both a guide and a prod to studying (9),the obvious question becomes one of how much quizzing is enough? How much time can he spared, and how much time must he snared in the classroom? Ten reasonablv short or "DOD" quizzes will devour approximately two hairs of lecture &me. as will four or five"maior"quizzes, yet Gavnor and Millham have shown ( 4 ) that t k e so spent can he~produrtive.(Studen- who were subjected to weekly quizzes in a psychology course Derformed better on the final exam than did students who tobk only a midterm prior to the final.) Few lecturers have the luxury of literally stealing time from lab in order to quiz lecture material, and not every program offers recitation sections during which quizzes may be given. If the goal of testing is pedagogical rather than for assessment, the results of this project imply that i t is indeed possible to spend more time testing than is necessary. Beyond a certain point, additional testing apparently produces no discernible increase in the retention of information ouer the short term. This study has not identified that point, though i t is likely that little is gained by going beyond four or five 20-minute quizzes that supplement two or three lecture exams. Would two or three ouizzes work as well as four or five? For assessment, prohadly. T o facilitate learning? Possibly, but we cannot answer that question. Perhaps we shall he ahle t o provide an answer in the near future. (It would have been informative to have given all of the students a Standardized ACS Exam one &two semesters after the completion of the course. Unfortunately, this could not be done.) We now come to the most important and vexing issue raised by this investigation: Is the final exam a proper tool for assessing learning in general chemistry? Can students study "to the final" t o a greater extent than traditionally assumed? Is consistent effort for an entire semester truly required for a good grade on the final exam or will several days of concentrated study before the test be suffi222
Journal of Chemical Education
cient? I believe that the lack of any real differences among the experimental groups allows fo; one of two conrlusions: (1) the extra discipline and suidanre offered by the weekly ouizzes was ineffective either because the course contained sufficient testing situations without the additional quizzes or the students were sufficientlv motivated hv other factors or (2) most of the students, p&ticularly those who did not take the auizzes. were able to studv and cram to the level demanded by thk final examination and, by extension, are able to do the same for final exams in other courses. This would defeat a major purpose for giving a comprehensive final exam. A grade on a cumulative exam is expected to reflect what the student has learned for the long term. I t is certainly possible that the course as t a u ~ h to t the control group c%.aina a level of testing sufficient to produce the maximum degree of active participation hy the students whoare susceotible to such an influence. Tosee whether this were the case; the performance of the students on the final exam was compared with their performance on the third (and last) lecture exam. A strong positive correlation hetween the erades on the two exams was exnected and was found. ~ i v i that, n the correlation between the quizzing project and the grade on the third exam was compared with the correlation Getween the quizzing project and the grade on the final exam. A similarity in the two correlations could indicate that it was appropriate to assume that the students were about as fully active in their studying throughout the semester as their limitations (~ersonal.social.. etc.). would allow. However, if the correlatik between quizzing and the third exam did not mirror the correlation between ouizzine and the final exam, then some other explanation wokd likely be rewired. unfortunately, the comparison hetween the third lecture exam and the final did not help explain the apparent uniformity of final exam grades in tl& study. In most of the lecture sections, the no-quiz group was ahle to match the performances of the other groups on the final, but i t did not match the performance of the quiz-for-credit group on the third exam. This leads to reiection of the sueeestion that all three groups of students maintained a continuous, equally high (or low) level of studvine throuehout the semester. The students who took the &zzesufor credit quite possibly did maintain a higher level of day-to-day inuolvement in the course. The improved performance of the "no-quiz" group (the control group) on the final exam relative to the two active groups may very well be the result of intensive preparation for that specific examination. Perhaps i t is easier, driven by the pressure of an upcoming final exam, to memorize facts and calculational approaches than many of us assume. An alternative exnlanation. that the Dressure studehts feel when taking a f k a l exam acts as a b e a t leveler, makine i t verv difficult for most of them to nerform to the best of their abilities, cannot be supported.-Students who have done poorly throughout a semester mirror that performance on the final. Given the uniformity of performance among the three experimental groups in this instance,.the control group would have had to improve relative to the active groups in order for all three groups to have done equally well. If pressure were the factor, we would have to conclude that the control group was less nervous, or felt less pressure. than the two active erouDs and this is hiehlv uniikely. This study strongly suggests that there is a ne& for more research into the function served by a cumulative final exam in a course such as general chemisiry.
--
Summary
This investigation has attempted to discern a relationship between subjecting students to frequent quizzing and the students' grades on the final exam. The grade on the final exam was selected because i t was thoueht to be an indicator of the amount of General Chemistry 11earned. Though this project was begun with the expectation that the students
who took weekly quizzes for course credit would, as a group, reach a higher level of achievement on a cumulative final exam than the students who did not take weekly quizzes for credit, such was not the case. Final exam grades were the same for students who took weekly quizzes for credit as for students who took quizzes hut not for credit and as for students who did no; take weekly quizzes at all. There are two possible explanations for this: Either a course that contains four or five major (20-minute) quizzes and two or three lecture exams already contains sufficient testing for guidance andlor prodding, or college students are sufficiently adept a t test taking that they can study to the level of afinal exam, even if that final covers the material for an entire semester in which each week builds upon the prior week(s). The latter explanation is supported hy a survey conducted during thisstudy (2).It revealed that many students whodid not take the weekly quizzes for credit, but who had taken the major quizzes andthree lecture exams, believe that they do not study enough and that using weekly quizzes to prod them and/or guide them is a good idea. Yet, on the final exam, these students performed a t the same level as the students who were quizzed frequently. A final exam grade may imply more about what is remembered for the short
term than what is learned, and a course grade based upon a heavily weighted final exam may imply far more about the potential a student has for learning chemistry than about the amount of chemistry already learned. Acknowledgment The author is grateful to John Y. Euhank, Jr., Dean of Admissions and Records, M. S. U., and Dan M. Phillips, of Institutional Research, M. S. U., for their assistance. Literature Cited 1. 2. 3. 4. 5.
Frsi1ieh.M. B. J. Chsm. Edw. L983,60,21&221. heilieh. M.B., unpublished result8 (fo besubmitted). Duty,R. C. J. Cham.Educ. LSB,59,21&219. Gsynor, J.: Mi1ihsm.J. J Educ. Pwch. 1916,68,312317. (a) Anderaon, J. R.: Kcs&m. S. M., me. Tvtariola in Leorninsond Momory;Freemao: New Yark,19%. (b) Horoarifz. F. D. Erplaring Devolopmsntol Theories: Brlbaum: Hillsdsle,NJ, 1987;Chepter3. (c) Novek, J. D. J. Chem.Educ. l984.61.6W-612. (dl Anderaon. B. F. Casnitioe Psychology. Academic: New York, 1975: Chapter 6. (el Wsldrop, M.M. Science 1981,237,1564-1567. IO Stewart, J. H.:Atkin, J. A. J. Re& Sci T m r h 1w2. ,9 ?,1,-?22 ~
0. Glass,G.V.; Stenley.J. C. Stotlsticol Mefhoda inEdwotian ondPaycholosy; Prsntieb HeU: Englewaod Cliffs. NJ, 1970; (a) Chapter 15. (b) pp 3-97, (el pp 292-300. Id)
Volume 66 Number 3 March 1989
223
