George G. Lowry
Western Michigon University Kolomozoo, Michigan 49001
The Effects of Quiz Sections on Performance in General Chemistry
In many colleges and universities chemistry class sizes in the hundreds are common, particularly in beginning courses. In some cases bigness is simply an accepted fact, hut in others various devices have been used to provide the students with a closer contact with faculty. One such approach uses the "quiz section" of about 20-30 students. As the economics of colleges and universities change in the direction of greater emphasis on faculty productivity it becomes appropriate to question seriously every aspect of our teaching methods in terms of efficiency. It is desirable. whenever oossible. to studv such techniaues and anproaches on an objective, quantitative basis. This naner renorts on an exneriment which was conducted 'deiiheracely to seek answers to several questions about quiz sections. It utilized a control group and the results were analyzed statistically in an attempt to verify or refute several hypotheses.
were made in class as to the subject matter and types of skills or problems which might be encountered on the next quiz. Also a list of performance objectives was posted a t least 1 wk prior to each exam. Thus the testing of students was as near identical in the two classes (W72 and F72) as was reasonahly possible, since the lecturer wrote all exams and quizzes and provided grading keys, including explicit instructions on partial credit, for quizzes graded by others. The only reasonable differences in the treatment of students were whether quiz section attendance was voluntary or mandatory and who the specific quiz section leaders were. In the W72 class (the control group) 157 students finished the course, and in the F72 class (the experimental group) 146 finished. These classes are sufficiently large and of similar size so that comparisons should he meaningful.
The Experiment
Statistical Method
Chemistry 101 is a first semester course taken by students majoring or minoring in chemistry. It bas no chemistrv and utilizes four lectures per week plus " ~rereauisites . a quiz section. This author taueht Chem. 101 in the Winter Semester, 1972 (hereafter labeled W72), and one lecture section of the same course during the Fall Semester, 1972 (labeled F72). Following the third quiz and the first hour exam in F72, attendance a t the quiz sections was made voluntary and auizzes were eiven during the last 15 min of a lecture hour.. The immediately notGeable result was that only about 10-2070 of the class attended the discussion sessions voluntarily, so several sections were combined very effectively. Results of other aspects of the experiment were less obvious and were suhiected to a statistical analvsis. " . as described below. The lecture, auiz, and exam schedule for W72 and F72 were identical, a n d the actual quizzes and exams were made as nearly equivalent as possible. Announcements
The effects studied here were examined in terms of null h.vpotheses, which are presented in the general form "no simificant difference exists between conditions X and Y " The definition of what constitutes a 5ixnifimnr diiterenre is sub.jecrive and is arhitrarily decidtd hy theinvestlgator. However. it is oossible to obtain a auantitative measure of the degree of confidence in a particular result, so that the onlv arhitrarv decision is what level of confidence is to he considered significant. Various statistical methods have been used to establish confidence limits. but those used in the present study are described in the following paragraphs. When the conditions X and Y correspond to the performances of two groups of individuals, then measures of the conditions are a & d, and p d,. In these, 1; and p are simply the mean performances of the individual members of the groups. The confidence limits, d, and d,, are equal to 3 a, for the two groups, where om is defined for condition X as
*
Volume51, Number 2, February 1974
/
121
p --
97 n(n 1) with x i the individual performances of the n members of the particular group. The interpretation of these values is, that thoueh we do not know the theoretical mean performance of the members of a group, we can say with 99.74% confidence that its true value lies within the range a f d,. To compare the performances of two groups, we obtain the ratios of the performance measures of the groups, namely =
T o obtain the confidence limits of z we utilize the wellknown computational formula for propagated error, which for this quotient is
From this we can say with 99.74% confidence that the d,. More true value of the ratio lies within the range z specifically, if the value of unity lies outside this range, we can reject the null hypothesis and say with 99.74% confidence that there is a significant difference between the performances of the two groups. If the value of unity falls within the range, we will accept the hypothesis, though usually not with the same degree of confidence as with rejection. When the conditions X and Y correspond to two types of performance of given individuals within a large group, then a measure of the relationship is the correlation coefficient, r,,, computed for the entire group by the usual formula
*
The square of this value represents the fraction of the variation of 1, values that is explained by, or related to (not necessarily causally) the variation of x values, or vice versa. In this paper, a value of rXy2< 0.5 will be considered a criterion for acceptance of the null hypothesis that "no significant relation exists betweenX and Y."
Quiz Section Attendance
Ideally, one of the first concerns should be whether a required part of a course affects how well students learn, as reflected in most cases by their performance on quizzes and exams. If we are fortunate, we can conserve faculty time without a deleterious effect on student performance. In order to study the effect of required quiz section attendance on performance, some type of "seasonal adjustment" is needed. ?t is common knowledge that some topics in a given course are more difficult than others and result in lower scores on exams and quizzes. Thus, simply examining the change in performance following the removal of attendance requirements would not necessarily give any valid information. Ideally, one would use a seasonal adjustment factor obtained from experience over several previous classes taught under similar circumstances. Such long-time evidence was not available and is unlikely to become readily available because of rotating teaching assignments, changing textbooks and course content, etc. As an approach to seasonal adjustments, a "performance ratio," P, was defined and computed for all scores. This is mean score obtained in F72 P= (1) mean score obtained in W72 in which both scores are for the quiz or exam over the same material. The performance ratios are shown in Figure 1 in which the length of each bar covers P 30,. The change to voluntary attendance came after the first exam. Immediatelv. the ouiz scores as measured hv the verformance ratio s;fferedLbadly. After 3 wk of this,accombanied by several "sermons" on self-reliance delivered hv the course instructor, quiz scores returned to a performance ratio within 99.7490 confidence limits of unity, though the value of a, after the change was somewhat greater than before. However, little change could be said to have occurred in the performance ratio on exams except perhaps for a very slight increase in scatter, a,, as represented by the length of the bars. To quantify these observations we formulate and test the first hypothesis: Hyp. I : Required attendance a t quiz sections has no effect ongrades. In testing this, we let y he the average P before the requirement of quiz section attendance was removed, and x be the average P after that time. Based on the results in Table 1, the h.vpothesis is accepted for exams and rejected for quizzes. If we assume that the first three auizzes after the change represent simply an adjustmen; period that reflects the chanze in attendance requirements rather than the difference in them, then the hipothesis might be accepted for quizzes. However, this remains speculative. Another aspect might he examined to-supplement the performance ratio results. In Figures 2 and 3 histograms of actual semester average grades for exams and quizzes are shown. These figures do not completely isolate the effect as only 75% of the exam grades and 70% of the quiz grades in F72 were under conditions of voluntary attendance, but they furnish some supplementary information. In both sets of data we see that the histograms become flatter and the modes shift to lower scores in F72 relative to W72. In this regard, the acceptance of h.vpothesis I for
*
Table 1. Test Data for Hypothesis 1: Effect of Required Quiz Section Attendance on Grades Exam eradee
auizmad-
TIME Figure 1. Pertormame ratios as defined in eqn. (1). Solid bars represent hour exams, and open bars represent quizzes. Lengths Of bars include 99% confidence limit ranges.
122
1 Journalof Chemical Education
0 In Tables 1-4, the definitions of r , y, and I a m etatad in the teat aewmpaoying the statement ofthe pertinent hyptheeis.
EXAM AVERAGE, PER CENT Figure 2. Distribution of exam scares during the two semesters involved in the experiment described. Both histograms are normalized to the same total area.
exams must be with some reseroations, but its rejection for quizzes is further justified. Quiz Section Leader
The next questions to he probed statistically deal with whether the identity of the quiz section leader is of any consequence. Students frequently complain that instmctors other than the course lecturer "get them confused" by saying things a different way. We usually point out that hearing several differently worded explanations for a phenomenon may actually help the student gain an understanding, hut many of them remain unconvinced. In fact, some students have said that "several" friends have decided to drop the course because they were confused by their quiz section leaders. To examine this, the second hypothesis is formulated and tested: Hvp. 2: Students tend to drop the course a t the same rate regardless of whether their quiz section leader is the course lecturer. For each quiz section of W72 (in which students were influenced b; their quiz section leader throughout the semester), the fraction of students initiallv enrolled in the various sections who completed the course was determined. Let x be the fraction of students in quiz sections under the course lecturer who did not withdraw from the course and y he the fraction of students in other quiz sections who completed the course. Then we obtain the results shown in Tahle 2. Thus, even though this test involved only four quiz sections of each type it appears that h.vpothesis 2 should be accepted. Some students have felt that those in quiz sections under the course instructor have an unfair advantage over students in other quiz sections. This idea is evaluated in terms of the third hypothesis: Hvp. 3: Students perform equally well in the course regardless of whether their quiz section leader is the course lecturer. In evaluating this and the following hypothesis, averages were computed for all scores in W72 and for the first exam and the first three quizzes in F72, over which the quiz section instructor could have had an influence.
QUIZ AVERAGE, PER CENT Figure 3. Distribution of quiz scares with the same normalization as in Figure 2.
In testing this hypothesis, we let x be the average grade for students in quiz sections under the course lecturer and y be the average grade of all other students. On the basis 3 in of the results in Tahle 3 we should acceDt . hvuothesis .. terms of exams but reject it in terms of Quizzes. Some facultv members have felt strondv that onlv those who teach or have taught the courseshould be a-' signed as quiz section leaders, because others will not have the same "feel" for the course. This concept is evaluated by means of the fourth hypothesis: Hyp. 4: Students perform equallv well in the course regardless of whether Table 2. Test Data for Hypothesis 2: Effect of Identity of Quiz Section Leader on Drop Rate
Table 3. Test Data for Hypothesis 3: Effect of Whether Quiz Section Leader is Currently the Course Lecturer on Grades Exam erades
Quiz eraderi
Table 4. Test Data for Hypothesis 4: Effect of Whether Quiz Section Leader Has Ever Been the Course Lecturer on Grades Exam grades
Quiz grades
Table 5. Test Data for Hypothesis 5: Correlation of Exam Grades and Quiz Grades of individual Students
-
. .-
W-1
Correlation Coefficient, r Pmdictabilitv.
0.583 0.340
SI.n,e\tCr
F?> 0.516
0.266
Volume 51, Number 2, February 1974
/
123
their quiz section leader has ever been the lecturer in charge of the same or a similar course. For the test of this hypothesis, let x be the average grade of students in quiz sections under persons who a t some time had been lecturers in charge of the course and y be the average grade of all other students. The results are presented in Table 4, from which it is seen that hypothesis 4 should be accepted in terms of performance on both exams and quizzes. Correlations Between Quizzes and Exams With histograms of both quizzes and exams shifting in the same qualitative manner from W72 to F72, it seems possible that quiz grades correlate well with exam grades and hence are merely predictors of exam performance without providing any additional information about the student's mastery of the subject matter. This notion is formulated in the fifth hypothesis: Hyp. 5: Quiz grades are accurate predictors of exam grades in the course. This hypothesis was tested by a correlation analysis with the results displayed in Table 5. In light of these results, hypothesis 5 is rejected. Discussion of Resulls On the average, mandatory attendance a t quiz sections seems to have little effect on performance on exams in the course. The effect that did occur on relaxing attendance requirements was a slight broadening of the exam grade distribution and a shift downward of the mode. Statistically the effect was not highly significant. In terms of quiz grades, a similar shift in the distribution occurred, but the average grades were significantly lower when quiz section attendance was voluntary. There is some evidence that the change in attendance requirements may have been largely responsible for this effect rather than the difference in requirements, but no positive, statistically verifiable evidence is available to test this suggestion. By examination of the grade distributions of the two groups (Figs. 2 and 3) i t appears that the performances of the poorer students are affected most by the requirement of quiz section attendance. This is not very surprising, as these students likely also have the poorest study habits and perhaps have not learned to discipline themselves even when their own best interests are involved. Considering these factors, a decision of whether to require quiz section attendance would seem to be based largely on how important it is to save poor students from themselvesthe performances of average-to-good students are largely unaffected. During the semester in which quiz section attendance was required 74% of those originally enrolled completed the course, whereas 79% completed the course when at-
124
/
Journal of Chemical Education
tendance was voluntary. Certainly the slightly poorer performance in the latter case did not appear to discourage students to the extent of withdrawing from the course. Beyond this, speculation as to whether there is a significant difference in the fraction completing the course, and if so what it means, is not justified. Although students frequently complain of a disadvantage if their quiz section leader is not the lecturer in charge of the course, there is evidence that this feeling does not discourage them t o the point that they drop the course. The only evidence that the identity of the quiz section leader influences performance in the course shows that those in sections under the course lecturer did significantly better in this study. Otherwise the identity of the section leader had little effect. Why this effect appeared is still a mystery, as all instructors were teaching the same methods for working prohlems and knew the content of the quizzes before the discussions. They were also briefed as to what material was being covered in lecture a t the time, and had their own copies of the textbook. Of course it is not known whether there would have been a similar result if each quiz section leader had written his own quizzes. In the interests of fairness through uniform treatment this was not done, but in light of these results it mav have been an unfair tactic. several possible recommendations could follow this result. P e r h a ~ seither all or none of the auiz sections should be led by the course lecturer to removethe apparent bias. Possibly having each quiz section leader write his own quizzes would accomplish the same purpose. A highly significant correlation between quiz grades and exam mades for individual students was not found in this study.-~videntlyskills tested by the two were significantlv different. This sueeests that a critical examination should be made reg&&ng the place of the two in the course. Generally the quizzes covered the more critical, basic facts and skills to give the students extra practice with them before taking exams over the same material. On the one hand, exams were multiple choice with only single correct answers, and they included numerical problems as well as concepts, definitions, etc. However, many students have confessed to guessing a t answers on these exams rather than working out a problem and then locating its answer among the choices given. On the other hand, quizzes were graded with no more than 10% the allowable credit being given for the correct answer-the problem must be set up properly to receive most or all of the remaining 90%of the credit. In this author's opinion, both types of testing should he retained in the course, but the value of using mandatory quiz section attendance rather than voluntary "help sessions" is doubtful.
