Joseph Casanova'
California State College 10s Angeles, California 90032
I
I I
An Inrhuctional Experiment in Organic Chemistry The use of a student response system
It is a widely accepted premise that small class size provides the most effectiveinstructional environment. The principal element undergirding this concept is the capacity, in a small class, for immediate feedback of information from the student to the instructor, so that progress can be monitored and immediate adjustments can be made in the pace and content of subsequent lecture material. As class size is increased, the student becomes a recipient of, rather than a participant in, the lecture. The greater the passivity of the student, the less benefit he is likely to derive from the lecture. Yet, the pressure to teach increased numbers of students within ever more severe budgetary limitations virtually demands that class size he increased-an action antipodal to optimum instructional effectiveness. With an objective of simulating, in a class of moderate size, the student-professor interaction characteristic of a small class, we examined the effectiveness of a commercially available electronic student response system in the one-year organic chemistry lecture sequence during the three quarters of academic year 1969-70. The System
A 50-station student response systemZwas installed in a medium size lecture room. The system, pictured in Figure 1, possessed the following characteristics (1) The lecturer could remotely project or mitnually display multiple choice questions (2) The correct response to the question was automatically entered in the console of the system (3) Each student wan required to select a number at his station (Fig. 2) corresponding to his choice of answer to the question (4) Responses were recorded as "correct" or "incorrect" both at the student's station and according to individual student at the lecturer's console (5) Apercentage summary was displayed for the lecturer, indicating the fraction of students who chose the correct answer (6) The student who choseincorrectly was able to subsequently search for the correct answer at his station, for his own edification (7) Student's initial selections were permanently recorded on a paper tape
The Experiment
The object of the experiment was to compare the performance of two classes, of about 50 students each, one of which utilized the response device and the other of which served as a control. These classes received lectures from the same lecturer (this writer). The basis for comparison was performance on conventional examinations and quizzes. A third class, of smaller size, was conducted simultaneously3 using the same
! Figure 2.
Student station.
lecture material, problem assignments, quizzes, and examinations. Students were initially unaware of whether or not they were enrolling in an experimental or a control section. After class conflict problems were resolved, one section was chosen at random to he the experimental section. The lecture notes served as a basis for the preparation of approximately 200 multiple choice questions, which were prepared for projection as 35 mm slides, each coded to the correct response. At the end of each unit of information or each concept (usually at 15-20 min intervals) students in the experimental section were asked to respond to a question regarding the immediately preceding lecture material. The questions were designed to query immediate comprehension and did not usually require elaborate 1 Address during academic year 1970-71: Chemical Center, University of Lund, P. 0.Box 750, 5-220 07 Lund 7, Sweden. Instructoscope, Teaching Aid Model 150-A, 1%.V. Weatherford Co., Glendale, Calif. The lecturer for thrs section of 26 students was Professor Stanley H. Plne.
Volume 48, Number 7, July 1971
/
453
Identify A in the reaction:
hut consistent superior performance. That this difference may not be real is suggested from the data of Table 3, which shows that a significant proportion of less qualified students-students who normally would withdraw from the class-remained on in the experimental section. To the extent that this is true, the average of the experimental section would be depressed proportionately. Unfortunately, the magnitude of this effect cannot be evaluated. The students' perceptions of the effect of the experiment was sought through questionnaires administered during the last lecture of each quarter to the students in the experimental section. Some comments, selected for their poignancy'and not for their bias, are reproduced in the Appendix of this report. The most significant indicator of the acceptance of the experiment by students was response t o a question: "If I had a choice now between two similar sections, one with and one without the response system, which would I choose?" Over 99 percent of students opted for the response system. Conclusions
Figure 3.
A typical queslion.
deductions. A typical question is shown in Figure 3. About one minute was allowed for response. If the class response was less than 50% correct, the same topic was reviewed again immediately, from a diierent point of view. I n order to maintain the same pace in all lecture sections, an equivalent period of time was allotted in the control groups for questions and answers. However, in these sections no student was called upon who did not volunteer to answer the question. The Results
All sections were reqllied to complete the same 10 problem sets, 5 quizzes, 2 1-hr examinations, and final examination in each of three quarters. The average percentage scores for each section in all of these categories is shown in Table 1. The distribution of grade assignments based on these scores is shown in Table 2. According to College policy, students are permitted to withdraw without penalty from a course until the end of the fourth week of classes. 'An interesting result is found when the total number of withdrawals is examined for each class. These results are shown in Table 3, and indicate a significantly lower withdrawal rate for the experimental sections. One potential difference between experimental and control sections was the distribution of students among various majors. Table 4 summarizes these data, showing that the difference between the two large sections was not large, but that the small control section contained a significantly larger fraction of chemistry majors. Discussion
It can be quickly adduced by an inspection of Tables 1 and 2 that present experiment failed to produce an overall improvement in the performance of stCtdeuts in the experimental group, as measu$~dby conventional written examinations. Indeed, in all categories evaluated (Table 1) the control section exhibited a small 454
/
Journal o f Chemical Education
The present experiment does not demonstrate a significant improvement in the performance of students who are regularly and "individually" questioned regarding the subject of the lecture. Neither can it be Table 1.
Percentage Scores for Three Sections Averaged over Three Quarters
--
Lecture section Control-large Control-small 70.1 61.9 43.8 46.6 49.6 44.4 52.2 48.8 54.8 50.8
Category Exp.-large Problem sets 64.3 Quiszes 42.4 Hour examinations 44.8 Final examination 48.9 T n t a l avernee 51.7
Toble 2. Percentage of Letter Grades Assigned for Three Sections Averoged over Three Quorters
-
a
Incomplete and withdrawal grades Table 3. Percent of Students Withdrawing from Three Sections Averaaed aver Three Quarters
-Lecture sectionEx-p.Control- Controllarge large small Percent of students withdrawing
10.2
16.8
20.0
Table 4. Percent Distribution of Students by Maior for Three Sections Averaged over Three Quarters
-Student major Chemistry and biochemistry Other acimre
Exp.large 24 76
Lecture sectionControlControllarge small 19 30 81 70
responded positively; strong support for the experiment, in the form of graduate assistance for grading problem sets and an und~rgmduntrreader for nhninistmtive durics freed the lecrurer's timr to meet frcaur~ttlvwith -... small groups of students. The value of the latter point cannot be overemphasized. The response device proved of significant value to the lecturer, as a means to constantly monitor the effectiveness of his presentation. On numerous occasions the failure of a majority of the class to comprehend a particular concept led to a re~t~atement of that concept, and to a more effective presentation. ~
0
I I I I I I I I I I ( 10 X) 50 70 90 CORRECT MACHINE RESPONSE ("1.3) . .
Figure 4. Relation of questions to molytical mental process required for exominationr.
said that class size was an important factor within the limits of this study. One possible defect in the experiment lies in the nature of the questions asked. That such questions are only slightly related to the analytic mental process normally required for examinations is demonstrated by the data of Figure 4. The correlation between performance using the machine response system and overall performance in the course is low. Some Additional Obsewations
Although not demonstrable from the numerical data, it is contended that the overall performance of all sections which participated in the experiment was qualitatively superior to that of other comparable classes of students, in the experience of the writer. This observation can be attributed to a combination of several factors: the degree of preparation and coordination required by the experiment exceeded that normally accorded the basic organic lecture sequence; all students, whether in a control or experimental section, were keenly aware that they were participating in an experiment in the improvement of instruction and
~
~ A ~ ~ ~ - - ~ - "
Appendix
Below are tallied class responses to a series of questions asked during the last meeting of class to examine the student's perception of the response device. Did I learn more in lectu~e? Affirmative, 23; Negative, 11; No answer, 3. " . . . I took 301A last apring and dropped.. .this time. I feel t h a t t h e constant reinfaroement.. .olesred many doubts.. ." ". . . i t pointed out whvhat I had t o give further study.. ." ". . seemed t o clear u p any question.. ." "...cleared up the point.. ." " . . . i t gave. ..an indioation of whether I understood.. ." ". . m a d e ma think and concentrate more. . ." ". . I oannot honestly shy t h a t i t helped. . ." ". ..Yes, realizing t h a t questions ~vouldbe asked.. .made me more responsive. . ." "...bhcking up of material with questions.. .=onfirm knowledge. . .see.. .deficiency. . ." ". . .seemed t o be "thinking" about the msterisl.. ." " ...I don't believe I learned more, but I became more aware of the areha I needed t o oover more thorauphly.. :' ". . . i t did help me know.. .misoonception about the material.. ." ". . .able t o pick up concepts easier.. .I listened more.. ." ". . d o n ' t think the device.. . increased the hmount I learned.. . t o justify.. .time i t took. . . " ". . m i s oonoeptions.. .,"ere dispelled. ..immediately not wben h test oame.. ." ". . . better understanding is obtained.. ." ". . . i t is like revieving.. ."
Did I listen more intently? Affirmative, 30; Negative, 7; No answer, 0. "...I found myself a t times listening for questions which could be used on the machine.. ." ". . .peer group oampetition is sotive.. "...trying t o learn or absorb what thelecture rves sboutinatead of just mntmgit down. . . " ...I certainly listened more.. ." ". ..possible for me t o apoly what t h e instructor waa lecturing in. . . i n a problem.. ." ". . O n l y for psrticu1srs. . ." "...knowing t h a t I would be oalled upon t o respond.. ." ". . . a s smatter of neoessitv.. . .ves.. . ." ". . . I don't believe I Listened anv more oarefullv than if there had been no answering machine.. . " ". . .St first-yes.. ." ". . .No. Machine didn't soare m e i n t o listening more. I t was tbose quizzes s n d exams t h a t kept me attentive. . ." " ...Yes.. .knowing.. .response mill be required . . .increased. . .awareness. . . "
:':
Volume 48, Number 7, July 1971
/
455
