Correlation between Instructor Evaluation and Student Test Scores
R. R. Martin University of Western Ontario London, Ontario N6A 587 Canada
The validity a n d significance of s t u d e n t evaluation of teaching ability continues to generate controversy ( 1 , Z ) . Even t h e m o s t quantifiable aspects often produce differing results (3-5).T h i s is t h e case with a t t e m p t s t o correlate s t u d e n t evaluations with student performance as measured by student scores i n subsequent examinations. T h e present p a p e r i s a n a t t e m o t t o correlate s t u d e n t eenerated teacher evaluation scores ro final e u a m i n i ~ t i o nscores using t h e first year c h e m istry clnsant t h e University ,,i\Vestern Ontario. Experimental
Design
The first year chemistry class a t Western consists of approximately 1000 students divided into seven lecture groups of approximately equal size. Each lecture section is the responsibility of a different lecturer. The sections use a common text, write eommon examinations. and receive identical treatment in laboratories and tutorials.
at the end of the year. The lecturers are evaluated late in the year. A multi-question multiple-choice format is used. The ratings are reported on a scale of 1-5,l being poor and 5 excellent. This paper concerns itself only with the ten questions which directly evaluate the lecturer. These are questions of the type: (1) Are the lectures well organized? (2) Does the lecturer stimulate interest? (3) Is effective use made of overhead projectors, demonstrations,
etc.? The average mark on these questions, the overall instructor rating, is reported in this paper. A problem arises from the fact that individual student evaluations are anonymous which renders a direct correlation with final grades impossible. The procedure adopted was to compare the behavior of each class section on a pre-test with that of the rest of the students in the course in order to learn if any individual class section differed significantly from the bulk of the students. Thesame procedure was employed with the final examination results. The assumption used in this paper is that should i n s t ~ c t 0ratings 1 produced by the students he correlated with student examination marks, then some change in the performance of individual classes
' Since the pre-test is multiple choice and many students are un-
familiar with this format, errors such as incorrect entry of student number are eommon.
relative ta the remainder of the students between the pre-test and the final examination should be clearly evident. The comparison of section pre-test and final examination scores was carried out by computing the section average on each test and omparing the results to a similar average obtained by pooling all the remaining sections.
and Discussion The bulk results are shown in Table 1. Only students for whom complete records were available are shown. I t should be noted that improperly filled pre-test cards1 are rejected from the study. Table 2 simplifies the presentation of results. This table showsno significant difference between the pre- and post-test scores of five of the seven sections comoared to the bulk student oerfarmance even thoueh " there is eonsidekahle variation in the instructor ratine Set lim U i, atw~nnlm-in that thrrln3s perhrmsntc i\ cigni~iranflg ~ T h u is an evening lerrurc vctltm lwlm thit d t h e i w t d t h rlns-. containing a large number of part-time students. Since the difference persists (becoming neither much worse nor much better) teacher inspired success or failure can be ruled out. Finally, section G scores significantly higher an the pre-test but returns to the class norm on the final examination. There are several possible explantions for this, but there is no evidence in this data t o suggest that instructor evaluation is one of them. These data strongly indicate that there is no clear relation between instructor effectiveness as evaluated by the students and the final examination marks of those students. It should be clearly recognized that these results apply only to a large freshman class where most factors except for the actual lecturer (i.e. text, examinations, etc.) are common to all students. It is probable that other facctors involved in the transition from secondary school to university are dominant in determining student performance. It Results
~~~
~
Table 2. Simplified Results Section
instructor Ratlna
G
3.4
Section Pre-Test -CIBSS Pre-Test
3.7Y
Section Final -Class Flnd
0.82
Significant. he probability mat medifferencein scares wassignificantwar determined by artandam !-test using the 0.05 level of significance ss criteria.
Table 1. Overall Data
SECTION
OVERALL A B
C 0 E
F
G
lnrtruction Rating
3.8 4.5 4.7 2.5 2.7 3.8 4.8 3.4
#ol Students
782 106 117 112 103 125 120 99
Senion pre-test wore
Standard deviation
58.95 59.00 61.09 59.57 52.87 58.51 59.18 62.22
14.67 13.24 14.69 14.39 13.50 15.49 15.72 14.49
Class pe.tert s o r e with sect.
removed
Standard deviation
Section flnal examination score
58.94 58.57 58.84 59.87 59.03 56.90 58.47
14.89 14.64 14.72 14.63 14.52 14.59 14.64
65.64 64.45 68.38 67.11 60.68 65.82 65.06 66.26
Standard deviation
16.69 16.02 17.71 16.92 17.73 16.08 18.09
,
Final examination scorewith sect. removed
deviation
65.72 65.06 65.19 66.28 65.49 65.63 65.44
17.12 17.20 17.27 17.01 17.05 17.02 17.32 16.99
Standard
Volume 56, Number 7, July 1979 / 461
is clear that the student/leeturerratio (12511)is such that prductive interaction cannot be expected. Under such conditions, this result is not surprising.Finally areassuchas student enthusiasm havenotbeen explored because they are not sufficiently quantifiable far inclusion in this study.
462 / Joornai of Chemical Education
Literature Cited M,,Rdin,B , , J . con. Educ,,S, 5i19731, i21 Bryson,Rebeecs,J,Edue, Rer,,68,12 i1974,, (31 smith, ~ y l R.,J. a ~ e s~. 8 t h~duc., . 8,195 (18771. I41 March. Herbert W. eta]., J. Educ Psych., 61,833 (1975). ( 5 ) Moody, R., Mad. Lang 5.60.454 (19761. (61 Martin, R. R.. Blizzard. A. C., Humphreya, D. A,, Srikamesvaran, sesrmenf Test, J. CHEM. EDUC., 52.808 (19751.
K..A Chemistry As
