A Student Evaluation of Teaching Techniques "None of Them is Unimportant" Mark B. Freilich MathlScience Division, Junior College of Albany, Albany, NY 12208 As Robert Travers points out in "Criteria of Good Teaching" ( I ) , it did not become necessary to formally evaluate teacher effectiveness until students were required to go to school (either as a result of compulsory education laws reeardine elementarv through secondarv school uunils or the c o m p u k n of the market a a c e in reg& to higher Ldncation). The effective teachers in earlv Greece were easv to recognize: they were the ones (such as ~ocrates)around whom those who wanted to learn would gather for discussion and argument. When the University of Paris opened in the tenth century, the effective teachers were the onlv ones who were uaid-the students paid fees directly to the professor and only those teachers who could attract enoueh students uroved able to support themselves. Obviously, the situation is very different today and, under the dual protections of tenure and faculty unions, the teacher finds himselflherself in a much more secure situation. However, while the teacher may have gained a great deal, the student, the ultimate consumer of the teacher's product, may have lost a great deal. Recently, an increasing number of our students have recognized that they are consumers and have a right to demand a quality product. This, combined with greater administrative pressures to ensure that shrinking budgets are not used to support poor teaching (poor teaching is very had from a public relations standpoint!), has led to a phenomenal growth in research into effective teaching and the use of various methods of evaluating individual teacher effectiveness. (At this point, let me emphasize that this paper is neither in defense of nor an attack upon the practice of student evaluations of faculty.) Only a few years ago, Richard Meeth felt compelled to write that "in spite of the numerous teaching improvement centers, research on teaching is nominal and incidental in most colleges and universities" (2). While this may have been true then, i t is certainly less so now. A bibliographical search performed in late 1979 usine ERIC (3) on certain asuects of teacher effectiveness turneb up one reference in 1966, two in 1967, nine in 1968, and 44 paEes of references (with an average . of almost three references per page) in 1977. Sadly, upon reading the literature on improving instruction, we encounter a great deal of conflicting advice. For example, we are told that the use of behavioral objectives (4) help students learn, hut we are warned that this may "discourage independent thinking on the part of students" (5). We are told that frequent testing is beneficial (61, but students complain that they are being tested so frequently across all of their courses that they have little time to digest the material. As a result, we begin to wonder which of the many things we could do are likely to be the most important. Until recently, this information has either been lacking or merely the chaff of other studies (7). Attempts have been made to correlate teacher effectiveness with non-instructional activities (such as uarticioation on committees, work as a consultant, etc.) (8j. OneLstudyby Martin (9) investigated the correlation between the evaluations different sections of a large course gave of the lecturer and the final exam averages of those sections. Several studies have dealt with teacher characteristics (IO-19) and some (11, 13,181have indicated that certain general qualities are more 218
Journal of Chemical Education
important than specific techniques. These studies of teacher characteristics and the more general analvses of student students;espond favorably to different teaching formais (19) and that faculty responses to student evaluations have "contributed to the improvement of college teaching" (24). may remember the papers preReaders of THIS JOURNAL sented in the Symposium: Student Eualuation of Teaching in March 1974 (25). Whereas the general tone of the svmuosium was somewhat critical of thestate of the art,' no participant questioned the importance of the actual items used in written questionnaires. In order to clarify student responses to the items in such evaluations, Larsen (25e) took six oftenused items (such as "the teacher is always well prepared") and asked students to describe the properties which would lead to favorable and unfavorable resoonses to those items To extend this type of analysisLandin an effort to get some idea of which of the variables are most important in instruction, 107 students in a general chemistry course for freshman engineering students at Purdue University and 106 students in a general chemistry course for liberal artslscience students at California State University, Hayward, were given a list of the 28 items duplicated in Table 1and were asked to evaluate them. The students were told to choose the five items which were most important in helping them learn and also the five least important, though i t was emphasized that none of the items was necessarily unimportant. In addition to choosing the five most and least important items, the students were asked to rank the items in order of importance. The surveys were conducted in the sprint, with the students nearing and/or recitation leaders. Additionally, several faculty members and a few experienced graduate teaching assistants (for a total of 23 teachers) in a wide variety of disciplines were asked to evaluate these items. The raw data were treated as follows: each time an item was chosen as most important it was assigned a value of +5; if second most important, +4, down to +1 for the fifth most important item. When an item was chosen as the least important of all, it was assigned a value of -5. . etc... to an asHignment of -1 for the f i f t i least important (or, alternatively, the twenty-fourth most important) item. Thus, if all of the students thought that connecting the material to he learned with their own future well being (item b ) was the most important item, it would accumulate 213 X +5 or +I065 points. The correspondingly least important aid to learning could accumulate a maiimum totalof -1065 points. An item to which the students were relatively indifferent would accumulate 0 points as would one which was ranked equally high and low by the respondents. Results Of the questionnaires which were distributed to the stu-
'
In contrast, see reference (24, which lends strong support to the reliability and validity of students evaluations.
dents, 192 were evaluated, and 21 were discarded because they were not completed properly. Table 2 lists all 28 items in order of decreasing importance as indicated by the total accumulated points of each item. Two separate listings are given: one for the results of the student survey and one for the results of the teacher survey. (More detailed data, including separate
data for the engineering and the liberal artsiscience students, may be obtained by writing to the author.) For the large fraction of readers of THIS JOURNAL who were active in higher education during the late sixties and early seventies, either as students or asfaculty, some of the results of the survey may come as a surprise. However, the general
Table 1. ItemsThought to Help Students Learn Students learn best when: They understand why the subject is being taught They can connect the material with their own future well being They feel they have some say in their own education They are allowed, within limits, to follow their own intere'sts individual initiative is not severely suppressed They are told what is to be learned and what is expected af them A Certain schedule is established and adhered to reasonably well They are provided with study problems, of varying degrees of difficulty. on new material and the opportunity for review of these problems is soon after exposure to the material. [Problem solving at different ieueis of Bloom's tax on om^]^.^ Thev,~ are~.ouizzed freouentlv ~. . Tnr, are pro" ucd b% i n promp! feecao. an 'me r dcne~emcnts(prompt., qraooo nomeHorrs and r i n m r for v m m p e Tnew is r s w i ~e as vema reinforcomenl Demonsirat onsl There are reading assignments as well as lecture They are told the goals and given learning objectivesr They are given concrete and varied examples Abstract conceots are attached to models
There are connections among the concepts and principles they are asked to learn New ideas can be related to already established ones New information is presented in a logical progression They are given practical (working) rules as well as theoretical concepts Distractive mannerisms of the teacher are held to a minimum The teacher is able to adapt the stress heishe puts on an item to the importance of that item They (the students) feel free to ask quedions or challenge the ideas of the instructor (either before. after, or during class) They feel confident that the instructor knows hisiher material They perceive the instructor as a person who is interested in them and their welfare They perceive the instructor as being well organized (as indeed helshe must be) They try to explain to or teach one another (2) (aa) They can "hash ofl' ideas in small groups (bb) They are allowed to test their ideas for themselves (to experiment wilth their ideas)
#The phrases in parentheses appear asthey were seen by the students. The phrases in brackets are either verbal explanations gtventothe students who asked for such or are intended to clarify the item for the reader. a y~nc..~ e ~wo r k 1956. , *Bloom. Benjamin S., IEdilw), "Taxonomy of Educational Objectives. Cognitive Domain." David ~ c ~ Go., rTho~ghonly the author. as lecturer or asrecitationclass ieader, provided the class with objectives. approximately 75% of all of the respondents in thesurvey were in the author'sclans ai the time the surveys were conducled.
Table 2.
ltems In Order of Relative Importancea
Student Evaluation -
Teacher Evaluation
-
Students learn best when 1) New information is presented in a logical progression. [r] 2) They are given concrete and varied examples. in] 3) They are provided with study problems . . . [h] 3) They are provided with prompt feedback an their achievements . . . [jl 31. The" feel confident the instructor knows the material. I .wl. 6 Tne, tee free to crla lenqe :lie nslr.c.of 3nO K .as6 q.esl ons 1 . I 6 Tnel arc, l o 0 Hnbt to .earn an0 *ha! . 3 expeci+:d of tnem I I ] RI Tnere are COnneCllOnS among the con:r,?'s an0 pr nc p e r ole) are aqkw t o learn. [pl 8) They can connect the material with their own future well being. [b] 10) New ideas can be related to already established ones. [q] 11) They understand why the subject is being taught. [a] 12) They perceive the instructor as being well organized. [y] 13) They perceive the instructor as a person who is interested in them. [XI 14) There is visual as weil as verbal reinforcement. [k]
.
*b
15) individual initiative is not severeiy suppressed. [el 15) The teacher is able to adapt the stress put on an item to the importance of that item. [u] 15) Abstract concepts are attached to models. [a] 18) There are reading assignments as well as lectwe. [I] 18) They are allowed, within limits, to follow their own interests. [ d l 18) They are told the goals and given learning objectives. [m] 21) They feel they have some say in their awn education. [c] 22) A certain schedule is established and adhered to. [g] 23) They try to expiain to or to teach one another [r] 24) They are allowed to test their ideas for themselves. [bb] 24) They are given practical (working) rules as well as theoretical conceots. -~,~~~is1 26) They are quizzed frequently. [ q 27) They can "hash ouP' ideas in smaii groups. [aa] 28) Distractive mannerisms of the teacher are held to a minimum. [t] ~
..
Students learn best when 1) They feel confident the instructor knows the material. [ w] 1) They are told what to learn and what is expected of them. I f ] 3) They are provided with prompt feedback on their achievements . . . [ I ] 4) They are provided with study problems . . . [h] 5) They perceive the instructor as weil organized. [ y ] 5) They perceive the instructor as a person who is interested in them. 1x1
10) They are given concrete and varied examples. [n] 11) There are connections among the concepts and principles they are asked to learn. [pl 11) They are allowed, within limits, to follow their own interests. [dl *b
13) 13) 13) 13) 17) 17) 17) 17) 17) 17) 23) 23) 251 26) 27) 28)
.
They feel free to challenge the instructor and to ask questions. [v] They can connect the material with their own future well being. [b] They understand why the subject is being taught. [a] They are given practical (working) rules as weil as theoretical concepts. Is] There is visual as well as verbal reinforcement. [k] individual initiative is not severely suppressed. [el The teacher is able toadapt the stress put an an item to the importance of that item. [u] They are told the goals and given learning objectives. [m] Abstract concepts are attached to models. lo] They areailowed to test their ideas for themselves. [bb] They feel they have some say in their awn education. [c] They can "hash out" ideas in small groups. [aa] Thev are ouizzed freouentk. . . .lil, There are reading assignments in addition to lecture. [I] They try to explain to or to teach one another. [z] Distractive mannerisms of the teacher are held to a minimum. [I]
.
.
Items given the same numerical rank had total point accumulationswhch were insignificantlydifferent. items above this point had positive total accumuiationn. lferns below this point had negative total accumulations, ltems clustered aboutthis point had statistically insignifcant differences in total accumulations.
b
Volume GO
Number 3
March 1983
219
tone of the results does lend suovort to those who claim that today's students are rather pra&cal, goal (job) oriented, and willing to look to the facultv for, or actuallv demand from the
portant, 70 (!) as being among the five least imoortant). is therefore important that we recoenizi the existence of this attitude on thebart of our students and other faculty and that we make it clear to the students iust whv thev are auizzed
their own interests (item d),or to &wing &em to feel that thev have some sav in their own education (item c ) than to having confidence in the instructor (items u , w, z,and y ) and in the way the course is structured (items f , h , j , n , and r). (Each of the items c, d , and e mentioned above actually accumulated significant negative totals.) The rather significant dependence upon teachercontrol of learning lends a &eat deal of support to Dudley Herron's suggestion (26) that students need a reason for learning; that the teacher must "establish a need to know" on the part of the students. Both teachers and students agree that i t is important to have confidence that the teacher knows the material (w),that there are study problems followed by a prompt review (h), that there is prompt feedback in general 0') and that the students must he told what is exoected of them (f).However, bv far the item was placed in the bottom five by fewer studeitsj was that new information be nresented in a loeical urogression (r). While this was important to the teachers, it onlftied for seventh dace! The teachers attached considerable imoortance
-
thirteenth. Apparently, the mechanical workings of a course are more imuortant than the uersonal ones as long as the teacher dispiays some degree of interest in the students. The students also thought i t extremely important that they be provided with concrete and varied examples (n). While we all recognize this as critically important in a course such as chemistry, i t is interesting to note that the cross-section of teachers included in this study ranked that item tenth. Ohviously, other faculty must be made aware of the importance of including examples in their presentations and we, the science faculty, must take cognizance of the fact that approaches which are essentially built-in facets of our subject are not second nature to the college community a t large. As a matter of fact, the teaching of general chemistry (and chemistry in general) lends itself very well to most of the items thought of as most important by the students. Some interesting results were hidden as a consequence of listing the items in the order of total accumulations. For example, item b , connecting the material with their own future well being (perhaps meaning a good grade in the course?) was rated as the most important item by more students than any other item, yet it ranked only eighth overall. Obviously, this is extremelv imuortant to a significant fraction of any given group of s
