Edwin M. Larsen University of Wisconsin Madison, 53706
Students' Criteria for Responses to Teaching Evaluation Questionnaires
You have undoubtedly been evaluated by your students. The evaluation instrument consisted of a series of questions or statements to which students responded hy checking a box with values of one through five indicating various degrees of agreement or disagreement with each statement. In the statistical treatment of the replies, a mean of the numerical values for each statement, perhaps with a standard deviation, was calculated to two or three significant figures. It is possible that the program used yielded a grand mean for all the statements; thus, in the process you were reduced to a numher purportedly related to the quality of your teaching. Is this what the numher means? I feel that this numher is more a measure of how well your "properties" as a teacher and as an individual match the expectations of the students. But what are the expectations of the students? Actually as I have seen the questionnaires administered, the students' responses must be an intuitive gut reaction to a variety of stimuli, rather than to well thought out ideas. Can the students' responses to the statements he defined more exactly? No one seems to have asked the students just what the criteria are in their minds when they react favorably or unfavorably to these statements, so we do not know. To explore the possibility of defining these criteria more precisely, the following six statements selected by members of a subcommittee of the Division studying the use of such questionnaires were used. These were selected in large part, on the basis of the fact that these statements, or others similar to these, appeared on many of the evaluation instruments used throughout the country. The teacher 1) is always well-prepared. 2) makes good use of class time. 3) has stimulated my interest in the subject area. 4) expects that much time and effort be given to the course outside of class. 5) is interested in the student's progress and is actively helpful. 6) is effective. In actual use the student is asked to respond to the statements by checking whether he: strongly agrees (51, agrees (41, neither disagrees or agrees (3), disagrees (2), strongly disagress (11, or has no opinion (X). (There may he some question as to whether these six statements constitute the six best possible statements, or whether they are worded properly, but this is a separate question.) A group of undergraduate students (40) and a group of graduate students (15) at The University of WisconsinMadison participated in the project. The undergraduates constituted 10% of the two classes from which they were selected. These students were about equally distributed between men and women, and grade distribution (A-F). In general they were not students majoring in chemistry, but were students who were taking a t least one year of chemistry. The graduate students were selected from among our current PhD candidates. They constituted about 6% of our current graduate students. No claim is made that a statistically significant sample of students was obtained. The directions given to both groups of students were the same. Think of one of your undergraduate instructors for
whom you would record favorable responses; now think of one of your undergraduate instructors for whom you would record unfavorable responses. Use a composite of individuals, if necessary. Now, what are the sentences, clauses, and adjectives which best describe the properties of the individuals in each of the two categories with respect to the six statements of the evaluation instrument. No attempt was made to confine the student responses to teachers of chemistry, as the properties being sought were assumed to transcend the nature of the discipline. A request was made that each student arrive at his (her) responses without consulting other participants, so as to preserve the individuality of the responses. I believe that this request was generally adhered to. However, I do feel that the statements themselves influenced the students' modes of response to some extent. Note that the graduate students were asked to reflect on their undermaduate instruction, rather than their current graduate rnstruction. The initial intent was to determine whether the time lapse since their undergraduate instruction moderated the graduate students' responses to their undergraduate instructors sufficiently to show up as differences in the criteria used by the graduate students compared to the criteria used by the undergraduate students. A little thought tells you that the prohlem is not that simple. First of all it must be recognized that by restricting one group to graduate students, a selection factor has been introduced which includes a differentiation in ability, interest in academic work, and other accompanying properties of such individuals. Secondly, whereas our undergraduates have all been exposed to large classes and teachina assistants, a larae fraction of the maduate students came from ~ o n s i d e r ~ h lsmaller y institGtions in which they encountered relatively small classes and individual professors. Be that as it may, the responses from the two groups of students make an interesting contrast. Now what I would like to do here is substantiallv a reporting job. Upon reading and rereading the responses from 55 students to each of the six statements, I could see that I was faced with a dilemma. The unexpected wide variety of statements, and the subtle nuances in meaning of the different replies made it impossible to present all the replies collected. For this reason only the favorable criteria for statements 1 and 6 (Tables 1-4) have been presented. But the problem of transmitting the freshness of the students' thoughts and the germ of the ideas still is present; for in trying to present this information in some organized fashion, I'm afraid that I may have distorted a po&t of view simply by the way the expressions were selected. The reporting pattern which evolved was to select sets of key words and phrases, and group these with others expressing the same concept. The numbers in parentheses indicate the frequency of that response for that specific statement. In the responses of the undergraduate respondents to the statement relating to the teacher's preparedness, note that seven of the students (out of 40) cite what might be called the verbal style of the teacher's presentation. This also appears in the graduate respondents' list but it is not given the same prominence. It is of interest to note that Volume 5 1, Number 3, March 7974
/
163
Table 1. Undergraduate Respondents re Teacher's Pre~aration -favorable
r
Table 3. Undergraduate Respondents re Teacher's Effectiveness
if the teacher
-favorable
kn,.wr rhc nnterml and i* :shlv ru prr+nr 8 1 'I 3 1 fl,.r~hlvm o w h to allow qurrrionn .-I ~ u c w r ehh~lpfuld r m o m t m m n s and l h ~ n n o m? rt,.ir* uh.,, hv la rovrrlne . A inr r o ~ ~ t cd r t the part and fwtrw eulli.rt
.
d
I. f
matter (I'. g) is able to state what is and what is not wing t o appear on exams (11 and stnLesexp1icit1y how thecourse will be graded and subject matter overed I
1
. .
