The ESSAY EXAMINATION in CHEMISTRY' FRED P. FRUTCHEY U. S. Department of Agriculture, Washington. D. C.
AND
B. CLIFFORD HENDRICKS University of Nebraska, Lincoln, Nebraska
A
RECENT inventoryZof limited spread indicated that in the series of examination sets examined the essay type of question was used in ninetyone per cent. of the sets as compared with its nearest competing type which was used in sixty-three per cent. of them. This but bears out the experience of most teachers that the essay question is, in spite of recent interest in other examination forms, more used than all other forms combined. Such general usage by a group of teachers who pride themselves upon reasoned conduct must not be without justification. One advantage of such a test form is that its cost of construction is low; however this does not imply that the cost of scoring is low. Essay responses made by the student require skill in the use of English. Many chemistry teachers seem to assume that there is an intimate correlation between scientific understanding and linguistic expression. Taking an examination by marking a paper with Presented before the Division of Chemical Education at the ninety-seventh meeting of the A. C. S., Baltimore, Md., April 6, 1939. a H ~ m n r c g AND s Hamonrr, "Examination practice in general college chemistry." J. &EM. EDIJC.,15, 179 (1938).
+'s and -Is or T's and F's, in their judgment, loses much of its value to those concerned. The essay question often calls for organization of the answer as well as its proper phrasing. The older teachers do not consider it possible to test for this skill in organization by any other method than by the essay response. Most teachers feel they are somewhat experienced in the preparation of essay test requirements. This experience makes teachers better qualified to prepare such examinations than they are to prepare the newer forms. Their comment would be, "Essay questions are easier to prepare." Since many teachers of college chemistry give only a part of their time to the teaching function they do not readily change methods especially if such a change requires loss of time from the otherthan-teaching part of their job. Giving an essay examination seems, to many college teachers, so easy as compared with the elaborate set-up required for some of the standard tests. Its very informality invites its use. The teacher saunters into the classroom, stops a t the desk, extracts a ream of paper, asks a student to pass i t out while he writes the
questions on the board, and the students begin writing a t once. So i t has been for generation after generation of students. No wonder such a fixed practice is slow to yield to change, especially when some of the limitations of the new forms are still so evident. However, the most ardent advocates of the essay examination are ready to confess its weaknesses. The most glaring of these, and yet one sometimes not well appreciated by its users, is its lack of objectivity. When the same examination paper in mathematics3 can draw grades all the way from twenty-eight to ninety-two per cent., depending upon the teacher grading it, there is certainly room for improvement in that respect. Truly i t appears that the grade on the paper may depend quite as much upon the scorer of the examination as upon the person taking the examination. It is saidZ that the typical essay examination has from five to ten questions. It is obvious that a subject matter sampling permitted by so few questions bas a very limited range. However good the character of these questions the least that can be said is, "The evidence which they elicit is insufficient." It hardly needs to be added that always when the student has to spend so much of the allotted time in a manual act, writing, and in organizing and phrasing the information needed in addition to remembering and understanding it, the extent of the information that can be expressed by the student is going to be correspondingly circumscribed. "The principle of comprehensiveness is clearly violated in the essay type of examination."* Two other criticisms of essay examination practices are: the influenceof irrelevant factors upon the scoring is very considerable and the uncertainty as to the meaning of the score after it has been determined. Any teacher of experience knows all too well how difficult it is to keep good penmanship from influencing the examination grade on a chemistry paper. When such a paper is graded there is reason to ask, "Does the grade indicate good or poor chemistry, good or poor penmanship, good or poor spelling, or good or poor English?" Since the essay examination is so generally used and has a number of attractive assets it is pertinent to give some constructive attention to its shortcomings. There may be two ways of caring for these. Either they may be corrected by modifying the examination form, or, if the defect is related to the examination's use for some certain objective, that objective may be better measured by some other means. It may be proper, then, to consider some uses of examination^^,^ with especial attention to those most effectively accomplished by essay examinations. But uses of examinations are almost invariably gaged by their service in measuring the attainment of
objectives of the course. Studies7have shown and the experience of teachers would also indicate that the objectives of different teachers are different. In fact, the objectives of the same teacher may change from year to year, or with the locality in which that teacher is working. However that may be, the objective must function if it is to serve. And the present criterion of functioning is the test. The purpose of this paper is to consider the place of the essay examination in testing. As previously stated, there are a number of qualities characteristic of the good examination. Some important ones of these may be re-enumerated as validity, reliability, objectivity, and practicality. By validity of an examination is meant that quality which has to do with the degree to which it measures the thing it is intended to measure. In the past teachers have, perhaps nafvely, assumed that the essay question is a valid method of measuring any objective of a course. Such an assumption is surely open to question. The authors are venturing to say that for some objectives there are no known valid methods of getting evidence, except to observe the person a t work, or to ask him to reply in his own words to a problem situation. For example, if you wanted to £ind out his approach in solving a problem, one could observe him in solving a problem or could describe a problem and ask him to solve it by a written response which is the essence of an essay examination. However, the description of the problem may not always be free from misconstruction. As an example, suppose the student were asked, "Discuss the electron theory." The student chooses a purpose and method of organizing his answer but it may not be the method the grader has in mind. Does the maker of the examination and the grader expect the student to discuss the electron theory: (1) so that the layman can obtain some understanding of it, as, for example, for publication in Harpers', Lqe, or for a talk before a group of business men; (2) as a high-school teacher's explanation of the theory to high-school students; (3) to show the instructor how much the student knows about the electron theory; and (4) to show how well the student can organize the information in a logical manner. The student may use one or more types of organization, (a) historical, (b) from the standpoint of particular problems, (6)fundamental assumptions and definitions, ( d ) basis of known facts and interpretations of these facts, or ( e ) some combination of these. Evidently the examiner needs to take his own examination before he gives i t to his students. He needs to ask himself, "Just what do I expect my students to say in answer to each question, and have I made the questions so clear that they know what I intend and that they will not misinterpret it? What kind of beg STARCH AND ELLIOT, "Reliability of grading work in havior on the part of general chemistry students conmathematics," School Rmk, 21, 254 (1913). "ANG, A. R., "Modern methods in written examinations," stitutes evidence that they are reaching each important objective? Specifically, what kind of evidence shows Houghton Mifflin Co., Boston, Massachusetts, 1930, p. 74. HENDRICKS. B. C. AND 0. M. SMITH, "Service tests for chem- that students are acquiring a fund of chemical inforistry." Sch. Sci. Math., 35, 488 (1935). HENDRICKS. B. C. AND F. P. FRUTCREY, 'The uses of exami' SMITE.0. M.. "Acceptedobjectives in the teaching of general nations." J. CHEM.EDUC.. 15, 237 (1938). college chemistry," ibid., 12, 180-3 (1935).
mation; what evidence shows that they are learning to use chemical facts and principles in solving problems new to them; what evidence shows that they are learning to interpret experimental data; what evidence shows that they are developing the ability to plan experiments for testing promising hypotheses; what evidence shows that students are skilled in the use of laboratory technics; what evidence shows that they have an interest in chemical phenomena? Each objective, in turn, must be clarified in terms of the evidence and the kind of student behavior which expresses the ~bjective."~The question then needs to be so formulated and phrased that it will unmistakably bring the written responses from the student that will be certain evidence of the degree of achievement in the objective measured. In an effortto attain this precision of phrasing many teachers need to pive conscious attention to the precise meanings of such terms9 as classify, compare, criticize, define, describe, explain, illustrate, interpret, justify, outline, review, summarize, verify and perhaps many others which are used over and over by teachers in preparing essay examination questions. To obtain a reliable estimate of the degree to which a student has acquired information, studies have shown that it is necessary to present a large sample of questions. Under ordinary conditions i t is not feasible to present students with a sufficiently large number of questions and have them answer the questions in their own words, either orally or in writing. This is an important practical problem in obtaining a satisfactory estimate of a student's progress in the direction of each objective. The test should measure accurately and consistently whatever i t attempts to measure-it should be reliable. The reliability is dependent upon the number of observations the examiner makes of reactions from a given student. Essay tests require so much time to administer that they create the problem of including enough questions to obtain a reliable estimate of students' achievements in an objective. There is either the need for longer periods for such examinations or for a frank recognition of the necessity of a cumulative record of a series of such tests before too much reliance is placed upon their implications. As another means of improving the reliability of an essay test, i t is suggested that the examiner take what may be called a cross-section sample. If the test is to cover the year's work the test should sample various aspects of the year's work. If the objective is the use of scientific method in all fields of human endeavor, the test should sample situations in other fields as well as those in the subject matter of the course. These two demands mean a longer test for reliable measures. However, there is evidence which shows that the size of the sample can be reduced by obtaining a representative sample of behavior rather than a chance sample. The non-agreement among readers of essay examina-
-
8 F ~ m c n eF. ~ ,P. AND B. C. H ~ ~ ~ n r c"Constructing ns, and validating examinations," J. CHEM.EDUC.,15, 40-3 (1938). LANG,1 0 ~ cit., . p. 67.
tion papers has been menti~ned.~I t is suggested that improvement in this characteristic of such examinations may be attained when each grader considers and judges an answer to a question as evidence of achievement in a particular and the same objective. If the question is to measure application of principles, then each grader should consider the answer and decide how well the answer shows ability to apply principles and not how much information the student has. If the latter is desired then the test should be scored for that purpose. Homeylohas shown the effect of such a procedure upon the objectivity of the essay examination in chemistry. Probably the poor objectivity of essay examination papers is due in a great measure to irrelevant factors. Good penmanship predisposes the reader to leniency to poor chemistry. Non-Websterian spelling has, possibly, been responsible for poor grades for many otherwise fairly good students of chemistry. It is obvious that if readers of examinations will consciously seek to minimize the influences of such irrelevant aspects of the paper there will result a more objective evaluation of it. It is further suggested that just as the precision of a thermometer is improved by the smallness of the units etched upon its stem, so the objectivity of an essay examination may be improved by finer test units, i. e., more specific test questions rather than coarse units of the general type. There is something to be said for essay examinations as a practical, usable means of testing. They take less time to prepare; they cost less to mimeograph; they may be easily made to include local emphasis of subject matter; and some students are accustomed to them and hence are less disturbed emotionally by the familiar than they would be by novel, unusual forms. On the other hand, unusual administration timedemands are made if essay examinations are to be acceptably comprehensive and dependably valid. Their use entails a heavy burden of reader time requiring skilled readers if reasonable objectivity is attained. Their grades or scores are less specific in their implications than are those of other types. Yet, to date, this form is without competition as a means of securing direct organization of the science. So intimately are organization and expression related to knowledre and understand in^ a " that i t is uncertain how much of the latter is possible without the former. If oral or written expression is essential to understanding, then i t would certainly be unfortunate to delete oral or written expression from its r41e as an element in the training of the student in the course. It follows that i t would be equally unfortunate to discard oral or written expression entirely from tests for information and understandings in the course. Until such time as the uncertainty stated in the first sentence of this paragraph is cleared the essay examination form should by all means constitute an important part of all wellbalanced examination programs.
-
lo HORNEY. A. G.. "An objective essay examination in chemi s t ~ . " S ~ Sci. h . Math.. 34, 380-7 (1934).
