C. C. Hinckley and J. J. Logowski T h e University of Texas Austin
A Versatile Computer-Graded Examination
A n important factor in successful teaching is the ability to establish an intimate exchange of ideas between instructor and student. Not only is it important for the instrnctor to present his subject to the student in an understandable manner, it is also necessary for the successful instructor to obtain an indication of student response to the subject. If a relatively small number of students is being taught, the nature of student response can be individualized, but even a t this level, examinations can he used as a teaching device. As the size of the class increases, the instructor's individual contact with students decreases, and very rapidly the flow of information from student to instructor is virtually restricted to results obtained on examinations and other written work. Accompanying the loss of intimacy which occurs with large classes is an increased load of clerical work. Grading soon threatens to occupy all the available time! The pressure of this clerical chore has resulted in the development and extensive use of the "multiple-choice" question on examinations in which the student chooses a correct answer from a list of possible answers, rather than generating his own answers; the most common form of question r e quires a choice from among five answers. Examinations of this type can he rapidly graded by hand or by machine. With this type of format it has become possible to give examinations to very large classes routinely, and it will become increasingly necessary to do so because of expanding college enrollments. Unfortunately, these developments enhance the loss of contact between instructor and student. The student cannot phrase his own response to the questions, and the instrnctor finds himself restricted in his formulation of questions by the format. This situation can lead to the use of questions which overemphasize a knowledge of detail a t the expense of general understanding. Clearly, a greater flexibility in question format, which retains the possibility of rapid grading, is desirable. An attempt has been made a t The University of Texas to obtain a partial solution to this problem by using a high-speed computer to grade examinations in the large
general chemistry course. The grading machines commonly in use do not permit questions with more than one correct answer, nor is i t usually possible to weight the questions on a standard machinegraded examination. Both of these factors have been partially accounted for in the examination format and computer program described here.' The student "answer-sheet" is a specially designed preperforated IBM Port-a-Punch card, the answers being indicated directly on the card by a punch made by the student rather than by the usual mark. A portion of the answer card showing a student's responses for questions 1-6 is shown in Figure 1 and the corresponding correct answer array (vide infra) is shown in Figure 2. Thus, the student answer card is used directly as an input card for the computer program. The student answer card I is divided into two areas, the first nine columns being designated for the I student's identification number and the remainder of the card representing answers to the examination questions. One question is assigned to each column giving ten possible answers for each question. The program permits from zero to 9 correct answers for each I A,Figure 1. A portion of the specially question' designed student cord used as input for the program.
'The program described hers was written for an IBM 1401 system, a unit which is often used for data processing. 2 A copy of the program will be supplied to interested readers upon request to J. J. Lagowski.
Volume 43, Number 7 7, November 7966
/
575
YC.IAC*I
TO GRADE
I*C.I*C., Y TO GRADE
ADD OdD7
I I
I I I
TO G U D E
!
Figure 3.
576
/
Tho computer program Row chart.
lournol o f Chemicol Education
the array of student answers with the array of correct answers. In the program described here,zthe computer is the device used to make this comparison. However, there is one additional feature in this program; the answer key for a particular examination not only indirates the correct answers but also assigns a credit value to the answer. The answer key is an array of numbers rather than an array of correct answers. In the examples shown in Figures 1 and 2, the student would get 9 points credit for each answer in question 1; 5 points for response 1 in question 2 but no credit for response 5; 9 points for each of the correct answers in questions 3 and 4; and so on. Thus, in the present program partial credit from zero to nine points may be awarded for any response. The input to the program includes (1) a heading card bearing the course and examination numbers, date, and the maximum number of responses allowed; (2) the key cards for the examination containing the correct answers; and (3) the students' answer cards. In grading an examination the computer essentially scans each student-answer card awarding the credit value for each response as indicated on the key cards. The output of the program is the student's identification number, his number of responses, and his grade. The correct answers from the key cards are placed into the computer's memory as an array of numbers (Fig. 2), that is, the ten possible answers for each of the thirty-one questions or a total of 310 positions. The credit value for a given answer is given by the number placed in the corresponding position of the key. Thus, it is apparent that the credit value for a given response to a question may correspond to full or to partial credit. After the key has been placed in the computer's memory, the program grades each student card in turn. Each student card is read into the computer in Column Binary. This feature is essential to the success of the program since it allows the computer to locate the posi-
QUESTION NUMER
1 ) . 2 ( 3 1 4 1 5 1 6 ) - 1
Figure 2. The corred answer orray (for quedionrinflgure 1 1 showing the relation between o punched answer card and the key stored in themachine.
The student is instructed to indicate, by circling with a pencil, those responses for each question that he feels are correct before punching out any of the responses. This procedure allows him to reconsider his answers, erase, and mark another answer, if necessary. After the student has finished his examination, be punches the circled responses using a stylus which is provided. Alternatively, if mark-sensing equipment is available, the student indicates his answers on a standard answersheet which is then transferred to a punch card by machine. Either punched card then serves as an input card for the computer program described below. Computer Program
The general grading procedure for any "indicated answer" examination involves a simple comparison of
Answers
Quesrionr Complete *he f a l l a w i n g r e a c t i o n s
1. A compound c o n t a i n s 37.5% carbon, 1 2 . 5 9 h y d r o g e n , a n d 5 0 . 0 % oxygen. What i s t h e s i m p l e s t f o r m u l a f o r t h i s compound?
0. CHO
\
\
1.C2H0 2. C.HO. 3
5 . CHy02
'
z
5 . C 2H UO 7. CH.0 9. CH'O CH 3 O2 2 2
2. The e l e n e n f ( r 1 i n Column 8 t h a t would for. 3 c o v a l e n t bonds w i t h c h l o r i n e atoms to f o r * a n e u t r a l
molecule
\
0. NO anaver g i v e n Lithium
i'
2. B e r y l l i u m
3. The e l e m e n f ( s ) i n :o1umn 8 t h a t e x i s t as d i a t o m i c m o l e c u l e s under sfandard conditions
3.
Boron
U.
The e l e m e n f ( s ) i n Column B t h a t would f o r m a compound w i f h w i t h t h e greatest number o f bonds
9. Carbon
5.
he e l e m e n t i n column 8 t h a r would form t h e most p o l a r c o n v a 1 e n t bond w i t h f l u o r i n e
6. oxygen
5. N i t r o g e n
7. F l u o r i n e
6 . The e l e r n e n f ( s ) i n Column B t h a t would nor form bonds to a n y o f h e r atoms 7. The e l e m e n t i n Column B t h a t would f o r m t h e most i o n i c compound w i t h fluorine.-
Figure 4.
Neon Hydrogen
I
A s a m p l e o f a n unknown gas w e i g h s " 2 . 7 g . and occupies a volume o f 5 . " 6 1. a t s t a n d a r d c a r . d i f i o n s . Answer r h e f o l l a w ing questions.
0.
1. 0.2*" 2.
1 3 . Haw many liters would t h i s gas occupy at lQ7OC a n d "20 m . of mercury?
no answer given
3.
15.2 115 4.65
.u. 5. 0.38 1". now many gram m o l e c u l a r w e i g h t s of g a s a r e present i n t h e original\, 5. 1 . 9 6 sample? 1 5 . What i s t h e m o l e c u l a r w e i g h t of the g a r ?
/T::
:0:3
9.
5.18
Typer of questionr which can be graded by the program.
Volume 43, Number 1 1 , November 1966
/
577
tion of each individual punch rather than to assign a punch or a combination of punches to characters that may or may not be valid. The program directs the computer to scan each question column in serial order. The computer, in effect, moves down each question rolumn looking first a t answer number 0, then 1, then 2, and so on (Fig. 3). If a punch appears, the program branches to the corresponding position in the key and the value stored there is added to the student's grade. A s part of this process, the computer also records the number of times the student has punched his card. This process continues until all of the answer columns have been examined. If the number of punches exceeds the maximum allowed, a penalty, the weight of which may be predetermined, is subtracted from the score. This procedure is designed to discourage wild guessing on the part of a student. The student's grade is the total accumulated credit, and the output data for each student, i.e., the identification, total number of responses, and grade, are printed. This simple version of the grading program also accumulates the total credit awarded and counts the number of students taking the examination so that an average grade may be calculated. A second version of the program now being developed provides a distribution of answers for each question and performs statistical analyses on these data. The development of an input technique and a computer program to grade multiple-response questions leads immediately to a rapidly graded examination with a format possessing a flexibility that arises from two sources: the possibility of allowing more than one answer per question, and the fact that partial credit may be awarded for any answer. Typical Questions
The most obvious type of question which can be devised for a computer-graded-examination format is an extension of the usual multiple-choice questions, but with a greater variety of choices (Fig. 4, question 1). However it is also possible to group questions that are related and have related answers (Fig. 4, questions 2-7, 8-12, and 13-14), or answers that are equivalent; thus, the program would give credit for either or all responses
578
/
lournol o f Chemicol Education
for questions 2, 3, and 11 in Figure 4, and the same response could be used for questions 3 and 5 (hydrogen) or 9 and 12 (on'). The same technique can be used to examine for simple recognition as well as for the ability to recognize the equivalence of two or more answers which may be of a specific and/or general nature. Moreover, in the latter instance it is possible to weigh the answers according to the discretion of the instructor. Using the program presented here, this technique can be used for questions which have numerical answers (Fig. 4, questions 13-15), giving partial credit for answers whch represent partial knowledge of the principals used to solve the problems, a possibility which cannot be realized with the standard machine-graded examinations. Conclusions
Computer grading provides an opportunity to use examinations as teaching devices for large classes. Presently, under the best of conditions, conventional machine-graded examinations require sufficienthandling so that the results are delayed beyond the point where the examination is fresh in the student's mind. Since the output of a computer-graded examination can include both the individual grades as well as a statistical analysis of the resultsfor each question, it is possible for the instructor to detect rapidly the questions which presented the most difficulty for the class and to apply the necessary remedial procedures at the next class meeting. The program described here is being modified for use on a more versatile computer (CDC 6600) which allows a student to record numerical results directly into an answer card. This technique will permit an even greater flexibility for the rapid grading of examinations on subjects which are best exemplified by numerical problems. This use of the more versatile computer will also permit a running record of student examinations t,o be kept on magnetic tape. We are grateful to the Research and Development Center for College Instruction in Science and Mathematics of The University of Texas for partial support in the development of this program, and to Mrs. Darleen Ferry, a representative of the IBM Corp., for assistance in the initial stages of programming and card design.
