In the Classroom edited by
Applications and Analogies
Ron DeLorenzo Middle Georgia College Cochran, GA 31014
Crime in the Classroom Part III: The Case of the Ultimate Identical Twin David N. Harpp and James J. Hogan Department of Chemistry, McGill University, 801 Sherbrooke St. W., Montreal, PQ H3A 2K6, Canada
Background In previous articles (1) we described our experience in the detection and prevention of cheating on multiple-choice exams. When students are confronted by evidence of cheating, they often adopt the excuse that “…the answers are similar because we studied together.” While this may appear reasonable, more careful evaluation shows it to be dubious at best. The likelihood of exam copying on multiple-choice tests usually involves evaluating wrong answers, particularly those answered identically (2). This parameter is called “identical wrongs” or “exact errors in common”. The rationale is that while two students studying together may both learn what is correct, they do not study incorrect responses. Recently (1a), we considered exams taken by several pairs of identical twins who studied together. It seems plausible that genetically identical persons might make more exact errors in common than other students. However, we found that, while identical twins wrote exams with similar grades (± 2%), there were “normal” numbers of differences in their answers consistent with those of tens of thousands of other pairings of students.1 Case and Discussion Recently, a curious case confronted us during a final exam where multiple version exams were employed as part of our university regulations. Two students (“S” and “Z”) were “flagged” by our computer program (Signum)2 as having very similar exam papers.3 The students were confronted with this information and stated that they had studied together. Eventually however, both students had misgivings over the situation and confessed to storing a memory calculator in the washroom, which was accessed by the weaker student (“S”) during the exam. Both were allowed to rewrite the exam. Unknown to them, they were given a new exam (92 questions) but with the same questions and answers scrambled. This situation provided a unique opportunity to gain insight into how much information is retained when writing an exam 3–4 months later. Would the two students again write similar papers? Much more importantly, would the same student writing the same exam three months later write a paper statistically similar to his earlier one? In effect, this is the case of the “ultimate identical twin”.
482
Intuitively, it might be expected that most of Z’s exam answers would be quite similar because he prepared once again. In the case of S, who admitted taking most of the answers from Z, the expectation would be that the results would be much different. Therefore, we compared the relevant combinations of performance: S with Z on the exam on which they cheated (S1,Z1), each student before and after (S1,S2; Z1,Z2), and the two students on the retake (S2,Z2). The results are collected in Table 1. Entry 1 is an “average” set of values for an exam of this length and overall grade (ca. 72%). Normally the ratio of exact errors in common to differences (EEIC/D) for more than 99.9% of all pairs is < 0.6, with the bulk of values < 0.3. Entry 2 shows the performance of S and Z on the exam in which they cheated (S1,Z1). Ratios of errors in common (EIC) to errors (E) and of EEIC to EIC are also both high.4,5 There were only 7 differences for an exam of 92 questions. The critical EEIC/D ratio (1a) was 1.86 (the odds against these exams being written independently is 5 × 1010/1, based on σ = 6.6). For comparison purposes, these ratios should be compared with the “av pair”. The differences in the exams of S and Z in the first exam went from 7 where S copied from Z, to 36 when they rewrote the exam (entry 3), and EEIC/D dropped from the high value of 1.86 to a normal, low value of 0.03. S and Z went from 13 exact errors in common to only 1. Clearly, S did not know the material on the first exam. Entry 4 compares the answers for student S on the exam where the answers were copied to a bona fide effort 4 months later. Note that there were significantly more errors (17 vs 28), the EIC dropped from 16 to 7, and the EEIC dropped from 13 to 2. For student Z (who provided answers to S in the first exam), entry 5 reveals a great deal. This is the case of the “ultimate identical twin”, the same student motivated on both exams. The number of errors barely changed (19 vs 18); however, the EIC and EEIC values were 12 and 5, respectively, and the number of differences between the two exams was 20. These values for the retake are typical for two different students who made nearly the same grade. In other words, the same student achieved essentially the same grade by a different path, completely typical of the way different students respond to questions. This is consistent with the identical twin result (1a) where, studying together, the twins achieved essentially the same grade but by different paths.
Journal of Chemical Education • Vol. 75 No. 4 April 1998 • JChemEd.chem.wisc.edu
In the Classroom
Conclusions
Table 1. Data on Errors
This result not only indicates that our method of evaluating similarities between exams has considerable validity, but strongly suggests that even a “good” student (ca. 80%) does not produce an excessively similar exam when studying the same material and given the same exam months later. The easier questions were still answered correctly, but answers to the more challenging ones varied considerably.
Pair
EEIC
EEIC/EIC
D
EEIC/D
σ
14
7
0.50
35
0.20
0.4
16
13
0.81
7
1.86
6.6
9
1
0.11
36
0.03
0.9
#E
EIC
1 (av pair)
28,28
2 ( S1,Z1)
17,19
3 ( S2,Z2)
28,18
4 ( S1,S2)
17,28
7
2
0.28
36
0.06
0.5
5 ( Z1,Z2)
19,18
12
5
0.42
20
0.25
1.7
Note: E = errors; EIC = errors in common; EEIC = exact errors in common; D = differences.
Notes 1. A recent analysis of final exams by this program has shown that flagrant copying has been reduced (1) by a factor of more than 10. This can be directly attributed to the use of scrambled exams; scrambled seating further prevents exam copying. 2. Our program (Signum) is available if a formatted 5.25- or 3.5in. disk is sent to us. The program has been rewritten in “C” language and runs much faster than the original version. It is designed to function on IBM PCs. A full analysis for a class of 350 (> 60,000 pairs) on a 100-question exam takes 7 min with a 80486 chip on a computer running at 25 MHz. 3. Normally, we use four versions of the same exam material distributed down one row as versions ABABAB… and then versions CDCDCD… down the adjacent row. 4. For a full analysis of the data, see refs 1a and 1b (Table and Figs. 1–3). In general, in exams where copying clearly took place, the ratios EIC/EEIC are > 0.75; and more importantly, the EEIC/D ratios
are > 1.0. The probability (in standard deviations, see ref 1a) for the pair writing the answers they did is high (> 5σ). This value measures the degree of independence for that pair writing their answers compared to the mean of the probabilities for all pairs; Gaussian statistics apply. 5. For a pair of students with grades in the 80% range and an exam of > 60 questions, 13 exact errors in common is high.
Acknowledgment We thank McGill University for continuing support for this work. Literature Cited 1. (a) Harpp, D. N.; Hogan, J. J. J. Chem. Educ. 1993, 70, 306. (b) Harpp, D. N.; Hogan, J. J.; Jennings, J. S. J. Chem. Educ. 1996, 73, 349. 2. See refs 5–8 in Harpp, D. N.; Hogan, J. J. J. Chem. Educ. 1993, 70, 306.
JChemEd.chem.wisc.edu • Vol. 75 No. 4 April 1998 • Journal of Chemical Education
483
