Laboratory Practical Examinations in Quantitative Analysis Evaluating Accuracy, Precision, and Speed John E. Frey Northern Michigan University, Marquette, MI 49855 Equitable Evaluation of Student Performance Before 1982 students in our quantitative analysis course were required to complete a series of assigned analyses a t their own speed, subject only to specific cutoff dates. Normally students were given one of the several commercial unknowns used for each analysis and were graded primarily on the accuracy, that is, degree of deviation of their results from a predetermined assay or "truevalue". Eachsection met for two 3-h periods per week; enrolment was limited to 16 in order to minimize crowding a t balances and instrument stations. Considering Efficiency Most students were able to complete the work satisfactorilv bv the last week of the semester. Thanks to excellent e x p e r k n t a l and organization;ll skills a few of them finished as much as 4-6 weeks ahead of schedule. Clearly the faster students performed at a superior level. ~ o w w e r , under the prevailing system it was difficult to evaluate and acknowledge equitably the many skills that contributed to their accurate, precise, and efficient analytical work.
as the means of measuring the achievement of our primary goals. Competent and Consistent Grading of Various Challenges After a decade of trials with these formats we developed a program that gives students experience with a wide variety of analytical procedures and techniques and has a comprehensive and consistent grading scheme. Our current course requires completion of nine determinations (Table 1) described in our own laboratory manual (3).The instructions for these analyses, while based on standard procedures, are more detailed than those found in most published manuals. Table 1. Quantitative Analysis Determinations
Developing a Fair Dispersion Scale The most difficult aspect of evaluating student performance on an analysis is establishing a uniform and fair dispersion scale for assigning grades. If results are graded on the basis of accuracy alone, what degree of deviation from the "true" value constitutes a grade of A, B, or F? Should it be the same for a11 types of analyses or difyerent? Should the scoring- dispersion criterion be based on a unlversa1 scale a~nlicableto all students recardless of local conditions or by an internal scale based gn the standard deviation of the students' results within a particular section? Should precision and efficiency be considered at all? Evaluations Based on Goals and Objectives After a detailed review of course objectives we determined that the work could be evaluated by a numerical estimation of three primary goals: accuracy, precision, and speed. We believed that the achievement of the most important objectives, including mastery of technique understanding of principles ' computational skill data and ermr analysis organization remrd-keeping safety and neatness contributed directly toward the fulfillment of these goals. We decided that the laboratory practical examination, or practicum, described by MacNevin ( I ) and the titration race described by Crabtree and Pickering (2)might serve Presenrea a! tne 203ra Amer~canChem ca Soc ery Nat~onalMeeting, San Franc sco. CA. Apri 8. 1992.
9.
Practicum I. Potassium Biphthalate by NaOH Titration Practicum Ii. Sodium Carbonate by HCI Titration Acid-Base Titration Curves: pKaand Equivalent Weight of an Acid or Base Gravimetric Nickel SpectrophotometricManganese in Steel Practicum ill. Hardness of Water by EDTATitration Practicum IV. Spectrophotornetric Iron(il)in Water Practicum V. Arsenic by Iodine Titration SpectrophotometricpKaof an Aci&Base Indicator
Four of them-numbers 3-5 and 9--require two or more laboratory periods; the time frame here is open-ended, although the results must be reported by a specified deadline. These determinations use lengthy chemical procedures, i n s t r u m e n t a l measurements, or extensive calculations. The other five--numbers 1, 2, &&-can be completed in less than 2 h; these are administered as practicums on a specified date under examination conditions. Organization and Grading of the Practicum The first practicum, the analysis of a sample of assayed potassium biphthalate (KHP) is scheduled for the fifth week of the semester. Preliminary Work Preliminary work includes the calibration of a 50-mL buret and 25-mL pipet as well as the preparation of standard solutions of sodium hydroxide (titrated against potassium biphthalate) and hydrochloric acid (titrated against sodium carbonate) that have been cross-titrated with each other. The results are checked by the instructor to insure that each student has a reliable standard solution. Standardizations are repeated until satisfactory results are obtained. During this phase of the work students learn the Volume 71
Number 1 January 1994
51
specific techniques and calculations needed for the analysis.
Table 2. Student Results for Practicum I
The Determination
%
X
On the designated date and time each student is given a sample of the same unknown sufficient for four determinations. If the number of students in the section is greater than the number of available analytical balances, the class is divided into groups and started in waves at half-hour intervals. Each student enters weight and volume data (in ink) into a standard data sheet and calculates a personal mean %KHP(IT), standard deviation (s'), and 90%confidence interval (A') of the mean.'Results for a t least two determinations must be submitted. The calculations are checked by the instructor and the results are accepted only after calculation errors have been corrected. The elapsed time is recorded on the student's data sheet. Grading Scale
The results are graded on a scale of 0-100 points. Each student is awarded 55 points for preparing a n acceptable standard solution. The remaining 45 points are split equally between three subscores-A, P, and S--at 15 points each, based on the accuracy, precision, and speed achieved in the analysis. The subscores are determined as described below.
A
A'
lime
Swres
mln
Total
1.
50.76
2.
50.86
3. 4.
50.86 50.89
5. 6.
50.90 50.93
7.
50.94
8.
50.96
9. 10.
51.02 51.03
11.
51.08
Mean 50.93
90 90 81 94 84 70 95 94 91 84 89 88
where t is taken from a table of "Student's t" values, and n is the number of results reported by each of the students. The mean of the A' values (A")is then calculated. The precision subscores P shown in column 7 of Table 2, are calculated using eq 4.
Accuracy Accuracv A is wuallv defmed as the absolute deviation of a student's mean vafue X from the "true value", that is, the assay provided by the supplier. Our experience over the last decade indicates that the mean valueX" of acceptable student results for the entire class on a particular unknown is usually within 0.5% relative of the assay value. This means that X" for the class is a reliable estimate of the "true value" and that the accuracy of a student result A may be defined as the absolute deviation betweenX' and
X".
In addition, the standard deviation (SF) of X" can serve as the basis of a dispersion scale for assigning Our .grades. . current system assigns equal weight to accuracy, precision, and speed. A student receives an accuracy subscore, A, of 15 for a deviation of zero and a subsmre of 0 for a deviation which is three or more standard deviation units from the class mean. The snbscore for accuracy is calculated wing
where 15 is the value of the weighting parameter, and 3 is the value of the dispersion parameter. In a recent practicum twelve students obtained an overall mean value ofX" = 50.93% with sx. = 0.09. The range of values reported was 50.7&51.08%. The A subswres are shown in column 6 of Table 2. Precision The precision of a student's result is measured using the "90% Confidence Interval" (A') as calculated by eq 3.
'Variables such as X, where the alphabetical character is followed by a single prime, refer to individual student results. Variables such as X', where the alphabetical character is followed by a double prime, referto overall class results. 52
Journal of Chemical Education
,-,
We use A' instead of some other value of dispersion, such as the average deviation or standard dcviatio-n,hecause .I' decreases and the rehability ofXincreases as the number of reoorted results increases. This rewards students for reporting more results and discourages them from discarding without good cause results that appear to be deviant. On the other hand, taking the extra time to do additional titrations may incur point deductions for speed. Students who work with marginal speed must balance the points gained for increased reliability against the points that might be lost by taking more time in titrations and calculat~ons. Speed Speed is an important measure of a student's organizational ability and grasp of concepts, as well as experimental and computational skill. However, in a n open-ended laboratory, schedules are often arranged so that every student is able to complete the required analyses on time. Consequently, students usually have little incentive to optimize the time spent in the laboratory, and students who do work efficiently receive no formal acknowledgement for their efforts. The pradicum, by virtue of its strict time frame places a premium on preparation and skill, thus fostering the idea that time is valuable, and timeliness and efficiency are crucial. Speed is evaluated in two different ways in our course. In the first four practicums students are assessed deductions up to 15 points from their speed subscore S for exceeding stipulated clock cutoffs. In the KHP analysis, for example, students who finish in 60 min or less receive 15 points for their speed subscore; the others receive a one-point deduction for every 3 min required beyond 60 min.
Students with t > 105 min receive a speed subscore of 0, and those who exceed 120 min receive a total subscore of 0 or a final score of 55. :
In Pradicum V, "The Great Titration Race", speed subscores are determined by numerical placement. The first student to submit a n acceptable result is awarded 15 points, whereas the remaining students receive a point total calculated using
where, r i s the numerical rank or placement ofthe student m the race, and N i s the number of student participants.
that even passersby sense it and comment on it. It is heartening to observe students preparing and memorizing procedures, practicing titration technique, preweighing reagents, and cleaning glassware in anticipation of the practicum. The fastest time recorded for the completion of a practicum-including weighing - and titrating three unknown samvles. com~ietineall calculations and r e ~ o r t i n eresults-is 28 min. Incidentally, this student had high scores on both accuracy and precision as well.
Discussion
lmproved Time Management over the Semester
Both the weighting and dispersion parameters can be adjusted to suit the experiment. Equations 2,4, and 5 are special forms of a general equation of the type
Students finish all the assiened course work on about the same date. Gone are the &agglers who dawdle until the last frenzied week of the semester and then plead for deadline extensions. Pacing is "in"! Procrastination is "out"! The students themselves are the first to observe that as their efficiency improves, their accuracy and precision improve as well.
.
,
A
where Q is any subscore; W is a weightmg parameter; d is a measure of dispersion; D is a dispersion parameter; and s is the standard deviation. Both W and D can be adjusted to suit the type of analysis and the circumstances in which it is conducted. In a sitnation where accuracy is deemed more important than other scoring criteria, it can be assigned a correspondingly larger weighting parameter. These scoring principles are also used in conjunction with analyses requiring open-ended time frames. For example,the gravimetric determination of nickel by precipitation with dimethvlelvoxime. which takes about nine laboratory hours, is n i t k i t a b l e for the practicum format. In this instance each student is awarded 60 points for submitting a satisfactory result and 40 points for the combined accuracv and precision subscores. The latter are mmvuted using weighting parameters of 20 in eqs 2 and 4.
-
,
--
Use of an Internal Scale to Evaluate Work
The practicum requires the use of only a single unknown for each section of the course. Because all students in the section analyze portions of the same sample, their overall mean percentage is a valid measure of the "true value", and the standard deviations provide unique dispersion scales for each determination, sample. and section. The d~spers~on paramrteri generated from the students' results are amrovnetr for the students'sk~lllc\,cls and the inherent p%sion of the analytical procedure and apparatus used. Critical Appraisal of Students' Results
A
Advantages of the Practicum System A practicum system in which selected analyses are scheduled on a fixed date under examination mnditions has several advantages over completely open-ended lahoratory scheduling. lmproved Attitude and Approach
Students faced with the prosped of losing grade points because of missed deadlines and cutoff times quickly learn to budget their laboratory time with thorough planning and careful execution. The mere act of referring to a n analysis as a "laboratory examination"catches the students'attention and rivets their focus on the task at hand in a way that is quite different from the attitude seen when requiring them to complete the analysis by a specified date. During the preceding week and on the day of a practicum the atmosphere in the laboratory is so intensely serious
Each class generates a collection of uniform data for each determination that can be analyzed to pinpoint problems in the laboratory procedures and the students' technique. It is interesting to point out to students that good accuracy does not necessarily correlate with good precision and speed. We note in Table 2 that student 6 has the highest score on accuracy, but the lowest scores on precision and meed. whereas student 1has the lowest score on accuracv. ". b k the highest scores on precision and speed. These results indicate that a scorine that relies - svstem " on several criteria in evaluating the results of analyses minimizes the chance of a student obtainine a high made on the basis of pure luck. It also rewards th; stuient who excels in all aspects of analysis. . but mav have made a single error. Literature Cited 1. MacNeuin,W. M. J C k m . Educ. 1961,38.144-145. 2. Crabtree, R.H.;PickeRng, M J Ckem.Edue. 1978.55.354. 3. Fmy, J. E. Quontitotiue Analysis; Northern Mlchyan University, 1992.
Volume 71 Number 1 January 1994
53
