Uncertainty Analysis by the “Worst Case” Method Roy Gordon Harvard University, Cambridge, MA 02113
Miles Pickering1 and Denise Bisson Princeton University, Princeton, NJ 08544
Uncertainty analysis is a traditional bugbear in freshman analytical chemistry lab. While many students can do a formal propagation of uncertainty, far fewer understand what they have calculated. Uncertainty analysis is a mechanical ritual done to appease the instructor. Our best students do not understand what it means; our worst students cannot do it at
By multiplying and dividing by Ax,
J. Chem. Educ. 1984.61:780. Downloaded from pubs.acs.org by UNIV OF SOUTH DAKOTA on 09/20/18. For personal use only.
or
all. Nevertheless most quantitative experiments require the reconciliation of observed values with expected values, with whole number ratios or with each other. Such a reconciliation cannot be done rigorously without some knowledge of the random uncertainty, so the topic is unavoidable. The last decade has seen the replacement of the slide rule with the electronic calculator, and this technological advance opens new options. This paper presents a new method of uncertainty propagation which is in many ways pedagogically superior to the traditional manipulation of absolute and relative uncertainty. This method is the manipulation of the limits themselves. The “Worst Case” Method
The new method concentrates on the calculation of upper and lower limits, the “worst cases,” bypassing absolute and relative uncertainties. For example, in traditional uncertainty analysis, given a quantity A with absolute uncertainty ±a, and a quantity B known to ±b, the relative uncertainty of the product is taken as the sum of the relative uncertainties a/A and b/B. (Strictly speaking, it should be the root-mean-square sum, but this distinction is largely neglected in freshman chemistry.) The upper limit is calculated as a separate step and is given by AB + AB
In the new worst case method, one merely computes (A + a) {B + b) to get an upper limit, and (A a) (B b) to get a lower limit. This replaces several steps with one step and eliminates the need to consider ratio quantities, a stumbling block for students. The same principle can be applied to all computations. The student simply computes the upper and lower limits (the “worst cases”) for his or her measurements, and then carries the worst case through the calculations to get the limits for the results. The rules for manipulation are: 1) If the raw data are converted to a final result by addition or multiplication, add or multiply like limits to get the same limit. Adding upper limits gives the upper limit, for example. 2) If the raw data are converted to a final result by subtraction or -
-
division, subtract or divide opposite limits. If the numerator of a fraction is an upper limit, the upper limit of the result is calculated. A formal proof of the idea for a function of one variable is illuminating. If the function being measured is f(x), then one of the limits will be given by f(x + Ax) and the absolute un-
certainty will be
Af(x)
1
780
=
|f(x)
—
f(x + Ax)|
Author to whom correspondence should be addressed.
Journal of Chemical Education
Af(x)
=
1
f(x)
—
one
obtains
f(x + Ax)[ Ax Ax
which, in the limit of small Ax, is equivalent to the classical uncertainty expression Af(x)
=
Thus the handling of upper and lower limits is just as mathematically rigorous as the traditional analysis. (What is happening is that we are taking derivatives without letting the students know!) The Advantages of the “Worst Case” Method 1) The worst case method involves the manipulation of concrete quantities. Relative and absolute uncertainties are abstractions. The upper and lower limits are easy to understand. 2) There is no need to convert from relative to absolute error or back again nor to convert relative error to limits at the end of the calculation. The limits on the final result appear
directly. 3) The worst
case method handles logs, trigonometric functions, complicated polynomials, etc., with little pain. One simply plugs in the appropriate limit values, and repeats the original calculation. 4) The procedure is less mysterious. The student who forgets the rules can either reason them out or work out all the possibilities and see which are the worst cases. The method can be completely understood without a knowledge of the calculus. 5) Most essential, the “worst case” method focuses the student’s attention on the limits, which are the real goals of the calculation, and does not allow him or her to bog down in the calculation itself. The student also knows exactly what has been calculated and why it is important.
The Advantages of the Traditional Method 1) The worst case method seems more laborious. Our findings argue this is not a problem. 2) The idea of relative error is useful because it pinpoints the least accurate measurement of a set. We recommend therefore that the summing of relative error in multiplicational problems be taught as a short cut to calculating the limits, and only taught once students really understand the worst case method. This is in accord with the general idea of teaching in order of increasing abstraction. 3) Since the new method is so obvious, one might ask why it was not previously used. However, the calculations required would have been impossibly laborious in the days before the electronic calculator. In many cases the value would be identical with its upper and lower limits to slide rule accuracy. Also, the uncertainty introduced by the slide rule would be comparable to the random uncertainty of the experiment for some work. Thus, laborious, longhand multiplication, or work with seven-figure log tables would have been needed. Now, however, the ubiquitous electronic calculator has circumvented this problem, and we can consider adjusting our teaching accordingly.
Table 1.
Results of
Question: Which method is... ?
Easier to understand Less laborious Easier to learn Quicker
Table 2.
a
again, nothing will have to be unlearned. The only refinements
Questionnaire New
About the same
59 55
9
7 4
61
6
10
52
9
Traditional 1
3
case.” One objective in the lab is to teach logical thinking about results derived from empirical data. To do this requires some estimate of the uncertainty, but this can be provided by use of very much simpler calculations than generally realized.
9
Reported Time to Complete Lab Report on Mineral Standard
1979-80 (traditional uncertainty analysis) 1981-82 (worst-case uncertainty analysis)
will be the introduction of other calculational methods, and more mathematical definitions of what is meant by “worst
N
Mean
Deviation
80 132
7.08 5.88
3.98 2.58
Testing the New Method in the Classroom We have tested this idea in the freshman laboratories at Harvard and Princeton. The educational study described was done at Princeton in the upper level of the two large freshman lab courses. The results are probably applicable for most courses where the teaching of uncertainty analysis is a goal. A large number of students in this course take freshman physics and freshman chemistry at the same time. The physics courses still use the traditional form of uncertainty analysis. Thus the students can compare the two methods themselves. A questionnaire was distributed to students taking both physics and chemistry, and the results are summarized in Table 1. The worst case method is strongly preferred by students. If the sign test is applied to the data, the results are statistically significant at the p < 0.01 level in all categories. We also were concerned that the worst case method might prove unwieldy and laborious. To answer this question, we polled students about the time required to do various lab reports. In Table 2 compares the reported workload for writing the lab report for “the formula of a mineral” report using the two methods. This is a several-week experiment in which a total analysis of a sulfate mineral is performed (water of
crystallization by gravimetric methods, sulfate by BaSC>4 precipitation, metal by ion exchange). This lab report was written up in the 1979 academic year by the traditional method, and in the fall of 1981 by the new method. (The experiment was run in 1980, but the students were not polled about their workload.) Using the new method there is a statistically significant reduction (t 2.66, p < 0.01) in reported work time. It is, of course, precarious to compare two different groups of students, since the 1979 group may not be an exact control for the present group. However, since the results of this survey are corroborated by the students’ own reports about the two methods, we feel safe in concluding that, at the very least, the worst case method is no more laborious than the traditional method. The objection may be raised by a purist that this method lowers the level of rigor. In fact, this is not true. Student questions are not concerned with how to calculate limits, but how to interpret them. The uncertainty analysis has become a tool to help make a decision about data, not just an arcane ritual. Thus, the students now focus on essential questions of scientific measurement, i.e., whether the results agree or disagree with each other, or with an expected value. While the uncertainty calculation is at a lower level of abstraction, the overall analysis of data is at a much more scientifically mature =
level. We feel very comfortable with this method of uncertainty analysis. When advanced courses look at random uncertainty
Appendix For readers who wish to use this technique in their classroom, numerical example done by both methods is given here.
a
The Problem
A solution is made up in a 100-ml, class A volumetric flask. The substance has a MW of 120.0 g/mole, and 2.6754 grams are weighed by difference. A filled weighing bottle containing the substance weighs 19.7005 g, and after the transfer, the bottle itself weighs 17.0251 g. The absolute uncertainty of the balance is ±0.0005 g, and the absolute uncertainty of the flask volume is ±0.08 ml. The final concentration value is 0.2229 M. What is the uncertainty range of this figure?
By the Classical Method Since the weighing is by difference, absolute uncertainties add. Weight of filled vial Weight of vial after transfer Weight of solid
19.7005 g ± 0.0005 17.0251 g ± 0.0005 2.6754 g ± 0.0010
The molarity is proportional to mass over volume, so to get the uncertainty in molarity we must add the relative uncertainty of the mass and the relative uncertainty of the volume. The relative uncertainty in the mass is 0.0010 g
=
2.6754 g
0.04%
The relative uncertainty in the volume is 0.08 =
100.0
0.08%
The relative uncertainty in the concentration
0.04% + 0.08%
=
=
0.12%.
To get the range, the relative uncertainty is then multiplied by the concentration to get the absolute uncertainty (0.0012)(0.2229M)
=
±0.0003 mol/1
Thus the real value lies between 0.2226 M and 0.2232 M. By the Worst Case Method This is best done by laying the data out in tabular form: lower
Weight of full vial Weight of solid transfered
upper
limit
value
limit
19.7000 17.0201 2.6744
19.7005 17.0251 2.6754
19.7010 17.0301 2.6764
(The opposite limits are combined as in rule 2. That is, the upper limit of the vial weight after transfer is subtracted from the lower limit of the full vial weight to get the lower limit for the weight of material transferred.) For the flask volume, the upper limit is 100.08 ml and the lower limit 99.92 ml. Combining the upper limit of weight with the lower limit of volume will give the upper limit of concentration: upper limit
2.6764 g
=
=
(0.09992 1)( 120.0 g/mole)
0.2232 M
Combining opposite limits gives the other value: lower limit
=
_2.6744 (0.10008
1)
g_
=
0.2227 M
(120.0 g/mole)
Because of rounding errors there are slight differences between this result and that from the classical calculation.
