Environ. Sci. Technol. 1997, 31, 3729-3731
Response to Comment on “An Alternative Minimum Level Definition for Analytical Quantification” SIR: We appreciate the comments made by Kimbrough regarding our paper published in the July 1997 issue of this Journal (1). Kimbrough expands on the very important issue of carefully defining data quality objectives (DQOs) prior to defining statistical characteristics of performance-based detection and quantification limit estimators. As he correctly points out, the level of relative precision required for a quantification limit estimator is very much dependent on the particular application for which it is intended. A high degree of relative precision is often required when the ultimate goal is to make comparisons at the low end of the calibration range, but less stringent requirements may be more than adequate if the ultimate use of the limit estimator involves concentrations at the high end of a well-behaved calibration function. We thank Kimbrough for expanding on this important point. Kimbrough goes on to point out that an alternative approach to quantification limits based on precision is to develop quantification limits based on accuracy. He points out that when there is bias (either positive or negative), then it is more realistic to compute relative precision based on average measured concentration rather than true (i.e., spiked) concentration. He shows using our own example (which exhibits considerable negative bias) that quite different estimates of the relative standard deviation (RSD) are obtained using the two different approaches. For example, at 50 ng/L, we obtain a RSD of approximately 10% based on true concentration; whereas, using average measured concentration, the RSD is 286%. The reason for the discrepancy is that for a true concentration of 50 ng/L, the average measured concentration was only 2 ng/L. Indeed, as the measured concentration approaches zero, the RSD based on the average measured concentration approaches infinity. Kimbrough then points out that if an important health effect were to occur at concentrations of 50 ng/L but not 5 ng/L then this level of accuracy and precision would be unacceptable. As one might expect, we take a somewhat different view of this paradox. The important question relates to the
distribution of true concentration in a particular environmental sample, and we draw inference to the true concentration by characterizing the distribution of measured responses across the calibration function. This highlights a fundamental difference between calibration-based methods such as the Alternative Minimum Level (AML) and single concentration based methods such as the U.S. EPA’s Method Detection Limit (MDL) and Minimum Level (ML). Constructing detection and quantification limits based on a multiple of the observed sample standard deviation at a single concentration does not incorporate bias or accuracy into the detection or quantification limit estimator. By contrast, the calibration-based detection and quantification limit estimators provide a direct way of linking measured concentrations to true concentrations (i.e., via the regression of measured on true concentrations and incorporating bias as a nonzero intercept of the regression function). Kimbrough appears to overlook this critical distinction when he states that “The statistical approach advocated by Gibbons et al., (1) is by far the most commonly used. It is the basis for the U.S. EPA’s method detection limit, reliable detection limit and minimum level as well as numerous other procedures”. Indeed these methods are quite different in both theory and practice [see Gibbons (2) for a review and detailed commentary from the proponents of these different approaches]. While they share a common theme in that they all involve estimators that are based on precision, the calibration-based methods incorporate both precision and accuracy (e.g., AML), whereas the single concentration based approaches are based solely on precision (e.g., MDL and ML). In part, the apparent discrepancy between the two approaches (i.e., relative precision based on true versus average measured concentration) is based on the fact that for this analyte, the U.S. EPA experienced a background problem, and the measured concentrations were background corrected to produce the data in Table 1 of our paper (U.S. EPA, personal communication). A more prudent analysis would be to fit the calibration function to the original instrument responses or uncorrected measured concentrations; however, only measured concentrations were available at the time of our analysis, and we had no knowledge of the background correction prior to publication of our paper.
FIGURE 1. Measured versus true concentration with 99% prediction interval for CD 111 in ppt.
S0013-936X(97)02016-6 CCC: $14.00
1997 American Chemical Society
VOL. 31, NO. 12, 1997 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
3729
FIGURE 2. Standard deviation vs concentration for CD 111 in ppt.
TABLE 1. Method 1638 ICPMS Data from U.S. EPA for Cadmium at Mass 111 (in ng/L) spike replicate
0
10
20
50
100
1 2 3 4 5 6 7
0.88 1.57 0.70 0.80 0.54 1.83 1.34
10.17 11.13 11.66 10.80 11.11 11.95 11.14
19.97 20.28 23.20 22.12 18.01 24.83 21.10
54.78 49.00 51.92 49.00 54.75 50.25 50.03
97.06 94.60 102.54 101.09 99.20 93.71 100.43
mean % RSD, spike % RSD, mean % bias
1.09
11.14 5.80 5.21 11.40
21.36 11.25 10.53 6.80
51.39 5.02 4.88 2.78
98.38 3.35 3.41 1.62
44.95
Nevertheless, use of a calibration-based method circumvents this problem. For example, inspection of Table 1 of our paper and Figure 1 of Kimbrough’s comment reveals that if true concentration were unknown, setting the quantification limit at 50 ng/L (i.e., only measured concentrations of 50 ng/L would be deemed reliable for quantitative purposes) would lead to a true concentration of 100 ng/L for which % RSDs for average measured and true concentrations are virtually identical (see Figure 1 of ref 3). In fact, in these backgroundcorrected data, a true concentration of 100 ng/L corresponds to 50 ng/L of cadmium in the sample plus background of approximately 50 ng/L. As a further illustration, Figure 1 displays the calibration function for cadmium at mass 111 for which there was no background problem for concentrations in the range of 0-100 ng/L. The raw data are displayed in Table 1 along with means, standard deviations, % bias, and % RSDs computed relative to both spiked and average measured concentrations. Inspection of Table 1 reveals that there is very little difference between % RSDs based on true versus measured concentrations for cadmium at mass 111, which is the mass specified in Method 1638 for quantification (mass 114, which we used as an illustration in our paper, is used for identification only). Figure 2 displays the relationship between variability and concentration and the fit of the Rocke and Lorenzato (4) model to these data. The Rocke and Lorenzato model provides a suitable fit to the observed standard deviations, and even at these remarkably low levels variance increases with concentration. Figure 3 displays computation of the AML for these data. The estimated AML is 8 ng/L. Note that using all of
3730
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 31, NO. 12, 1997
FIGURE 3. AML worksheet for CD 111. the data (i.e., from 0 to 25 000 ng/L), the estimated AML was 10 ng/L, again demonstrating robustness of the AML to range of spiking concentrations. In terms of accuracy at the AML, the 99% prediction interval for measured concentration at the AML (i.e., the interval surrounding the calibration line in Figure 1) is 7-11 ng/L, which incorporates the small positive bias observed in the raw data presented in Table 1 (see Table 2). Table 2 displays 99% confidence prediction bounds for measured concentrations at true concentrations ranging from 0 to 100 ng/L. These bounds, which are a direct byproduct of the AML procedure, describe the distribution of measured concentrations for any given true concenration. Returning to Kimbrough’s example of a standard of 50 ng/L,Table 2 reveals that when the true concentration is 50 ng/L, the measured concentrations can range from 46 to 55 ng/L with 99% confidence. This is a direct measure of the range of expected accuracy that can be
TABLE 2. 99% Prediction Intervals for Measured Concentration for CD_111 (in ppt) measured concentration true concn
lower bound
upper bound
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98 100
-0.36627 1.59944 3.54471 5.47049 7.37835 9.27021 11.14808 13.01393 14.86958 16.71661 18.55640 20.39007 22.21859 24.04276 25.86323 27.68054 29.49515 31.30742 33.11767 34.92617 36.73312 38.53873 40.34316 42.14652 43.94894 45.75054 47.55138 49.35155 51.15110 52.95010 54.74861 56.54667 58.34430 60.14157 61.93848 63.73507 65.53138 67.32741 69.12319 70.91873 72.71406 74.50919 76.30414 78.09890 79.89351 81.68796 83.48229 85.27646 87.07051 88.86445 90.65827
2.74534 4.72124 6.71761 8.73344 10.76720 12.81698 14.88073 16.95650 19.04248 21.13706 23.23891 25.34686 27.45996 29.57741 31.69857 33.82289 35.94990 38.07925 40.21063 42.34376 44.47841 46.61443 48.75164 50.88989 53.02909 55.16913 57.30991 59.45137 61.59344 63.73605 65.87917 68.02273 70.16672 72.31108 74.45578 76.60082 78.74615 80.89174 83.03759 85.18366 87.32995 89.47646 91.62313 93.76998 95.91700 98.06417 100.21149 102.35892 104.50649 106.65419 108.80198
obtained from the calibration function. As such, measured concentrations in excess of 55 ng/L can be taken as evidence that the true concentration has exceeded the standard of 50 ng/L. Of course, as with any environmental monitoring data,
it is of critical importance to verify the result with an independent verification sample. Similarly, the data from which the prediction interval is constructed must also be representative of the real world problem to which they are applied. The data used here as an illustration were obtained in a single laboratory under highly controlled clean lab conditions in distilled water and may have little bearing on the levels of precision and accuracy that are attainable in routine practice with multiple instruments, analysts, and laboratories under blinded conditions (i.e., without knowledge of true concentrations) in real world matrices. In summary, a major difference between calibration-based estimators such as the AML and single concentration-based estimators such as the MDL and ML is that the calibration approach allows us to incorporate both precision and accuracy into our quantification limit whereas the single concentration approach is limited to precision. We take exception to Kimbough’s lumping these quite dissimilar approaches together. We strongly agree with Kimbrough’s point that the statistical properties of any detection or quantification limit estimator (e.g., confidence and coverage levels) should be tied to well-specified data quality objectives. In terms of his comments regarding our example, we selected cadmium at mass 114 to illustrate robustness of the AML procedure to numerous real world problems, including bias. We hope that this response clarifies his concerns with our original example and that the current illustration demonstrates the linkage between our seemingly different approaches to this interesting problem.
Literature Cited (1) Gibbons, R. D.; Coleman, D. E.; Maddalone, R. F. Environ. Sci. Technol. 1997, 31, 2071-2077. (2) Gibbons, R. D. J. Environ. Ecol. Stat. 1995, 2, 125-167. (3) Kimbrough, D. E. Environ. Sci. Technol. 1997, 31, 3727-3728. (4) Rocke, D. M.; Lorenzato, S. Technometrics 1995, 37, 176-184.
Robert D. Gibbons* University of Illinois at Chicago 912 South Wood Street Chicago, Illinois 60614
David E. Coleman Alcoa Technical Center Applied Math and Computer Technology AMCT (483)-D-10 100 Technical Drive Alcoa Center, Pennsylvania 15069
Raymond F. Maddalone TRW Building 01, Room 2040 1 Space Park Redondo Beach, California 90278 ES9720164
VOL. 31, NO. 12, 1997 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
3731
