A New Approach to DRINKING-WATERQUALITY DATA: LowestConcentration Minimum Reporting Level jupiterimages
A new procedure focuses on precision and accuracy in analytical measurements.
JOHN J. M ARTIN THE CADMUS GROUP, INC. STEPHEN D. W INSLOW SH AW EN V IRONMENTAL, INC. DAV ID J. MUNCH U.S. EPA OFFICE OF GROUND WATER A ND DRINK ING WATER
O
ne of the key factors in ensuring the integrity and reliability of drinkingwater-quality data is the level of confidence associated with the analytical results. Precision (reproducibility of data) and accuracy (closeness of measured value to true value) are critical data-quality objectives (DQOs) for drinking-water regulations. Targeting these objectives, the U.S. EPA has developed a process for determining the single-laboratory lowestconcentration minimum reporting level (LCMRL). The LCMRL is the lowest true concentration for which future analyte recovery is predicted (with at least 99% confidence) to fall between 50 and 150%
© 2007 American Chemical Society
(inclusive). This recovery interval has been used in recent gathering of occurrence data, including the Information Collection Rule and the Unregulated Contaminant Monitoring Regulation. Although the LCMRL is a laboratory- and analyte-specific value, LCMRLs from multiple laboratories can be used to determine a minimum reporting level (MRL) that can be considered for national application for any environmental regulatory program. The LCMRL, and by association the MRL, allows for simultaneous incorporation of precision and accuracy in analytical measurements. As a result, a specified confidence is inherent in any analytical result at or above the MRL. To make policy decisions regarding the protection of human health, including decisions about whether to regulate a particular analyte, EPA needs data of known precision and accuracy. Recently, a Federal Advisory Committee on Detection and Quantitation Approaches was established to review and consider formal adoption of detection and quantitation procedures. The LCMRL procedure is one of many being considered. Debate and research about procedures for determining detection limits (i.e., presence) and quantitation levels (i.e., reliable quantitation) have been ongoing for decades (1–14). Many of those procedures consider only the precision of the measurements (1–11), whereas others take both precision and accuracy into account (12–14). However, of those that consider both precision and accuracy, all but the LCMRL procedure apply precision and accuracy independently of each other. The LCMRL procedure applies them simultaneously. Thus, the advantage of the LCMRL procedure over others is that the determination of the LCMRL is simultaneously influenced by the variance of the measurements and the prescribed DQOs of ±50% of the true value. The other procedures are discussed in detail by Winslow et al. (15). February 1, 2007 / Environmental Science & Technology n 677
One of the key issues associated with replicate analyses over a range of concentrations—nonconstant variance over the range—has been investigated and addressed by many researchers over the past several years (8, 9). LCMRL procedures also address the problem of nonconstant variance (see Supporting Information). These LCMRL procedures have been documented by EPA in further detail (15–17).
study. Thus, to obtain three complete data sets for each of the seven analytical methods, and to provide as representative a cross section of laboratories as possible, five laboratories were selected: one commercial laboratory, one state laboratory, one Public Water System laboratory, and two EPA regional laboratories. As part of the study planning process, it was determined that a lowest spiked (or true) concentration of ~2× each laboratory’s method detection limit (MDL) was the minimum level to which laboratories could reasonably be expected to calibrate and report data. Therefore, as a matter of practicality, the range of true concentrations used to determine LCMRLs was tailored to fit into the range between the MDL and the practical quantitation limit (PQL) of each primary analyte. Several exceptions to the “twice-the-MDL” guideline can be seen in Table 1 if the lowest true concentration is compared with the MDL. Some of these were associated with calibration near the MDL to create serial dilutions that allowed for maximum overlap of spiking concentrations, and some exceptions were made to decide whether concentrations near the MDL could be included in an LCMRL determination. The amount of experience of individual laboratories with these analytes was the basis for these decisions. The primary analytes were selected on the basis of the following criteria: the selected analytes
Interlaboratory study: proof of concept An interlaboratory study was conducted to evaluate the soundness of the LCMRL concept and to determine whether LCMRLs could be used to establish realistic, yet reliable, MRLs. The goal was to obtain three complete sets of data from each of seven EPA methods currently in use for the analysis of regulated analytes in drinking water. A primary analyte was selected from each method; other analytes included in the study are beyond the scope of this article. Each data set consisted of seven replicate analyses at each of four concentrations. Thus, an LCMRL could be calculated for each analyte and laboratory, and the LCMRLs could be used to determine MRLs, as described later. The analytical methods, primary analytes, and spiking concentrations are summarized in Table 1. One of the three laboratories initially selected for inclusion in the study was not set up to perform all of the analytical methods that were the subject of the TA B L E 1
Analytical methods, primary analytes, and true concentrations EPA method Primary number analyte
200.7
Chromium
200.9
Cadmium
300
507
508
515.3
525.2
Lab MDL a number (μg/L)
True concentration (μg/L) Low Intermediate
High
Spiking ratio (high/low)
1 2 3
4 6 6
6 8 8
8 10 10
10 12 12
2.5 2 2
0.1 0.1 0.1 25 10 25 0.4 0.05 0.05 0.004 0.0075 0.008 0.2 0.8 0.4 0.05 0.25 0.25
0.2 0.2 0.2 50 50 50 0.5 0.1 0.1 0.008 0.075 0.08 0.4 1.2 0.8 0.25 0.5 0.5
0.5 0.5 0.5 100 100 100 0.75 0.4 0.4 0.08 0.15 0.16 0.8 1.6 1.2 0.5 1.0 1.0
1 1 1 250 250 250 1 0.5 0.5 0.16 0.3 0.32 1.2 2 1.6 1.0 1.5 1.5
10 10 10 10 25 10 2.5 10 10 40 40 40 6 2.5 4 20 6 6
1 2 4 Nitrate as N 1 2 3 Atrazine 1 2 5 Heptachlor 1 2 5 2,4-Dichloro1 phenoxyacetic 2 acid 3 Hexachloroben- 1 zene 2 3
2 3 4 (267.7 nm); 6 (206.1 nm) 0.042 0.03 0.045 26.4 5 6.5 0.202 0.045 N/A b 0.002 0.004 N/A b 0.165 0.039 0.09 0.017 0.09 0.02 c
a Method b Not
detection limit. applicable.
c Estimated.
678 n Environmental Science & Technology / February 1, 2007
LCMRL determination process The following steps were used to determine the LCMRL. All of these steps can be performed automatically with EPA’s LCMRL calculator (17) or manually by procedures discussed by Winslow et al. (15) and in EPA’s LCMRL statistical protocol (16). An example graph is presented in Figure 1. First, for each analyte, the measured concentration (y axis) was plotted against the spiked or true concentration (x axis). The replicate data were summarized in Microsoft Excel and imported into STATA statistical software. Second, the LCMRL data were regressed with ordinary least squares (OLS) and variance-weighted least squares (VWLS; see Supporting Information), which were performed simultaneously by STATA. The regression line was not forced through the origin. Regression was performed with a straight-line regression equation, and a 99% prediction interval was constructed around the regression line. Third, the STATA output was downloaded to Microsoft Excel, including plots for each regressed data set, with lines corresponding to 50% and 150% recovery of the spiked concentration. The LCMRL is determined by dropping perpendicular lines to the x axis, starting at the intersections of the upper and lower prediction-interval bounds with the 50% and 150% recovery lines. The larger of the two x values is the LCMRL (0.28 µg/L from Figure 1), because this concentration is the lowest level for which the upper and lower recovery requirements are met. See the Supporting Information for details about the calculation of the LCMRL from the intersections of the prediction-interval limits with the 50% and 150% recovery lines. The DQOs of precision and accuracy are depicted in Figure 1. Precision is reflected by the variance, or spread, of the data at each true concentration. Accuracy is reflected by the degree to which the data points are clustered about their true value. Note that the results at the highest true concentration (0.5 µg/L) are biased slightly high and are centered closer to 0.6 µg/L than 0.5 µg/L. Both precision and accuracy dictate the breadth of the prediction interval and, hence, the magnitude of the LCMRL. High precision (i.e., low variance) and high accuracy (i.e., low bias) will tend to tighten the prediction interval and will move the intersection points with the recovery lines to the left, thus reducing the LCMRL. Low precision (i.e., high variance) and low accuracy (i.e., high bias) will move
FIGURE 1
Example LCMRL determination LCMRL determination from multiconcentration replicate analyses. LCMRL = 0.28 µg/L. 0.8
150% recovery line
0.7 Measured concentration (µg/L)
must have established maximum contaminant levels (MCLs) and PQLs, and the selected analytes must each consist of a single component (unlike toxaphene, for example). Analytes were processed through all steps of each analytical method, including extraction (where applicable) and the use of method-required preservatives. Seven replicate samples, at each of the four predetermined spiking concentrations, were prepared and analyzed by participating laboratories with each applicable method. LCMRLs and MRLs were determined from the data by the procedures described in the following sections.
Upper limit of 99% prediction interval
0.6
Regression line
0.5
Lower limit of 99% prediction interval
0.4 0.3
50% recovery line
0.2 0.1 0.0 0.0
0.1
0.2 0.3 0.4 True concentration (µg/L)
0.5
0.6
the intersections to the right, raising the LCMRL. LCMRLs were determined for each laboratory and analyte in the study. The mean of the LCMRL values from various laboratories was calculated for each analyte, and MRLs were determined as follows. When LCMRLs from three laboratories were available, the MRL was calculated from Equation 1: MRL = Mean + 3s
(1)
where Mean is the average of the three LCMRL values, and s is the standard deviation of the three LCMRL values. When LCMRLs from only two laboratories were available, the MRL was calculated from Equation 2: MRL = Mean + 3 |LCMRL1 – LCMRL2|
(2)
In this case, the absolute value of the difference between the LCMRLs serves as a surrogate for the standard deviation, because of the uncertainty of estimating a standard deviation with only two data points. In statistical theory (Chebyshev’s inequality), an interval of 3 standard deviations around the mean of any distribution incorporates the majority (at least 88.9%) of the data points (18). The MRL for each analyte was determined by then rounding the value obtained from either Equation 1 or Equation 2 to one significant digit. The MRL is designed to be a nationally attainable value for laboratories that are to participate in a particular regulatory program. The LCMRL is a laboratory-specific value, but the MRL must be more flexible in that it represents a reasonable performance metric for numerous laboratories. The use of 3s provides that flexibility by ensuring that a high percentage of laboratories across the nation will be able to meet the DQOs, as indicated by Chebyshev’s inequality. Because MRLs are derived from LCMRLs, the DQOs that are inherent to the LCMRL are retained by the MRL. February 1, 2007 / Environmental Science & Technology n 679
TA B L E 2
Single-laboratory LCMRLs and resulting MRLs LCMRL, lowest-concentration minimum reporting level; MDL, method detection limit; ML, minimum level; MRL, minimum reporting level; N/A, not applicable; OLS, ordinary least squares; PQL, practical quantitation limit; VWLS, variance-weighted least squares. EPA method Primary Lab LCMRL number analyte number (μg/L)
How determined?
Lab MDL (μg/L)
ML (3.18 × MDL) (μg/L)
200.7
6
OLS OLS VWLS OLS
2 3 4 4
0.1 0.77 0.1 0.33
OLS OLS OLS VWLS
39 25 25 0.4 0.45 0.21 0.092 0.004
200.9
300
507
508
515.3
525.2
Chromium 1 2 3 3 (no outlier) Cadmium 1 2 4 2 (no outlier) Nitrate 1 2 3 Atrazine 1 2 5 2 (no outlier) Hepta1 chlor 2 5 2,4-Di1 chlorophen- 2 oxyacetic 3 acid Hexachlo- 1 robenzene 2 3
4 7.7 a
a
0.008 0.58 0.8 0.62 0.05 a
0.26
PQL (μg/L)
ThreeTwo-laboratory laboratory MRL MRL (μg/L) from (μg/L) from Equation 2 Equation 1
6.4 9.5 13 13
10
20
N/A
N/A
10
0.042 0.03 0.045 0.03
0.13 0.095 0.14 0.095
2
N/A
1
N/A
0.6
VWLS OLS VWLS OLS VWLS VWLS VWLS
26.4 5 6.5 0.202 0.045 N/A 0.045
84 16 21 0.64 0.14 N/A 0.14
400
N/A
50
1
N/A
0.7
N/A
0.7
VWLS VWLS VWLS OLS OLS OLS
0.002 0.004 N/A 0.165 0.39 0.09
0.0064 0.013 N/A 0.52 1.2 0.29
0.4
0.02
N/A
1
N/A
1
VWLS VWLS VWLS
0.017 0.09 0.02 b
0.054 0.29 0.064
1
0.8
N/A
a Cannot
be determined. b Estimated.
Table 2 summarizes the individual-laboratory LCMRLs and possible values for the MRL. When the LCMRL for one laboratory is greater than the highest spiked concentration, MRLs are determined from data from the two remaining laboratories with determinable LCMRLs. Removal of outliers allows for inclusion of all three laboratories in the calculations for three of the analytes in Table 2. The table also includes each laboratory’s MDL, 10 sigma—3.18 × MDL, or the minimum level (ML)—and the PQL for each primary analyte. Ten sigma has been proposed by the American Chemical Society (5) and in EPA Method 1631 (4) as a reliable level of quantitation. The PQL has long been used by EPA as a level of reliable quantitation. The relationships among the MDL, the ML, and the PQL are as follows. The MDL is determined by multiplying the standard deviation of replicate results at a single concentration (typically seven replicates) by the Student’s t value for n – 1 degrees of freedom and 99% confidence (3). For n = 7, t = 3.14. 680 n Environmental Science & Technology / February 1, 2007
The ML is obtained by multiplying the MDL by 3.18 (4). This is ≈10× the standard deviation in the case of 7 replicates. PQLs have historically been determined through linear regression of proficiency testing (PT) data (national laboratory passing rates), or by multiplying the MDL by 5 or 10 (19). In the regression procedure, the concentration at which 75% of participating laboratories are predicted to meet an analyte’s acceptance criteria is set as the PQL. Thus, in general, MDL < ML < PQL. The main difference between the MRL and these three target quantities is that the MRL is derived from the LCMRL with simultaneous incorporation of precision and accuracy. The MDL is a detection limit, and it is based on precision only; the ML, a multiple of the MDL, is a reporting limit, and it is based on precision only. The PQL is a reporting limit, and it can be established as a multiple of the MDL. In that case, it is based only on precision. Alternatively, if the PQL is derived from linear regression of PT data, it con-
siders accuracy, but it does not explicitly consider precision. Table 2 demonstrates how the MRLs that were calculated for these analytes relate to the ML and the PQL. Outliers are discussed in Supporting Information and by Winslow et al. (15). The information summarized in Table 2 indicates that the MRL typically falls in between the values for the ML and the PQL. Chromium and 24-dichlorophenoxyacetic acid are two exceptions; their MRLs are similar in magnitude to both their MLs and PQLs. The simultaneous incorporation of both precision and accuracy in the determination of the LCMRL provides confidence in the quality of the derived MRLs. Further, because the MRLs are comparable with (but typically less than) PQLs, the MRL represents a reporting level that meets EPA’s need for data of known quality. The MRL also takes into account the ability of laboratories to incorporate precision and accuracy into low-level measurement. The results of the interlaboratory study indicate that use of the LCMRL in determining MRLs is a valid new approach to ensuring quality and consistency in low-level analytical measurements. Simultaneous incorporation of precision and accuracy results in MRLs that are generally lower than existing PQLs but indicative of low-concentration laboratory performance. The LCMRL procedure is flexible, and DQOs can be tailored to fit the needs of any analytical regulatory program. Acceptance limits could be made more or less stringent than ±50%, and the confidence levels could be set at any desired level. John J. Martin is an associate with the Cadmus Group, Inc., in Watertown, Mass. Stephen D. Winslow is a chemist for Shaw Environmental, Inc., and an on-site contractor at the Technical Support Center of the EPA Office of Ground Water and Drinking Water in Cincinnati, Ohio. David J. Munch is the chemistry laboratory manager for the Technical Support Center of the EPA Office of Ground Water and Drinking Water in Cincinnati, Ohio. Address correspondence about this article to Martin at
[email protected].
Acknowledgments This work has been funded in part by the U.S. EPA, Office of Water, Office of Ground Water and Drinking Water, Technical Support Center under contract 68-C-02-026 to the Cadmus Group, Inc., and contract 68-C-01-098 to Shaw Environmental, Inc. This paper has been subject to EPA review and has been approved for publication as an EPA document. Mention of trade names or commercial products does not constitute endorsement or recommendation for use. The authors appreciate the input of the following individuals who contributed to peer review of the design of the interlaboratory study and LCMRL/MRL determinations and/or performed laboratory analyses: Richard Albert, U.S. Food and Drug Administration (retired); William Horwitz, AOAC International; William Foreman, U.S. Geological Survey; Andrew Eaton, MWH Laboratories; Michael D. Wichman, University Hygienic Laboratory; Phillip Godorov, Philadelphia Water Department Laboratory; John V. Morris, EPA Region 5 Laboratory; Lisa Wool, EPA Region 6 Laboratory; William Telliard and other personnel, EPA Engineering and Analysis Division; and Brad Venner, EPA National Enforcement Investigations Center.
Supporting information Supporting information is available for this paper. It contains discussions of the evaluation of curve fitting for instrument
calibration, nonconstant variance over the range of true concentrations, and outlier evaluation as well as graphical examples of how nonconstant variance and outliers affect the determination of an LCMRL.
References (1) Currie, L. A. Detection and Quantification Limits: Origins and Historical Overview. Anal. Chim. Acta. 1999, 391, 105–126. (2) Currie, L. A. Limits for Qualitative Detection and Quantitative Determination. Anal. Chem. 1968, 40, 586–593. (3) Glaser, J. A.; et al. Trace Analyses for Wastewaters. Environ. Sci. Technol. 1981, 15, 1426–1435. (4) EPA Method 1631, Revision E: Mercury in Water by Oxidation, Purge and Trap, and Cold Vapor Atomic Fluorescence Spectrometry; EPA-821-R-02-019; 2002; www.epa. gov/waterscience/methods/1631e.pdf. (5) Keith, L. H.; et al. Principles of Environmental Analysis. Anal. Chem. 1983, 55, 2210–2218. (6) Eaton, A. D.; Clesceri, L. S.; Greenberg, A. E., Eds. Standard Methods for the Examination of Water and Wastewater, 19th ed.; American Public Health Association: Washington, DC, 1995; Method Detection Level, Part 1030 E, pp 1-10–1-11. (7) Childress, C. J. O.; et al. New Reporting Procedures Based on Long-Term Method Detection Levels and Some Considerations for Interpretations of Water-Quality Data Provided by the United States Geological Survey National Water Quality Laboratory. USGS Open-File Report 99-193; 1999; water.usgs.gov/owq/OFR_99-193/ofr99_193.pdf. (8) Gibbons, R. D.; Coleman, D. E.; Maddalone, R. F. An Alternative Minimum Level Definition for Analytical Quantification. Environ. Sci. Technol. 1997, 31, 2071–2077. (9) American Society for Testing and Materials. 2002 Annual Book of ASTM Standards, Standard Practice for Interlaboratory Quantitation Estimate; ASTM International: West Conshohocken, PA, 2002; ASTM D 6512-00, Section 11 Water, 11.01, pp 888–902. (10) Hubaux, A.; Vos, G. Decision and Detection Limits for Linear Calibration Curves. Anal. Chem. 1970, 42, 849–855. (11) Sanders, P. F.; Lippincott, R. L.; Eaton, A. A Pragmatic Approach for Determining Quantitation Levels for Regulatory Purposes. In Proceedings of the Water Quality Technology Conference, Boston, MA, 1996; American Water Works Association: Washington, DC, 1996; pp 104–114. (12) Oxenford, J. L.; McGeorge, L. J.; Jennis, S. W. Determination of Practical Quantitation Levels for Organic Compounds in Drinking Water. J. AWWA 1989, April, 149–153. (13) Hertz, C. D.; et al. Minimum Reporting Levels Based on Precision and Accuracy for Inorganic Parameters in Water. In Proceedings of the Water Quality Technology Conference, Toronto, Nov 15–19, 1992; American Water Works Association: Washington, DC, 1992; pp 647–665. (14) Kimbrough, D. E.; Wakakuwa, J. Quality Control Level: An Alternative to Detection Levels. Environ. Sci. Technol. 1994, 28, 338–345. (15) Winslow, S. D.; et al. Statistical Procedures for the Determination and Verification of Minimum Reporting Levels for Drinking Water Methods. Environ. Sci. Technol. 2006, 40, 281–288. (16) EPA. Statistical Protocol for the Determination of the Single-Laboratory Lowest Concentration Minimum Reporting Level (LCMRL) and Validation of Laboratory Performance at or below the Minimum Reporting Level (MRL); EPA-815-R-05-006; Nov 2004; www.epa.gov/ safewater/methods/sourcalt.html#Mlcmrl. (17) LCMRL Calculator. EPA Office of Ground Water and Drinking Water, 2005; www.epa.gov/safewater/methods/sourcalt.html#Mlcmrl. (18) Montgomery, D. C.; Runger, G. C. Applied Statistics and Probability for Engineers; John Wiley: New York, 1994. (19) Analytical Feasibility Support Document for the SixYear Review of Existing National Primary Drinking Water Regulations (Reassessment of Feasibility for Chemical Contaminants). EPA-815-R-03-003; EPA Office of Ground Water and Drinking Water: Washington, DC; 2003; www. epa.gov/safewater/standard/review/pdfs/support_6yr_ analytical_final.pdf. February 1, 2007 / Environmental Science & Technology n 681
