Environ. Sci. Technol. 2001, 35, 3350-3355
Meeting Data Quality Objectives with Interval Information CHARLES K. BAYNE* Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831 AMY B. DINDAL AND ROGER A. JENKINS Chemical and Analytical Sciences Division, Oak Ridge National Laboratory, Oak Ridge, Tennessee 37831 DEANA M. CRUMBLING Environmental Protection Agency, Washington, D.C. 20460 ERIC N. KOGLIN Environmental Protection Agency, Las Vegas, Nevada 89193
Immunoassay test kits are promising technologies for measuring analytes under field conditions. Frequently, these field-test kits report the analyte concentrations as falling in an interval between minimum and maximum values. Many project managers use field-test kits only for screening purposes when characterizing waste sites because the results are presented as semiquantitative intervals. However, field-test kits that report results as intervals can also be used to make project-related decisions in compliance with false-rejection and false-acceptance decision error rates established during a quantitative data quality objective process. Sampling and analysis plans can be developed that rely on field-test kits to meet certain data needs of site remediation projects.
Introduction The Data Quality Objectives (DQO) process (1-3) is a sevenstep systematic planning approach based on the scientific method recommended by the U.S. Environmental Protection Agency (EPA) and other agencies for project planning to characterize and clean up waste sites. The seven steps of the DQO process are as follows: (1) state the problem; (2) identify the decision; (3) identify the inputs to the decision; (4) define the boundaries of the study; (5) develop a decision rule; (6) specify limits on decision errors; and (7) optimize the design. Implementing the DQO process is conceptually straightforward. In steps 1-4, the project team establishes a clear definition of the problem, gathers background information, and evaluates the historical perspective of the waste problem. These steps are very important but will not be the focus of this paper. In step 5, the DQO process combines the actions of the previous planning steps into a statement of action. A decision rule takes the form, “If Statement, then Action” and causes the decision-maker to choose among alternative courses of actions. An example of a decision rule is as follows: if the estimated mean of polychlorinated biphenyl (PCB) concentration in waste oil is equal to or above 50 ppm, then classify the waste as Toxic Substance Control Act (TSCA) waste; * Corresponding author phone: (865)574-3134; fax: (865)574-3527; e-mail:
[email protected]. 3350
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 35, NO. 16, 2001
otherwise, classify the waste as Resource Conservation and Recovery Act (RCRA) waste. Note that the decision rule includes the statistic used for the decision, the concentration action level, and the specific course of action. Step 6 of the DQO process specifies that decision-makers establish acceptable limits for the decision rule error rates. Decision errors occur either by incorrectly rejecting the action of the decision rule and accepting the alternative course of action or by incorrectly accepting the action of the decision rule and rejecting the alternative course of action. A false rejection of the decision corresponds to the more severe decision error of falsely rejecting the “Action”, and a false acceptance of the decision corresponds to the less severe decision error (1). An acceptable probability of a falserejection decision (i.e., error rate) is usually determined in consultation with a regulatory agency or in compliance with specific regulations that are primarily concerned with minimizing risks to human or environmental health. The probabilities of making either false-rejection or false-acceptance decision errors depend on the variability of the measurement process and the number of environmental samples measured. Step 7 of the DQO process identifies the most resourceeffective sampling design for generating data that are expected to satisfy the specified decision rule error rates. The project team must decide where to select the sampling locations and the number of samples to measure. Various sampling plans are usually considered in order to balance the available resources with the desired confidences in the data and in the decisions based on the data. Many analytical methods are performed in the laboratory environment because traditional laboratory methods tend to minimize analytical measurement error in order to meet DQO requirements. However, matrix heterogeneity, sampling procedures, sample holding times, and shipping conditions introduce additional sources of variation that make the total variability of the sampling and measurement process much larger than the analytical error (4). Field analytical technologies offer the ability to address issues related to representativeness and other sampling concerns. These and other considerations such as cost can make these alternative methods more attractive than traditional laboratory methods (5). The U.S. EPA’s Office of Solid Waste (6, 7) has accepted for inclusion in SW-846 a number of field methods. Furthermore, The U.S. EPA’s Environmental Technology Verification (ETV) program tests commercially available field technologies to objectively characterize and document their performance under field conditions (http://www.epa.gov/etv, 8-11). Some of these field-test kits (12-14) report analytical results in the form of an interval falling between minimum and maximum values. For example, a reported analyte result that is greater than or equal to 0 ppm but strictly less than 1 ppm can be represented by the half-open interval notation [0, 1). A set of possible intervals for a particular field-test kit may be [0, 1), [1, 10), and [10, ∞) or could be some other set of intervals. Frequently, dilution schemes can be used to accommodate user-defined intervals that can better address site-specific data needs. For instance, if a sample result falls in a [10, ∞) interval, the sample can be diluted by a factor of 4 to see whether it falls in a [10, 40) interval or a [40, ∞) interval. Many times these field-test kits are viewed as suitable only for rapid screening because of the semiquantitative nature of interval measurements. However, as shown in this 10.1021/es001572m CCC: $20.00
2001 American Chemical Society Published on Web 07/12/2001
discussion will be limited to the simplest type of decision rule. The number of samples classified in intervals less than the TV interval follows a binomial probability distribution (see refs 16, p 162, and 17):
Pr(K ) k) ) B(n, k)[Pr(Y ) 0)]k[1 - Pr(Y ) 0)]n-k (1)
FIGURE 1. Illustration of assigning Y ) 0 or Y ) 1 to reported analyte concentrations relative to the target value. paper, these field-test kits can also be used to rigorously address steps 5-7 of the DQO process.
Methods The balance of this paper discusses the statistical application of interval data to project-specific decision rules. The decision rule is based on a comparison of a statistic (e.g., single measurement, average, median, maximum, etc.) of the analyte concentrations to a target value (TV), which is usually the regulatory action level. The TV must be contained in one of the reporting intervals for a field-test kit designated as the TV interval. Let Y be a function that assigns the value of 0 or 1 to the reported intervals. Figure 1 illustrates the function Y that has a value of 0 if a reported analyte concentration (CR) is contained in any interval with maximum concentrations less than the TV interval and that a value of 1 if a reported analyte concentration statistic is in the TV interval or in any interval with a minimum concentration greater than the TV interval. The Y function defines all reporting intervals into two interval types: those below the TV interval and those including or above the TV interval. The outcome of the decision rule will depend on whether the Y values are equal to 0 or 1 for a number of measured of samples. To illustrate, suppose the project team selects a sampling plan that will analyze n samples from a drum of waste by a field-test kit in order to make a decision about the disposal of the waste on a drum-by-drum basis (i.e., the decision unit is a single drum). The simplest situation arises when the team believes that an assumption of homogeneity for the waste inside each drum is reasonable (e.g., based on process knowledge). The decision rule would involve a statistic that counts the number of results reported for a drum for which Y ) 0 and the number of reported results for which Y ) 1. The simplest type of decision rule would be as follows: If all reported results have Y ) 0, then take action A. If any reported results have Y ) 1, then take action B. Action A will usually be less expensive or easier to implement (e.g., classify the soil drum as a RCRA waste for disposal purposes) than action B (e.g., classify the soil drum as a TSCA waste for disposal purposes). There can also be variations on the decision rule by allowing some portion of the reported results to have Y ) 1 and yet still perform action A rather than requiring that 100% of the reported results to have Y ) 0 to perform action A. These variations have been examined in acceptance sampling methodology (15), but this
where K is the number of samples classified as Y ) 0; k ) 0, 1, 2, ..., n; B(n, k) is the number of ways that n items can be partitioned into sets of k items; B(n, k) ) n!/[k!(n - k)!] where “!” is the factorial operation (e.g., 4! ) 4 × 3 × 2 × 1 ) 24); and Pr( ) is the probability of an event. Decision error rates can be defined in terms of this binomial probability distribution. The EPA policy assigns the allowable false-rejection error rate (FR) to the decision error with the most severe consequences (1). This event occurs when all samples are classified in intervals with Y ) 0, but the actual true sample concentrations are greater than or equal to the TV value. The FR can be expressed mathematically by
FR ) Pr(K ) n given CT g TV) ) B(n, 0)(PU)n(1 - PU)0 ) (PU)n (2) where CT is the true analyte concentration; Pr(K ) n given CT g TV) is the conditional probability that all samples are classified in an interval less than the TV interval given that the true analyte concentration is actually greater than or equal to the TV value; and PU is the probability of underestimating the analyte concentration interval for a single sample by assigning it to an interval less than the TV interval when the true concentration is greater or equal to the TV value. The probability PU depends on the true concentration and becomes smaller as the true concentration increases above the TV value. The false-acceptance error rate (FA) is equal to the probability of any reported sample concentrations being classified in an interval with Y ) 1 given that all true sample concentrations are less than the TV value. The number of CR values classified as Y ) 0 is less than n(K < n). The FA depends on the probability of the field-test kit overestimating an analyte concentration for at least one sample. This probability is equivalent to 1 minus the probability of classifying all n reported sample concentrations as Y ) 0:
FA ) Pr(K < n given CT < TV) ) 1 - Pr(K ) n given CT < TV) ) 1 - (1 - PO)n (3) where PO is the probability of overestimating the analyte concentration interval for a single sample by assigning it to the TV interval when the true concentration is less than the TV value. The probability PO also depends on the true concentration and will get smaller as the true concentration decreases below the TV value. Note that if PU in eq 2 and PO in eq 3 were 0, then both FR and FA would be 0 and only one sample is required. Estimating Sample Size. The sample size (n) required to meet the desired FR and FA specified during project planning can be solved by using eq 2 to form eq 4a and eq 3 to form eq 4b:
log(FR) log(PU)
(4a)
log(1 - FA) log(1 - PO)
(4b)
n) and
n)
VOL. 35, NO. 16, 2001 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
3351
The procedure would be to calculate the sample size from both formulas and round up the sample size to the next integer. Equations 2 and 3 show that FR will decrease due to the rounding while the FA will increase. The actual FR and FA values should be recalculated using eqs 2 and 3 with the integer sample size from eq 4a. To complete these calculations, values for PO and PU may be derived from analytical performance information, estimated from a preliminary pilot study, or modeled as a probability distribution of measurement error. Estimating PU and PO. Estimates of the probabilities PU and PO are not equivalent to the percentage of false-negative analytical results that are determined from samples containing the analyte and of false-positive analytical results that are determined from blank samples (18) because those falsenegatives and false-positives are relative to the detection limit of the target analyte(s). In contrast, the probabilities PU and PO are defined relative to a selected target concentration of the target analyte(s). In some cases, PU and PO may be derived from performance information documented through field (http://www.ornl.gov/etv, 19-22) or lab studies (7, 13, 14). Vendor information may also help establish initial estimations of these probabilities. Direct applicability of those derived PU and PO values to any particular site-specific matrix will depend on the degree of similarity between the projectspecific matrix and the matrix of previous studies. If direct applicability cannot be demonstrated, a pilot study is needed to determine project-specific values for PU and PO that depend on the true concentration of the sample and its distance from the target value. This means that PU and PO are not single values but are distributions of values. Two experimental approaches can be used to approximate PU and PO: (i) run replicate samples with known concentrations near the TV value that are both below and above the TV value; (ii) run replicate samples with known concentrations that are equally spaced over the concentration range of interest. The first experiment is a conservative approach that gives nearly maximum PU and PO values because the selected experimental concentrations are near the TV value. These conditions should give the highest misclassification probabilities. The second experiment can be used to develop probability models for PU and PO as a function of the true concentration (13). For practical reasons and to be conservative, the first approach can be used, and the PU and PO approximations would be considered upper bound constants on the misclassification probabilities over all true concentration levels. The number of samples needed for the pilot study will depend on the desired certainties for the PU and PO estimates. A 95% confidence interval (CI) (16, p 333) about an estimated probability can be used to express the allowable uncertainty. As more samples are used in the pilot study, the width of the 95% CI will decrease (i.e., the uncertainty on the probability estimates decreases). Suppose a pilot study was performed to estimate PU that analyzed 50 samples with spiked analyte concentrations at or just above the TV value, and the fieldtest kit reported five intervals less than the TV interval (i.e., Y ) 0). The point estimate for PU would be 5/50 ) 0.10 with a 95% CI of (0.033, 0.218) (16, p 333). Note the 95% CI is not symmetric about 0.10; there is a difference of 0.067 on the lower side (0.10 - 0.033) but a difference of 0.118 on the upper side (0.218 - 0.10). Because of this asymmetry, the uncertainty about the PU point estimate is defined as the average half-width of the CI, which is the width of the CI interval divided by 2 [e.g., (0.218 - 0.033)/2 ) 0.0925]. A similar result can be achieved for a pilot study to estimate PO by spiking samples just below the TV value and counting the number of reported intervals classified as Y ) 1. Figure 2 shows the number of samples needed for a pilot study to achieve various half-widths for a 95% CI for either PU or PO. The half-width lengths increase with the number 3352
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 35, NO. 16, 2001
FIGURE 2. Half-widths of a 95% confidence interval (CI) for probabilities estimated from the number of nonconforming samples in experiments with n ) 25, 50, 75, 100, and 200 samples.
FIGURE 3. Half-widths of a 90% confidence interval (CI) for probabilities estimated from the number of nonconforming samples in experiments with n ) 25, 50, 75, 100, and 200 samples. of nonconforming samples. So more pilot study samples are required to maintain the same degree of statistical confidence when there is an increased possibility that a kit will either overestimate or underestimate an analyte concentration. Figure 3 shows the number of samples needed for different half-widths for a 90% CI. Reasonable half-width values can be achieved with fewer samples if a 90% CI is used rather than a 95% CI. Selection of the level of confidence in the pilot study depends on the desired confidence needed for PU and PO when calculating FR and FA for the decision rule.
Results and Discussion The following hypothetical example serves to demonstrate how semiquantitative field-test kits that report results as intervals may be used to make statistically defensible project decisions according to steps 5-7 of the DQO process. Additionally, since the focus is on how performance information for an analytical technology is applied to statistical decision-making, it will be assumed that the waste samples for this example were homogeneous.
Background and Problem Statement. Various parcels of land were discovered to be diffusely contaminated with PCBs from past land disposal of dredged contaminated river sediments. Previous site-characterization efforts had revealed a few defined hotspots with concentrations greater than 100 ppm and were removed separately. The rest of the areas of concern exhibited PCB values ranging from less than 1 ppm to 90 ppm but in no predictable areal or depth patterns. Mean concentrations for each soil area that exceeded permissible risk levels required remedial action in the form of soil removal. The soil was to be drummed for disposal according to RCRA/TSCA classification discussed at the start of this paper. This action generated a large number of drums, but those that exceed the TSCA limit of 50 ppm could not be identified. Disposal all of the drums as TSCA waste would have been cost-prohibitive, so it was decided to test each drum to determine whether the PCB concentration of the contents exceeded the regulatory threshold. The excavation and drumming process involved a thorough mixing of the soil that comprised each drum. The general decision rule states that a drum would be treated as RCRA waste if its average PCB concentration is less than 50 ppm and treated as TSCA waste if its average PCB concentration is equal to or greater than 50 ppm. The project team will use a PCB immunoassay test kit to measure the PCBs in each drum. Randomly collected soil samples from each homogeneously mixed drum will be analyzed to determine if the measured concentrations are in one of the four intervals reported by the kit: [0, 1), [1, 10), [10, 40), or [40, ∞). This interval notation describes the concentration ranges of 0 ppm e PCB < 1 ppm, 1 ppm e PCB < 10 ppm, 10 ppm e PCB < 40 ppm, and PCB g 40 ppm, respectively. For this example, the TV value of 50 ppm is not an interval limit but is within the highest interval. The project team decided that a drum would be processed as TSCA waste if any one of the analyses from a single drum indicated a concentration in this interval [40, ∞). Otherwise, the drum would be processed as RCRA waste. In agreement with the regulator, the project team determined that a decision rule based on the performance of the analytical method would determine the number of samples to be tested from each drum. Project-Specific Decision Rules. If all of the PCB sample results from a drum are in intervals less than the [40, ∞) interval, then process the drum according to RCRA requirements. If any of the PCB sample results from a drum are in the [40, ∞) interval, then process the waste drum according to TSCA requirements. Setting Decision Error Limits. The project team’s goal was to ensure that all PCB-contaminated soils were disposed in accordance with regulatory requirements. The more severe decision error would be for a drum with PCB concentration that exceeded the 50 ppm limit to be erroneously processed as RCRA waste. A false-rejection decision is made when it is concluded that a drum contains less than 50 ppm PCBs when actually the drum is “hot” (i.e., drum concentration exceeds 50 ppm). The regulators and the project team agreed that the FR error percentage for processing a “hot” drum erroneously as RCRA waste should be less than 5% (i.e., FR e 0.05). Therefore, a sufficient number of samples must be taken from each drum to address the uncertainties of the analytical method so that the maximum risk for incorrectly classifying a drum as RCRA waste is 5%. The project team did not want to process an excessive number of drums as TSCA waste if the average PCB concentration was truly less than 50 ppm because of the expense. A false-acceptance decision is made when it is concluded that a drum is “hot” when in actuality the drum contains soil with less than 50 ppm PCBs. After considering
TABLE 1. Number of Field-Test Kit Results Relative to Target Value of 50 ppm Level field-test kit results
no. of samples with known concn
[10, 40)
[40, ∞)
50 (at 50 ppm) 50 (at 35 ppm)
3 44
47 6
the guidelines presented in Section 1.1 of the U.S. EPA’s Guidance for Data Quality Assessment (1), the project team set the false-acceptance decision error rate at FA ) 0.10 in order for a 10% chance of processing a drum as TSCA waste if the true PCB concentration for a drum is less than 50 ppm. Calculating the Number of Samples. The project team needed to determine values for PU and PO before calculating the number of samples that must be taken from each drum to meet the decision goals. The project team decided to perform a pilot study because the characteristics of the ageddredged materials were dissimilar to the matrixes used in the technology performance information on the field-test kit. The pilot study was designed to approximate maximum misclassification errors as discussed previously. The pilot study consisted of 100 experimental samples: 50 experimental samples having PCB concentrations at 35 ppm and 50 experimental samples having PCB concentrations at 50 ppm. The lower concentration falls in the interval [10, 40), which is the first interval below the TV interval. The sitespecific experimental samples for the pilot study were drawn from stored soil samples that had been collected and analyzed during prior site-characterization activities. Table 1 shows the data for this hypothetical pilot study. The project team used the experimental data from Table 1 to estimate the upper bound probabilities for the two types of misclassifications. Samples with known concentrations at 50 ppm are used to estimate PU as PU ) 3/50 ) 0.06. Samples with known concentrations at 35 ppm are used to estimate PO as PO ) 6/50 ) 0.12. Table 2 shows the estimates of PU and PO as well as their corresponding 90% and 95% CIs. The 95% and 90% CIs on PU and PO would have been smaller, as seen in Table 2, if the team had performed the pilot study with 200 samples (100 at 50 ppm and 100 at 35 ppm) given the same estimated PU and PO (e.g., PU ) 0.06 and PO ) 0.12) values. Using the misclassification point estimates of PU and PO for the performance of the immunoassay test kit with their site-specific matrix, the project team then determined the number of samples needed from each drum to make a statistically valid decision at the specified decision error limits (i.e., FR ) 0.05, FA ) 0.10). The number of samples required to meet the specified FR is calculated using eq 4a:
n)
log(FP) log(0.05) -1.301 ) 1.06 ≈ 2 ) ) log(PU) log(0.06) -1.222
The sample size is rounded up to the next integer, an operation that will decrease the FR for the decision rule. The project team would thus analyze two samples from each drum to meet the decision rule’s FR requirement. Table 2 shows that using the larger pilot study to estimate PU makes the 95% CI smaller but that the corresponding number of drum samples calculated by eq 4a for the upper and lower confidence limits remain the same. This effect is due to rounding up the sample sizes to the next integer. Although rounding up the number of drum samples maintains the required limit on the FR, it has the opposite effect on the FA (the possibility that the team will process drums unnecessarily as TSCA waste). The error rate of a falseVOL. 35, NO. 16, 2001 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
3353
TABLE 2. Effects of the Number of Pilot Study Samples on Confidence Intervals (CIs)a 90% CI
95% CI
pilot study total no. of expts
PU (lower, upper) n (nlow, nupper)
PO (lower, upper) n (nlow, nupper)
PU (lower, upper) n (nlow, nupper)
PO (lower, upper) n (nlow, nupper)
100
0.060 (0.017, 0.148) 2 (1, 2) 0.060 (0.026, 0.115) 2 (1, 2)
0.120 (0.054, 0.223) 1 (2, 1) 0.120 (0.071, 0.187) 1 (2, 1)
0.060 (0.013, 0.165) 2 (1, 2) 0.060 (0.022, 0.126) 2 (1, 2)
0.120 (0.045, 0.243) 1 (3, 1) 0.120 (0.064, 0.200) 1 (2, 1)
200
a The number of drum samples to meet the criteria of FR ) 0.05 and FA ) 0.10 are calculated for each estimate and its corresponding CI. CIs are based on the F-distribution (see ref 14, pp 333-337).
Different sampling plans can be evaluated graphically using decision performance curves. These curves plot the probability of taking action B on the y-axis versus the true analyte concentration on the x-axis. Classical analytical methods that measure analyte concentrations on a continuous scale produce S-shaped curves (1, 2) for the decision performance curves. Analytical methods that report results as intervals produce a step function with a constant value of FA for true analytical concentrations less than the TV interval and a constant value of 1 - FR for true analytical concentrations greater than or equal to TV. Figure 4 shows the decision performance curve for the above example for the sampling plan of n ) 2, PU ) 0.06, and PO ) 0.12. The false-rejection decision error is FR ) 0.004, and the falseacceptance decision error is FA ) 0.226.
Acknowledgments FIGURE 4. Decision performance curve; an example of a PCB test kit using 2 soil samples per drum. acceptance decision actually increases with increasing n per drum because the chance that the field-test kit will produce at least one result that overestimates the PCB concentration increases with continued testing. Using eq 4b, the sample size required to meet the planned FA requirement (0.10) is
n)
log(1 - FN) log(1 - 0.10) -0.046 ) 0.82 ≈ 1 ) ) log(1 - PO) log(1 - 0.12) -0.056
The fractional sample size must, of course, be rounded up to n ) 1. When n ) 1, the value of FA is 0.12 which is only slightly larger than the project team’s goal of FA ) 0.10. However, because two samples must be taken in order to meet the FR criteria, the FA must be calculated using n ) 2, and then FA increases to 0.23, which doubles the risk for unnecessary TSCA disposal. This increase in risk is due to the estimated PO for the chosen field-test kit and should be evaluated when selecting among different field-test kits. The PU and PO estimates for a field-test kit translates into the number of samples and FA risk to the waste generator that must be balanced with other costs during the systematic planning process. After all options were considered, the project team in this example decided that the sampling procedure would randomly collect two soil samples from each drum and analyze the samples with the field-test kit. The project team will process the drum as RCRA waste if both results are less than 50 ppm and will process the drum as TSCA waste if either sample tests greater or equal to 50 ppm. The actual decision error limits for this sampling plan are 0.004 for FR (the falserejection decision error rate of erroneously disposing of a drum as RCRA waste) and 0.23 for FA (the false-acceptance decision error rate of erroneously disposing of a drum as TSCA waste). 3354
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 35, NO. 16, 2001
This work was supported by the U.S. Environmental Protection Agency’s Environmental Technology Verification Program in conjunction with the U.S. Department of Energy. Oak Ridge National Laboratory is operated by UT-Battelle, LLC for the U.S. Department of Energy under Contract DEAC05-00OR22725. We thank R. W. Counts and E. L. Frome of the Computer Sciences and Mathematics Division and S. K. Treat of the Computing, Information, and Networking Division for reviewing this manuscript and offering valuable suggestions. We also thank the reviewers and Associate Editor Dr. Mitchell J. Small for their comments that improved this paper.
Literature Cited (1) U.S. Environmental Protection Agency. Guidance for the Data Quality Objective Process, EPA QA/G-4; EPA/600/R-96/055; U.S. EPA: Washington, DC, August 2000. (2) American Society for Testing and Materials (ASTM). Standard Practice for Generation of Environmental Data Related to Waste Management Activities Development of Data Quality Objectives; D5792-95; ASTM: Philadelphia, 1997. (3) American Society for Testing and Materials (ASTM). Standard Practice for Generation of Environmental Data Related to Waste Management Activities Quality Assurance and Quality Control Planning and Implementation; D5283-92; ASTM: Philadelphia, 1997. (4) Barnard, T. E. Extending the Concept of Data Quality Objectives to Account for Total Sample Variance. In Principles of Environmental Sampling, 2nd ed.; Keith, L. H., Ed.; American Chemical Society Professional Reference Book; American Chemical Society: Washington, DC, 1996; Chapter 9. (5) Nesbitt, K. J.; Carter, K. R. Integration of Immunoassay Field Analytical Techniques into Sampling Plans. In Principles of Environmental Sampling, 2nd ed.; Keith, L. H., Ed.; American Chemical Society Professional Reference Book; American Chemical Society: Washington, DC, 1996; Chapter 36. (6) Lesnik, B. Food Agric. Immunol. 1994, 6, 251. (7) U.S. Environmental Protection Agency. Test Methods for Evaluating Solid Waste-Physical/Chemical Methods; EPA Publication SW-846; U.S. Environmental Protection Agency, Office of Solid Waste: Washington, DC; http://www.epa.gov/SW-846/sw846.htm. (8) Oak Ridge National Laboratory. Technology Demonstration Plan: Evaluation of Polychlorinated Biphenyl (PCB) Field
(9) (10) (11) (12) (13) (14) (15) (16) (17) (18) (19)
Analytical Techniques; Chemical and Analytical Sciences Division, Oak Ridge National Laboratory: Oak Ridge, TN, September 1998. Powell, Dan, EPA TechTrends 1997, October; EPA542-N-97-005. Billets S.; Koglin, E. Environ. News 1997, Summer. Hansen P. EM Mag. 1997, May. Immunochemical Technology for Environmental Applications; Aga, D. S., Thurman, E. M., Eds.; American Chemical Society: Washington, DC, 1997. Waters, L. C.; Palausky, A.; Counts, R. W.; Jenkins, R. A. Field Anal. Chem. Technol. 1997, 4, 227. Waters, L. C.; Smith, R. R.; Stewart, J. H.; Jenkins, R. A. J. AOAC Int. 1994, 77, 1664. Duncan, A. J. Quality Control and Industrial Statistics; Richard, D., Ed.; Irwin, Inc.: Homewood, IL, 1974. Sachs, L. Applied Statistics: A Handbook of Techniques, 2nd ed.; Springer-Verlag: New York, 1984. Esterby, S. R. Environ. Monit. Assess. 1989, 12 (Nov), 103-112. Berger, W.; McCarty, H.; Smith, R.-K. Environmental Laboratory Data Evaluation; Genium Publishing Corp.: Schenectady, NY, 1996. Dindal, A. B.; Bayne, C. K.; Jenkins, R. A.; Billets, S.; Koglin, E. N.; Environmental Technology Verification Report, Immunoassay Kit: Hach Company PCB Immunoassay Kit; EPA/600/R-98/110;
U.S. Environmental Protection Agency, Office of Research and Development: Washington, DC, August 1998. (20) Dindal, A. B.; Bayne, C. K.; Jenkins, R. A.; Billets, S.; Koglin, E. N. Environmental Technology Verification Report, Immunoassay Kit: Strategic Diagnostics Inc., D TECH PCB Test Kit; EPA/600/ R-98/112; U.S. Environmental Protection Agency, Office of Research and Development: Washington, DC, August 1998. (21) Dindal, A. B.; Bayne, C. K.; Jenkins, R. A.; Billets, S.; Koglin, E. N. Environmental Technology Verification Report, Immunoassay Kit: Strategic Diagnostics Inc., EnviroGard PCB Test Kit; EPA/ 600/R-98/113; U.S. Environmental Protection Agency, Office of Research and Development: Washington, DC, August 1998. (22) Dindal, A. B.; Bayne, C. K.; Jenkins, R. A.; Koglin, E. N. Environmental Technology Verification Report, Immunoassay Kit: EnviroLogix Inc., PCB in Soil Tube Assay; EPA/600/R-98/ 173; U.S. Environmental Protection Agency, Office of Research and Development: Washington, DC, December 1998.
Received for review August 9, 2000. Revised manuscript received May 23, 2001. Accepted May 23, 2001. ES001572M
VOL. 35, NO. 16, 2001 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
3355
