Analvtical issues in compliance monitoring 1 .
J
e.
e
The E P A methods f o r verifying the adherence of a water discharger to the terms of his permit do not ensure accurate results, argues the author. Some modifications are suggested
James K. Rice Consulting Engineer Olney, M d . 20832 Four specific analytical issues arise whenever a regulatory agency and a pollutant discharger collect and analyze a water sample to determine compliance with the terms of a N a tional Pollutants Discharge Elimination System (NPDES) permit: Are all of the steps that are actually followed in the sampling and analysis spelled out in the standard, va I i d a t ed met hod ‘? Does the method contain a statement of overall precision, or only a single-laboratory precision? Is the method applicable to matrices-that is, types of aqueous environments (such as drinking water)other than those for which it was validated? Are consistent definitions used for all such terms as “limit of detection,” and are they based on overall precision? Settling these questions is essential to obtaining the reliable, accurate information necessary for the permit system to work. The sanctions for violating the limitations set down in an N P D E S permit are severe: Fines and penalties can total tens of thousands of dollars per day. Thus there must be means of known accuracy to monitor each source of discharge.
A group particularly concerned about these issues is the steam electric power industry, which discharges large volumes of water at many of its plants. Through the Utility Water Act Group (UWAG), the industry has studied the adequacy of analytical methods published or proposed by the EPA, in an effort to ensure that methods of known precision and accuracy are available. The results of these studies suggest how the standard methods should be modified to address the four key questions raised above. 1. Steps both inside and outside the laboratory should be included in the standard method. At present, the methods established by EPA cover specific steps to be followed in the analysis, including their sequence, the use of specific reagents, and the use of appropriate instruments. Steps such as sample container preparation, filling, and transport are either part of the particular method or part of the general instructions applicable to many methods (and set forth in 40 CFR Part 136 and associated references). All of these steps, both inside and outside the laboratory, are deemed essential to the reliable performance of the methods, and are thus described in detail. Interlaboratory uaiidation of the methods, however, has generally omitted-without explicitly stating the fact-the sample-handling portion of the procedure, which is the portion
0013-936X/80/0914-1455$01.00/0 @ 1980 American Chemical Society
most greatly influenced by conditions outside of the laboratory. Many of the validations-and thus the interlaboratory precision datahave, for example, been obtained by sending replicate samples of concentrated solutions to the participating laboratories. A better procedure would use replicates of a homogeneous sample of an actual waste discharge at the concentration at which it was collected. When the concentrations of pollutants in discharges were limited to the mg/L, or even the 0.1-mg/L ranges, omitting the preparation, handling, and transport of actual samples from the validation protocol was arguably a practical, justifiable decision. These steps were not thought a significant source of random error relative to the errors of the other portions of the procedure. But the analysis of pollutants in the 0.1-10 y g / L range has now made the random errors occurring outside the laboratory a major part of the overall error. The significance of results in this range is thus likely to be greatly overstated if the precision data is based only, or largely, on insidethe-laboratory error. Consequently, apparent violations of limits set within these low-concentration ranges may not be violations a t all. 2 . A determination of ooerall precision is required, When regulated and regulator each secure a split sample of an effluent and Volume 14, Number 12, December 1980
1455
Table 1 presents the overall precision data obtained by UWAG from an interlaboratory test of the graphitefurnace atomic absorption methods proposed by the EPA in the Dec. 3, 1979, Federal Register. The tests included preparation of sample bottles by each participant, splitting of an actual sample in the field, and transport to the laboratories. Table 2 illustrates the effects on precision of the random errors introduced by including many laboratories and by including the sample-handling steps. The UWAG-pooled singleoperator precision, SO,is much larger than the S Oreported by EPA, the latter presumably obtained in a single laboratory. The overall precision, St, is much larger than either reported SO. Finally, a U W A G test of the standard amperometric methods for free available chlorine and total residual chlorine showed a single-operator
each then analyzes his sample according to precisely the same standard method, there is a high probability that each will report a different answer. One laboratory may report a parameter “not detected” while the other reports a numerical value; one may produce a value that is within the legal limit, the other a value that is not. The only scientific means to resolve the conflicting answers is to assess them in terms of the overall precision of the analytical method employed. The overall precision, S,, is determined by conducting comparative tests with a number of different laboratories on identical samples. S , is a measure of the random variation in the results and is equal to the standard deviation, a statistical measure of such variation. (The procedures for obtaining S , are fully set forth in A S T M D2777, “Standard Practice for Determination of Precision and Bias of Methods of Committee D-19 on Water.”)
TABLE 1
Overall precision of a standard method Concenlratlon range
Preclslon
Arsenic Chromium Copper Nickel Zinc
+ + S, = 3.18 + 0.120 x S, = 13.43 + 0.203 x
St = 5.11 0.130 x S, = 3.97 4- 0.120 x S, = 3.59 0.073 x
0-100 pg/L
0-150 0-150 0-150 0-150
r = 0.70 r = 0.88 r = 0.74 r = 0.91 r = 0.78
S, = overall precision, pg/L x = concentration, pg/L Overall precision was determined for graphite-furnace atomic absorption spectroscopic analyses carried out according to EPA 600/4-79-020, “Methods for Chemical Analysis of Water and Wastes,” modified to include sample handling steps. Source: “Round Robin InterlaboratorySampling and Analysis Program,” Utility Water Act Grwp, January 1980.
TABLE 2
Comparison of precision data UWAG
so
s,
Arsenic Chromium Copper Nickel Zinc
11.6 pgIL 10.0 7.2 9.2 23.6
5.5 pglL 5.1 3.9
6.5 5.7
EPA SO
1.1 pg/L 0.2 NA NA NA
NA: no published results available Overall precisions, S,, and single-operator precisions, So, were calculated for graphite-furnace atomic absorption methods using 50-pg/L samples. Source: “Round Robin Interlabfatory Samplingand Analysis hogram,” Utility Water Act Group, January 1980.
1456
Environmental Science & Technology
precision similar to that reported by EPA, but an overall precision of more than 2.5 times that. Overall precision should also be taken into account in setting technology-based effluent limits. Studies of the application of a particular control technology to a discharge may project unrealistically low concentration limits if the limits are determined by analytical methods for which only singleoperator or restricted interlaboratory precision data are available. It may be argued that errors in the results of a single operator cancel out when the efficiency of a treatment process is determined-the bias in the input concentration canceling the bias in the output concentration; thus the singleoperator precision of the method could be a satisfactory yardstick. This might be true if the process were to be used only by the single operator assessing it. But verification by other laboratories is obviously necessary; single operators frequently find themselves with clearly anomalous results. Thus, application of single-operator results to others requires a t the very minimum the use of analytical methods for which the overall precision has been determined. 3. A method must be applied only to those matrices f o r which it was ualidated. Determination of the overall precision of a method for a single matrix does not allow application of the method universally. Although the importance of matrix effects, and the need to employ special techniques to minimize them, is recognized throughout the current compliance monitoring methods, validation protocols take them into account only to a limited extent. Currently, for validation of methods proposed as equivalent to the published methods, EPA requires that data be collected on discharges from industries in the five Standard Industrial Classification (SIC) codes or point source subcategories that involve the most N P D E S permits or the largest number of total dischargers. (A further requirement is that samples be collected over a representative concentration range for each type of industry or discharge.) While this may be the only practical approach to validating a method on a national basis, it must be recognized that even within an S I C category or source subcategory there may be discharges of widely different character. Thus there may be substantial matrix effects that may render the overall precision previously found for the selected discharge incorrect for results obtained on other discharges.
that has been used by EPA in related situations.
When a method has been validated over a limited range of matrices, it is important that the determining characteristics of the matrices tested accompany the precision data. Statements such as those contained in many of the published methods, saying only that the methods are applicable to industrial wastewaters, are simply not enough. 4. Terms must be defined consistently and be based on overall precision. L. A. Currie (“Limits for Qualitative Detection and Quantitative Determination,” Analytical Chemistry, March 1968) emphasized the importance of a consistent set of definitions for expressing confidence in results obtained by the process of measurement. Standard definitions that embody either SOor St must be adopted for terms such as “not detected” and “limit of detection,” the latter especially since discharge limits of certain pollutants have been proposed in terms of multiples of the “limit of detection.” The definitions given in the accompanying box have been proposed by U W A G for use with compliance monitoring methods. These definitions employ St, rather than SO,in order to deal with the problem of two laboratories testing the same sample. They also employ the 99% confidence level
Conclusions The cost of properly validating an analytical method is small compared to the cost of errors in the monitoring data. There is enough disagreement on the costs and benefits of a given effluent limitation or water quality standard without having to contest the validity of the data upon which those costs and benefits are based.
4
James K. Rice is a consultant t o the research and deuelopment diuisions of seuera1 industrial f i r m s and to the Electric Power Research Institute. He also serues as a consultant to the Utility W a t e r Act Group in its dealing with EPA. Mr. Rice has 30 years of professional experience in the areas of treatment and control of water used f o r and discharged f r o m steam electric power plants, and in analytical methodology f o r measurement of water quality. H e is the former senior uicepresident of NUS Corp. and the former president of Cyrus Wm. Rice & Co.
EPA’s reaction ES&T asked EPA to comment on the four issues raised in the accompanying article. Dwight G. Ballinger, director (now retired) of the agency’s EnvironmentalMonitoring and Support Laboratory in Cincinnati, supplied the following statement of EPA’s positions: Issue 1. EPA conducts interlaboratory validation studies of an analytical method by preparing concentrated samples of an analyte of precisely known concentration. This analyte is sent to participating laboratoriesto add a specific amount of the concentrated solution to an actual waste sample selected by the participant. This procedure is used because: The concentration of the analyte added can be obtained from a single study. The established concentration of the study samples is not dependent on a single laboratory’s measurement using the method that is being validated. A wide variety of waste samples are used for validation instead of a single waste.
The concentrations can be established at levels appropriate to regulatory requirements. Random errors not representative of actual field operations, such as obtaining a high volume of sample, maintaining homogeneity of this sample, and sample splitting, are not introduced. This study design does not require the analyses to be conducted simultaneously by all participants and therefore results in a larger number of participants. Issue 2. There should not be confusion between validation of analytical methodology and establishment of an effluent limitation. The precision and accuracy of an analytical method are only part of the variability of any single measurement obtained to represent the concentration of a pollutant in an effluent. Single-operator, single-laboratory precision data, as represented by the standard deviation, should be smaller than the overall precision. This is due to the introduction of the additional variables of operators and laboratories.
It is imperative to establish the accuracy of the method during validation studies so that proper judgment may be made of the performance of the operators and laboratories participating in the study. EPA does not substitute single-operator, single-laboratory precision and accuracy data for interlaboratory data. All statements clearly indicate the source and limitations of data. Issue 3. EPA attempts to obtain data from the widest diversity of wastewater by conducting validation studies in the described manner. The applicability of a validated method to a single discharge at a particular time is established through a laboratory’s quality assurance procedures. Issue 4. The agency is currently reviewing published work and developing terminology that is scientifically defensible. The agency recognizes differences in laboratory capabilities and does not find it desirable to prevent a laboratory from reporting data of known quality based on its internal quality control program.
Volume 14, Number 12, December 1980
1457
