Environ. Sci. Technol. 1993, 27, 2247-2249
h Log-Normal Statistical Methodology Performance Steven W. Hlnton National Council of the Paper Industry for Air And Stream Improvement, Inc., Department of Civil Engineering, Tufts University, Medford, Massachusetts 02155 Introduction In the past, most environmental measurements were made on constituents that were expected to exist in substantial concentrations. With these measurements, the results typically are far removed from zero and have considerable precision compared to the magnitude of the chemical or physical state being measured. Today, many important environmental quality questions focus on chemicals that are expected either to be absent or to exist at very low concentrations. Under these conditions, it can be expected that a set of data on trace chemicals will include some observations for which the chemical analyst has reported that the analyte concentrations are “below” the limit of detection (LOD). For many of these measurements, the analytical process actually has produced a numerical value which has been censored by the chemist because the value is judged “unreliable” due to noncompliance with quality control or assurance criteria such as low signal-to-noise ratio or poor recovery of spiked compounds. In order to make informed decisions during regulation development or permit compliance monitoring, the regulated as well as regulating community needs guidance on (a) the available statistical approaches for analyzing data sets containing some values that are below the LOD and (b) the limitations of these statistical approaches. Previous reports (1)have described numerous approaches and Haas and Scheff (2) provide an almost complete summary of these. However, distinctly missing from peer review and evaluation is the A lognormal statistical (D-LOG) methodology, which is the most commonly used approach in regulatory related analyses of receiving water and effluent quality conditions due to its promotion and use by the US. EPA. As described below, the D-LOG methodology differs from the technique for the A distribution and from the ad hoc approach of replacing nondetected observations with a fixed value. Although the latter two methodologies have been evaluated to some degree (2,3),the performance of the former was apparently heretofore unknown. The purpose of this report is to present the results of a simulation study conducted to evaluate the bias and root mean square error (RMSE) properties of the mean estimator for the D-LOG methodology under analysis conditions typically encountered during water quality assessments. To allow the results to be easily related to the results of previous investigations, the simulation studies also evaluated two other methodologies that are commonly recommended for use in regulatory related statistical analyses (4, 5).
underlying assumptions, and worked examples for the MLE and RNOS methodologies have been completely described elsewhere (1,2).Particulars specificto this study include the use of the power series approximation (2) for the MLE auxiliary function and the use of Blom plotting positions (1, 8) in the RNOS technique. The A distribution (3) is a variant of the two-parameter log-normal model in which a proportion, 8, of the observations are 0, and the 1- 8 proportion of nonzero values follow the log-normal distribution with log-space parameters p and a2. The estimated mean of a A distribution with estimated parameters 6,y, and s2,is given by
M A = (1- 6) exp@ + 0 . 5 ~ ~ )
(1)
In the above, 6 = (n-k)/n, = (Cyi)/k, s2 = C(yi - yI2/(k - l),yi = ln(ri), and the summations are over the k fully quantifiable measurements of the n total observations. This model has been proposed for air quality assessments (3) where population values of 0 are plausible. For receiving water and effluent quality assessments, the US. EPA (6) has proposed using this model for censored data set analyses, but with the major difference that all censored values are assumed to be concentrated at D = LOD (instead of at zero) with the remaining observations distributed log-normally. With this assumption, the mean and variance estimators given by the US. EPA (6) are MD-LOG = 6D + (1- 6) exp@ + 0 . 5 ~ ~ ) vD-LOG
= (1- 6 ) exp(2y + s2) [exp(s2)- (1- 611
(2)
+
6(1- 6)D[D- 2 exp(y + 0.5s2)1 (3)
In the D-LOG methodology (6),a sample’sestimated mean is the weighted combination of (a) the mean estimated for the fully quantifiable measurements assuming they are log-normally distributed and (b) the mean assigned for the nondetected observations, assuming a point distribution occurs at the LOD (Le., nondetected observations are assigned the LOD value but not transformed and treated as fully quantifiable measurements). Published accounts of the US. EPA’s rationale for proposing the D-LOG methodology instead of recommending an established alternative technique with known bias and RMSE properties appears nonexistent, except for the statement “[it] often provides an appropriate and computationally convenient model for analyzingsuch data” (6).
Simulation S t u d y Approach
The three statistical methodologies tested were the D-LOG procedure as defined by US. EPA (6), the maximum likelihood estimator (MLE) originally described by Cohen (7), and the regression of normal order scores (RNOS) technique discussed by Blom (8). The equations,
Monte Carlo simulations programmed in FORTRAN and executed on an IBM-PC compatible computer were used to determine estimates of the average bias and RMSE of the three alternative statistical methodologies under five levels of censoringand four distributional assumptions. Simulations were conducted with censoring points corresponding to the 5, 20, 40, 60, and 80 percentile values
0 1993 American Chemical Soclety
Envlron. Sci. Technol., Vol. 27,No. I O , 1993 2247
Statistical Methods
0013-936X/93/0927-2247$04.00/0
e e 0
7 -
/\ //
+---e
(Y
_..
'
..........D. , .
...........................
/ ............. ..........
'
0
20
40
60
80
DETECTION LIMIT VALUE (PERCENTILE) Figure 1. Bias for sample size of 10 drawn from a log-normal MLE with CV = 1.0; 0 , RNOS with CV distribution with p = 2.0 (0, = 1.0; A,DLOG with CV = 1.0; U, MLE with CV = 0.3; e, RNOS with CV = 0.3; A, DLOG with CV = 0.3).
of the sampling distributions. For ideal test cases, lognormal distributions with a mean of 2.0 and coefficients of variation (CV) of 0.3 and 1.0 were used because these conditions encompass the variability typically found in many regulatory related water quality data analysis situations (6). To test the robustness of alternative estimators of the mean under realistic analysis conditions when the underlying distribution is unknown, two empirical distributions created from actual industrial effluent discharge monitoring records were used. Each record contains 102 fully quantifiable measurements and their histograms indicate substantial right skew as is typical of many environmental water quality data sets. For each simulated condition, 1000 sets of 10,15, or 20 randomly drawn values from a distribution of known mean and variance were created (9) with values less than an assigned censoring point being designated "nondetect". The means of the data sets were estimated using each methodology, and these results were used to determine the bias and RMSE associated with the means. Sampling sizes of less than 20 were used because many important data analysis situations involve fewer observations. For example, during the recent rulemaking for the organic chemical and plastics manufacturing industry (IO),national effluent guidelines were established using data sets with as few as three quantified measurements of seven total observations.
Results and Discussion For sample sizes of 10, the average bias and RMSE of the three estimators as a function of detection limit expressed in terms of percentiles are shown for log-normal distributions in Figures 1 and 2, respectively. All estimators are positively biased (i.e., they overestimate the mean) at all levels of censoring, and bias increases with increased censoring and population variability (Le., increased CV). Except for the CV = 1.0 condition and censoring at less than the distribution's 20th percentile, the D-LOG estimator is more biased than the MLE estimator. For the CV = 0.3 condition, the D-LOG estimator is more biased than either alternative, particularly when the censoring point is greater than the 40th percentile. The RMSE for the D-LOG and MLE estimators are similar for censoring at less than or equal to 2248
Envlron. Scl. Technol., Vol. 27, No. 10, 1993
I
100
20
'
l
40
'
l
60
'
...... l
'
80
l
100
DETECTION LIMIT VALUE (PERCENTILE) Flgure 2. RMSE for sample size of 10 drawn from a log-normal distribution with p = 2.0 (0, MLE with CV = 1.0; 0 , RNOS with CV = 1.0; A, DLOG with CV = 1.0; W, MLE with CV = 0.3; RNOS with CV = 0.3; A, DLOG with CV = 0.3).
+,
P 0
20
40
60
80
100
DETECTION LIMIT VALUE (PERCENTILE) Flgure 3. Bias for sample size of 10 drawn from an empirical 45-DCC dlstribution with p = 8.3 and u = 5.1 (0, MLE; 0 , RNOS; A,DLOG).
approximately the distribution's 40th percentile, above which the D-LOG RMSE becomes inferior. For the CV = 1.0 condition, the RNOS estimator has consistently greater RMSE compared to the other two estimators, but when population variability is less, its RMSE can become superior to that of the D-LOG estimator. The simulation experiments using numbers drawn from empirical distributions indicated the same general trends as those above for the log-normal distributions. For simulations employing a 4,5-dichlorocatechol (45-DCC) data distribution, the average bias of the D-LOG estimator was greater than the other estimators when the censoring point was greater than the 20th percentile (Figure 3) while RMSE for the D-LOG method was similar to the alternative estimators for censoring at or below the distribution's 40th percentile (Figure 4). In contrast to the lognormal simulations for the CV = 1.0 condition, RNOS method performance was often similar and sometimes superior to that of the D-LOG method. Similar results were obtained for simulations employing a distribution of 2,3,4,6-tetrachlorophenoldata with p = 2.1 and u = 1.1; these are omitted for brevity. Increasing sample size has the effect of reducing the bias and RMSE of all three estimators, but the reductions in bias are proportionately greater for the MLE and D-LOG estimators. Table I shows the simulation results for sample
P
when the censoring point was greater than the 20th percentile and sample size was greater than 15. Conclusions
I
W
DETECTION LIMIT VALUE (PERCENTILE) Flgure 4. RMSE for sample size of 10 drawn from an empirical 45-DCC dlstribution with = 8.3and u = 5.1 (0,MLE; 0 , RNOS; A, DLOG).
Table I. Bias and RMSE for Sample Size of 15 And 20 Drawn from Log-Normal Distributions
det limit percentile D-LOG
bias MLE
RMSE RNOS D-LOG MLE RNOS
Log-Normal with p = 2, CV = 1,and n = 15 5 20 40 60 80 5 20 40 60 80 5 20 40 60 80 5 20 40 60 80
0.0968 0.5011 0.5183 0.5394 0.1199 0.4851 0.5150 0.5525 0.1641 0.5022 0.5183 0.5804 0.2490 0.6696 0.5205 0.6540 0.5609 3.356 0.8675 1.426 Log-Normal with fi = 2, CV = 0.3, and n = 15 0.0153 0.0062 0.0208 0.1678 0.1708 0.1714 0,0527 0.0029 0.0275 0.1641 0.1701 0.1737 0.1421 0.0033 0.0389 0.1968 0.1762 0.1864 0.2969 0.0084 0.0645 0.3167 0.2002 0.2285 0.6176 0.1104 0.2115 0.6225 0.2553 0.4007 Log-Normal with p = 2, CV = 1, and n = 20 0.4668 0.0230 0.0590 0.4410 0.4505 0.0045 0.0178 0.0790 0.4247 0.4509 0.4780 0.0300 0.1700 0.0243 0.1083 0.4395 0.4529 0.4884 0.4875 0.0322 0.1765 0.6202 0.4629 0.5481 1.368 0.5012 57.44 0.1668 2.482 1.318 Log-Normal with p = 2, CV = 0.3, and n = 20 0.0067 -0.0018 0.0100 0.1494 0.1524 0.1523 0.0452 -0.0043 0.0140 0.1449 0.1523 0.1542 0.1367 -0.0035 0.0236 0.1817 0.1566 0.1632 0.2095 0.2930 -0.0055 0.0411 0.3093 0.1895 0.6060 0.0499 0.1310 0.6099 0.2555 0.3719 0.0350 0.0563 0.1932 0.5059 1.368
0.0547 0.0417 0.0490 0.0680 0.3035
sizes of 15 and 20 drawn from log-normal distributions. Except for the slightly negative bias of the MLE estimator, the relative performance of the MLE estimator compared to the D-LOG and RNOS estimators was similar to that for sample sizes of 10. However, in contrast to the results for the smaller sampling size (Figure 1)) the bias of the RNOS estimator was less than that of D-LOG estimator
Of the three statistical methods for calculating the mean of data sets containing observations below the analytical chemistry LOD, the MLE method (7) provided estimates of the mean with substantially less bias than those of the D-LOG method with approximately the same or less RMSE. For censoring within the 5-60 percentile range, D-LOG method bias was on average at least 2.5 times greater than MLE method bias. In situations where the CV was less than approximately 0.6, the RNOS method also provided estimates of the mean with less bias than the D-LOG method without sacrifice of RMSE performance. Acknowledgments
Richard M. Vogel and Jay P. Unwin provided helpful discussions and comments during manuscript preparation. Erik P. Drake developed the initial version of the FORTRAN simulation code. Literature Cited Berthouex, P. M. Estimating the Mean of Data Sets that Include Measurements Below the Limit of Detection; Technical Bulletin No. 621; National Council For Air and Stream Improvement New York, 1991. Haas, C. N.; Scheff, P. A. Environ. Science Technol. 1990, 24,912-919.
Owen, W. J.; DeRouen, T. A. Biometrics 1980,36,707-719. U.S. EPA. Statistical Analysis of Ground-Water Monitoring Data at RCRA Facilities, Addendum to Final Guidance, 7/92; U.S.EPA: Washington, DC, 1992. Mayer, F. L.; Krause, G. F.; Ellersieck, M. R.; Lee, G. Statistical Approach to Predicting Chronic Toxicity of Chemical to Fishes from Acute Toxicity Test Data; U.S. EPA ERL: Gulf Breeze, FL, 1992. U.S.EPA. Technical Support Document for Water Quality-Based Toxics Control, 9/91; U S . EPA Office of Water: Washington, DC, 1991. Cohen, A. C. Technometrics 1959,1, 217-237. Blom, G. Statistical Estimates and Transformed Beta Variables; John Wiley and Sons: New York, 1958. Hwang, J. H. Descriptions of Univariate Statistical Models For Use in Environmental Data Analyses and Means for Predicting Their Goodness-of-Fit;Technical Bulletin No. 530; National Council For Air and Stream Improvement: New York, 1987. U.S. EPA. Development Document For Effluent Limitations Guidelines and Standards for the Organic Chemicals,Plastics and Synthetic Fibers Point Source Category, 10/87; U.S. EPA Office of Water: Washington, DC, 1987. Received for review February 18, 1993. Reuised manuscript received May 25, 1993. Accepted June 3, 1993.
Environ. Sci. Technol., Vol. 27, No. 10, l Q Q 3 2249
