Environ. Sci. Technol. 2004, 38, 6824-6831
Balancing Measurement Uncertainty against Financial Benefits: Comparison of In Situ and Ex Situ Analysis of Contaminated Land P A U L D . T A Y L O R , * ,† MICHAEL H. RAMSEY,† AND PHILIP J. POTTS‡ Centre for Environmental Research, School of Life Sciences, University of Sussex, Falmer, Brighton, BN1 9QJ United Kingdom, and Department of Earth Science, The Open University, Walton Hall, Milton Keynes, MK7 6AA United Kingdom
The uncertainty of measurements taken for the purpose of characterizing contaminated land can subsequently cause decision errors, which can produce significant, financial consequences. Given the site-specific costs, such as those associated with the measurements or with site misclassification, the important question addressed is “Are the measurements of acceptable quality for that given objective or fit-for-purpose”? It is often considered by investigators that using a standard operating procedure (SOP) with an approved analytical method will give an acceptable level of uncertainty. This is despite evidence that sampling is usually the predominant source of uncertainty, not the chemical analyses, mainly as a result of the contaminant heterogeneity within sampling locations at a site. One in situ and one ex situ measurement technique were used to represent these two contrasting approaches to characterizing a site contaminated with lead in topsoil. The measurement uncertainty, from both sampling and analyses, was estimated for the two techniques, and its fitnessfor-purpose was assessed using the innovative optimized contaminated land investigation (OCLI) method. It is objectively demonstrated by the OCLI method that the in situ method (portable X-ray fluorescence) was three times more cost-effective than the ex situ (AAS) method at characterizing a contaminated site, despite generating higher uncertainty on individual measurements.
Introduction Sampling uncertainty has been recognized as an important issue in the characterization of contaminated land (1) because decision errors based on data of poor quality can lead to a pronounced negative effect on the cost and effectiveness of contaminated site cleanup. Measurement uncertainty has been defined by ISO (2) as “an estimate attached to a test result, which characterizes the range of values within which the true value is asserted to lie”. Therefore, this uncertainty is applied to each individual measurement and does not refer to the population of measured concentrations taken * Corresponding author phone: +44 (0) 1273 877688; fax: +44 (0) 1273 678433; e-mail:
[email protected]. † University of Sussex. ‡ The Open University. 6824
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 38, NO. 24, 2004
across the site during the investigation. Several methods are currently available for the estimation of measurement uncertainty applied to site characterization, and they vary in both complexity and practicality (3, 4). Measurement uncertainty can never be entirely eliminated. However, using a more precise analytical method can reduce the analytical variance. The sampling variance (s2samp) is generated mainly by the inherent short-range heterogeneity of the contaminants (also referred to as fundamental error, and grouping and segregation error, in the terminology (5) of Gy) and can generally be reduced by either increasing the sample mass or by taking composite samples at each sampling location (as s2samp ∝ 1/x mass of the sample) (5). Increasing the number of sampling locations provides an improved estimate of the mean and the variability of the population across the site, but it does not alter the uncertainty associated with each individual measurement. A reduction in measurement uncertainty therefore requires an increase in expenditure, and considering the sitespecific costs, an important question is “Are the measurements fit-for-purpose (FFP)”? Currently, FFP is often judged by the results from the quality control applied to the analytical technique only (6), when using a standard operating procedure (7), for example. This approach diminishes the importance of sampling variance (1), which is often the significant source of uncertainty by up to a factor of 20 or more as compared to the analytical variance (8) and can be significant for particularly heterogeneous sites (e.g., >(80% of the measured concentration, at 95% confidence) (7). A conceptual site model and the judgment of the acceptability of uncertainty, such as the USEPA’s Data Quality Objectives (7), are usually employed to design the characterization work-plan (i.e., the sampling design). However, this approach largely ignores the contrasting levels of shortrange contaminant heterogeneity and thus sampling uncertainty found at different sites. Existing FFP criteria (9) and current investigation guidance (10) fail to address the financial issues that are involved. Recently, there has been an increase in the number of analytical techniques capable of measuring contaminant concentrations in situ (11-13) (i.e., in the sample’s original location). The real-time measurements of in situ techniques allow adaptive sampling to be conducted as part of a flexible work-plan (6) and can have a marked reduction on the overall time and cost required for site characterization before cleanup (Figure 1). However, the higher measurement uncertainty often generated by in situ techniques usually results in a preference being given to more traditional, but timeconsuming, laboratory based (ex situ) analytical methods. Little attention has been given to the question as to whether the site information produced by in situ methods is less acceptable than an ex situ method in terms of both overall uncertainty and cost. A case study is presented here of an investigation of contaminated land using both an in situ (portable X-ray fluorescence spectrometry) and an ex situ (atomic absorption spectroscopy) technique. The Optimized Contaminated Land Investigation (OCLI) method (14) is applied for the objective assessment of whether the measurements generated by both in situ (PXRF) and ex situ (AAS) techniques are FFP given their uncertainty and the costs of both the measurements and the consequences of possible decision errors.
Experimental Procedures In this case study, both an in situ and an ex situ measurement technique were employed for an investigation of one 10.1021/es049739p CCC: $27.50
2004 American Chemical Society Published on Web 11/17/2004
FIGURE 2. Balanced design of sampling and analysis to be applied to a small proportion of sampling locations (e.g., 10%) for the estimation of measurement uncertainty.
FIGURE 1. Schematic illustration of the potential saving in the overall time taken for the investigation of potentially contaminated land by in situ PXRF (a few days) when compared against the more traditional approach of ex situ methods (several months). potentially contaminated site. The site was situated within a heavily vegetated subarea (10 000 m2) of Heathland in, West London, England (grid ref. latitude 51°27′30′′ N, longitude 0°22′56′′ W). The site is currently used as a nature reserve and is typically used by the public for recreational activities such as walking or picnics. Historical records suggest that the sub-area of the Heath surveyed was previously used as a military firing range (ca. 1920); thus, the topsoil might subsequently contain significant levels of Pb, which is associated with this activity (15). The process of characterizing a potentially contaminated site generally involves a comparison between the measured values of contamination, of either individual locations or the site mean, against some threshold value (e.g., soil guideline value (16) or action limit (7)). The primary aim of both particular site investigations here was to establish whether the site contained any individual areas of topsoil where the Pb concentration within the topsoil exceeded the U.K. action limit of 2000 µg g-1, upon which the need for remedial action would subsequently be based. This value of 2000 µg g-1, for a proposed use of land of parks and playing fields (i.e., its current land use) and which the decision support is based, is taken from the ICRCL trigger threshold values (17), which were current in the U.K. at the time of the investigation. These threshold values have subsequently been revised in the U.K. as soil guideline values (16). This revision does not, however, affect the general validity of the conclusions that are presented here. The sampling design employed at the site was the same for both measurement techniques and consisted of 36 sampling locations within a regular square grid with a 20 m spacing, which was set out using measuring tape and canes. The number of sample locations and the sample spacing was judged to be dense enough to give satisfactory spatial information regarding the Pb concentration within the topsoil. As with many site investigations, the area of the site, the finite time and financial resources available, limited the number of sampling locations (n ) 36) which are inadequate (n < 100) for the construction of a reliable variagram or applications of other similar geostatistical techniques. A nonjudgmental regular sampling pattern was selected because the likely spatial distribution of Pb contamination, such as the location of any Pb hotspots, was unknown. The use of a regular sampling pattern also conforms to the current guidance (10) for this type of investigation. This same 10 000 m2 sub-area of the Heath was investigated by both the ex situ and the in situ measurement techniques using the same
nominal sampling design. The location and specific characteristics of this particular site do not influence the validity of the conclusions from this study, however, because the methodology presented here can be applied to any site. The uncertainty associated with each individual measurement was estimated by implementing a balanced design (9) (Figure 2), which is derived from previous work in regional geochemical surveys (18, 19). As part of the balanced design, sample duplicates (Figure 2) (also known as co-located field duplicates (20)) were taken at randomly selected sampling locations (n ) 10 for the in situ method and n ) 12 for the ex situ method). The sample duplicates were taken at a distance of 2 m away from the original sample location in a pseudo-random direction to represent the measurement uncertainty that would arise in repeating this particular sampling pattern when using this surveying technology. The balanced design methodology, used to estimate measurement uncertainty arising from sampling, does not necessarily include all possible sources (e.g., sampling bias). The selection of 2 m for separation of the sample duplicate does, however, reflect the best estimate of surveying error (i.e., survey repeatability) in this particular survey. Previous studies (21) suggest that the precise distance ((50%) of duplicate sampling does not make a significant difference to the estimate of the sampling uncertainty. The variance of the measurements taken as part of the balanced design (Figure 2), which ultimately quantifies the uncertainty, is separated into three components by the application of robust analysis of variance (ANOVA) statistics. Robust ANOVA is preferred here over Classical ANOVA as it downweights outlying values often found in geochemical surveys, which is important as the application of ANOVA requires an approximately normal (Gaussian) population distribution (9). The three components of total variance separated by ANOVA are the analytical (s2anal), sampling (s2samp), and geochemical variance (s2geochem). The variance generated by the analytical procedure (s2anal) is therefore estimated by the variability between the measured concentrations of the analytical duplicates that are part of the balanced design (Figure 2). Similarly, the taking of sample duplicates represents the uncertainty in relocating the original sampling location and thus provides an estimate of the sampling uncertainty (s2samp). The degree of sampling uncertainty is therefore a function of the inherent short-range contaminant heterogeneity that is present at the site. The geochemical variance (s2geochem) represents the variability of contaminant concentration across the site (i.e., between sampling locations). The measurement variance (s2meas) is defined here as the sum of the sampling and analysis variance and hence
smeas ) (xs2meas) ) x(s2anal + s2samp)
(1)
Assuming a normal distribution of the measurements (sampling and analysis) taken as part of the balanced design, after any outlying values have been accommodated by the application of robust analysis of variance, and ignoring the systematic errors at this stage, the uncertainty of a measured VOL. 38, NO. 24, 2004 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
6825
concentration c can be expressed as a percentage (U%) (i.e., the relative expanded uncertainty) using the standard deviation of measurement relative to the concentration for 95% confidence (3) (i.e., with a coverage factor of 2) as
U% ) 200smeas/c
(2)
Measurement Technique 1: Ex Situ Chemical Analysis by Atomic Absorption Spectrometry. The ex situ method, by definition, involved the removal of topsoil samples from the site for chemical analysis by an off-site laboratory at a later time. Topsoil samples were removed by hand auger at each of the 36 sample locations with the additional 12 sample duplicates. The soil was removed to a depth of 150 mm with a diameter of approximately 30 mm after first removing any surface vegetation and other organic matter. This depth of sampling of 150 mm was chosen as a commonly accepted definition of topsoil associated with the rooting depth of many plants. It was also chosen because the upper part of the topsoil represents the most significant health risk to those using this subarea of the Heath. For example, in terms of human health risk assessment, Pb would most likely reach the receptor through a pathway of ingestion or inhalation. The topsoil samples were taken from the site, weighed, dried in a laboratory oven, and reweighed, giving a typical mass of 90 g per sample. The samples were disaggregated and sieved to remove the >2 mm fraction (22), and the 99% to the measurement variance in both cases. It is therefore most likely that the high levels of expanded uncertainty (83 and 102%) can be attributed mainly to the high level of short-range heterogeneity of lead within the topsoil at the site, caused primarily by the mode of deposition (military munitions). The large measurement uncertainty is generated therefore by the inherent heterogeneity of the contamination. The measurement uncertainty cannot be entirely removed but can be reduced in theory by either taking composite samples or a larger sample mass. The important question of whether the measurements are fit-for-purpose will be considered next. Lead Concentrations Measured by the In Situ and Ex Situ Techniques. In terms of the objectives of the site investigation, the measurements by both methods indicate that the topsoil is significantly contaminated with Pb (Table 2). The population mean values for the measurements of Pb concentration taken at each of the 36 sample locations across the site are 3004 µg g-1 for the ex situ technique and 2254 µg g-1 Pb for the in situ PXRF technique. Both methods also identified the same two separate hotspots where measurements of Pb concentration in topsoil exceeded the threshold value of 2000 µg g-1 (Figure 4a,b). Supplementary in situ measurements were taken following the completions of the original sampling design with the objective of delineating the two Pb hotspots located during the initial survey. This was possible because of the real-time nature of the measurements made by the in situ PXRF
FIGURE 4. Ex situ (AAS) laboratory based measurements (b) gave similar spatial results to the in situ PXRF technique (a) in that two subareas of the site indicated concentrations for Pb in topsoil above the threshold value of 2000 µg g-1 (marked with an ×). Supplementary in situ PXRF measurements were taken (a) after the initial sampling design was completed based upon the two Pb hotspots (Pb > 2000 µg g-1) that were discovered on-site by the real-time results produced by PXRF. technique. To achieve this, two regular sampling grids with a sample spacing of 10 m were constructed, while still onsite, around the two Pb hotpots. The supplementary in situ PXRF measurements provided greater spatial information in delineating the two hotspots (Figure 4a) than the ex situ technique (Figure 4b). The measurements of Pb concentration taken by both techniques at corresponding sampling locations were compared using regression analysis. The regression model indicates a strong relationship between the two sets of measurements (R ) 0.7 at 95% conf.). Analysis of the regression model indicates a bias of -42% ((5%) for the in situ PXRF measurements. This estimate of bias does not strictly adhere to the ISO definition, the difference between the expectation of a test result and the accepted reference value (26), as it is not directly traceable to an accepted reference value. However, the apparent bias between the measurements was possibly caused by water within the sample and by surface irregularity effects (25), which were not corrected for in the measurements taken by PXRF. This comparison of Pb concentrations is also limited due to the different sample depths that were analyzed. For example, the ex situ method measured a homogenized composition of the top 150 mm of topsoil, whereas the in situ PXRF method measured a depth interval of only ∼0.5 mm at a depth of ∼25 mm below the surface. The bias between the two techniques could, however, also be an indication that the Pb is less concentrated at the upper layer of topsoil (i.e., analyzed by PXRF), although previous similar estimates of this bias in PXRF measurements at other sites have been shown to be due primarily to soil moisture (24). It is important to note, however, that both measurement techniques provided a similar indication of the extent and spatial distribution of Pb contamination within the topsoil at this site (Figure 4), upon which the same broad conclusions will be drawn. This apparent bias between the two techniques therefore does not limit the broad interpretation of the in situ measurements. Assessment of Fitness for Purpose. One published FFP criterion (9) recommends that the relative contribution of measurement variance to total variance be less than 20% (e.g., s2meas < 20% s2total), with a supplementary criterion that the analytical variance should not exceed 4% of total variance. Both ex situ and in situ measurements do not significantly exceed either of these FFP criteria (Table 1). The contribution of measurement variance toward total variance (10.5%) by the ex situ AAS method is half that of the in situ method (20.5%) (Table 1). A one-tailed F-test indicates that the estimated measurement variance is statistically greater for the in situ (PXRF) than for the ex situ (AAS) VOL. 38, NO. 24, 2004 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
6827
TABLE 1. Estimation of Variances by Robust Analysis of Variance (ANOVA) Statistics Indicates that the Measurement Variance of the Ex Situ, Laboratory Based Method Has Less Measurement Variance (10.49% of Total Variance) as Compared to the In Situ PXRF (20.5%)a parameter
ex situ (AAS) method (n ) 48) (Pb µg g-1)
% total variance, ex-situ method
in situ (PXRF) method (n ) 20) (Pb µg g-))1
% total variance, in situ method
robust mean sgeochem smeas ssamp sanal s total U% at 95% confidenceb
748.79 907.63 310.66 310.33 13.79 959.31 83%
89.51 10.49 10.46 0.03 100
1044.95 1048.68 532.47 528.96 61.07 1176.12 102%
79.50 20.50 20.23 0.27 100
a This is reflected by the disparity between the relatively high estimates of expanded uncertainty of 83% for the ex situ method and 102% for the in situ PXRF. b U% ) 200 smeas/robust mean.
technique, which implies that the ex situ method is a more precise method, as already indicated using the relative uncertainty values. The important question of whether the ex situ technique is better overall in terms of the investigation objectives needs to also include the financial considerations. Financial Consequences of Site Misclassification. Contaminated sites tend to differ in terms of their complexity and costs. Therefore, the assessment of FFP needs to consider the various site-specific costs involved, such as the measurement expenditure, the financial value of the site, and the potential costs arising from the failure to correctly characterize the site. The consequential costs due to poor site characterization can be significant, with one example in the U.K. of £18 million (27) (U.S. ) ∼$30 million). The potential financial consequences can arise from either false negative (type 2 error) or false positive (type 1 error) classifications. The potential cost arising from a false negative case can occur when an area of land is erroneously classified as uncontaminated when it is really contaminated. The financial consequences of a false negative may arise, for example, from future litigation or a subsequent loss of corporate reputation. A different consequence cost could arise, for example, from unnecessary remediation due to a false positive misclassification where the measured concentration is above the threshold value but the true contaminant concentration is really below it. Assessment of Fitness for Purpose and Cost-Effectiveness by the Optimized Contaminated Land Investigation (OCLI) Method. An early approach at achieving cost-effective water treatment (28) and the optimal remediation of contaminated groundwater (29, 30) did not directly consider the uncertainty associated with each individual measurement. These approaches were specifically concerned with implementing the most effective technological strategy from a range of different scenarios from a range of different remediation scenarios. The approach applied here is the Optimized Contaminated Land Investigation (OCLI) method that can be used to assess whether the estimated level of uncertainty is FFP in terms of both measurement and potential financial consequences. This methodology is derived from one description of a general measurement optimization (31) which has been specifically applied to food analysis (32). A similar method, exemplified by the investigation of contaminated land, uses decision theory to determine the most appropriate analytical and sampling strategy based upon the cost of measurement and the end-user losses due to measurement error (33). A full description of the OCLI method can be found elsewhere (14) but is based upon the minimization of eq 4.
E(L) ) C[1 - Φ(|1|/smeas) ] + D/s2meas 6828
9
(4)
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 38, NO. 24, 2004
TABLE 2. Measurement Statistics for Pb Concentrations in Topsoil Taken by In Situ PXRF (2254 µg g-1) and an Ex Situ, Laboratory Based Method (3004 µg g-1)a statistic
ex situ (AAS) (n ) 36) (Pb µg g-1)
in situ (PXRF) (n ) 36) (Pb µg g-1)
mean 1 SD median maximum minimum
3004 6419 381 31 684 14
2254 4580 368 22 620 85
a Both indicate that the site has significant Pb contamination when compared with a threshold value of 2000 µg g-1.
TABLE 3. Approximate Costs Used as Input Variables in the Optimized Contaminated Land Investigation (OCLI) Method To Estimate Whether the In Situ and Ex Situ Measurement Methods Were Fit-for-Purpose Given the Potential Financial Consequences of Misclassificationa cost per consequence cost measurement (£) of misclassification (£) threshold Lsamp Lanal C (remed) C (legal) (µg g-1 Pb) ex situ (AAS) in situ (PXRF)
10 13
10
42 000 10 500
10 000 10 000
2000 2000
a The cost of sampling and analysis is combined for in situ analysis due to their high interdependence. b ‘remed’ ) cost of remediation.
where E(L) is expectation of loss; C is consequence costs (e.g., potential losses resulting from misclassification); Φ is the standard normal cumulative distribution function; 1 is the error limit (absolute value of the threshold limit T minus contaminant concentration c); s2meas is the measurement variance; and D is the combined cost per variance for sampling and analysis. Explained briefly, the OCLI method is a two-stage procedure that estimates the optimal level of measurement uncertainty for any given contaminated land survey. To estimate the optimal uncertainty value, the OCLI method considers the financial expenditure per variance apportioned to both the sampling and the analysis D, the uncertainty of the measurements taken during a site investigation, s2meas, and the consequential costs associated with measurement misclassification C. The input parameters for the OCLI method are given in Table 3 for the in situ and ex situ methods used in this example. The OCLI method is a site-specific decision-support tool, for which these parameters are selected for each new site. The costs estimated for this application are intended to be illustrative and are therefore not universally applicable. The sampling costs of the ex situ technique are based upon the expected cost of £350 per day for a site investigation
using a hand auger. The rounded value of £10 per sample for the sampling costs for the ex situ technique is derived from the total number of samples taken for ex situ analysis (n ) 36). The analytical costs for the ex situ technique (£10 per sample) are based on the expected laboratory costs for single inorganic chemical analysis of topsoil. This gives a total measurement cost of £20 per sample for the ex situ technique. The costs apportioned to sampling for the in situ technique reflects the time taken to complete the PXRF site survey and is estimated to cost £700 per day based upon the expected cost of equipment hire (£200 per day) and the time of the PXRF operator. The cost per PXRF measurement has therefore been calculated as £13 (£700/n ) 54 in situ samples). This cost of £13 per sample is considered to include all analytical costs, which include costs of instrument registration, operator training, and the periodic replacement of the radioactive sources, which are all difficult to quantify separately on a site-specific basis. Because the steps of sampling and analysis are almost inseparable for in situ techniques, the value of £13 represents the total measurement cost per sample of the in situ technique. An important decision that has to be made for the OCLI method is the particular contaminant concentration at which to optimize the system. For the false negative situation, an initial value of 1800 µg g-1 Pb (0.9 T) has been selected as being sufficiently far below the action level (2000 µg g-1 Pb) to make misclassification apparently unlikely. For the false positive situation, an initial value of 2200 µg g-1 Pb (1.1 T) was similarly investigated. The consequence cost for litigation that may arise from a false negative classification is estimated here as £10 000. This value is derived from a conservative estimate of total costs from a typical court case (27) (ca. £1 000 000) and the probability, estimated for this example as 0.01 (i.e., 1%) per single sampling location, of the contamination being subsequently detected. This calculation demonstrates that the consequence cost related to a false negative classification tends to have a low probability of detection but also to have high financial consequences. Although there are clearly difficulties in accurately estimating the costs, the scale and impact of the potential consequences make them worth considering. Rough estimates, even to the nearest factor of 10, are often adequate for a first approximate application of the OCLI method and can be revised and reapplied when updated estimates are available. For this example, the consequential costs used in the OCLI method are taken from the worst-case financial scenario of a false positive (i.e., unnecessary remediation). The potential cost of remediation for the ex situ technique was estimated as £42 000 per sampling location that was misclassified. Given the 20 m spacing of the sampling grid (Figure 4b), the maximum area that might be contaminated around one location (where the Pb concentration exceeds the threshold value of 2000 µg g-1) is a 40 m square (e.g., 1600 m2). Given the suspected source of contamination at this site and using a conservative remediation strategy, the soil contamination is estimated to be within a depth of 0.3 m, which generates a volume of soil of 480 m3. Assuming a topsoil density of 1.75 g cm-3 and the current remediation cost (27) for the digand-dump method in the U.K. as £50 ton-1, this gives a cost of £42 000 for each isolated sampling location. The reduced remediation costs (£10 500) given for the in situ technique is based upon the reduction in the mass of topsoil that requires remediation due to the closer grid spacing of 10 m (400 m2 × 0.3 m) surrounding the Pb contamination (top right and bottom left of Figure 4a). This value is derived from an approximate calculation but is adequate for the purposes of this feasibility study.
FIGURE 5. Optimized contaminated land investigation (OCLI) method indicates that the value of actual uncertainty for in situ PXRF (2) is higher than that for the ex situ method (b). In terms of the expectation of loss value at the actual uncertainty value, the loss from the ex situ, laboratory-based method (£10906), is much higher than the in situ PXRF method (£3724). The OCLI curve also indicates that the in situ method is more fit for purpose than the ex situ method over most of the uncertainty range.
Results and Discussion The optimal measurement uncertainty for the in situ and ex situ methods (88 and 68 µg g-1 Pb), estimated by the OCLI method (Figure 5 and Table 4), indicate that the actual uncertainties (532.5 and 310.6 µg g-1 Pb, respectively) are both too high (by a factor of approximately 5). Although both actual uncertainties are therefore suboptimal, the expectation of loss value for the ex situ method (£10 906) is more than three times greater than that of the in situ (PXRF) method (£3 724). The graphical curve provided by the OCLI method (Figure 5) provides a rapid, visual appreciation of this disparity. Optimization of the Ex Situ Technique. A more detailed inspection of the OCLI curves (Figure 5) shows that the expectation of loss at the actual uncertainty value for the ex situ (AAS) method (£10 906), is nearly 40 times greater than that at the optimal value (£292). The sampling variance would need to be improved by a factor of 4.7 to achieve the optimal uncertainty. This improvement would theoretically require a 22-fold composite sample (x22) or an increase in sample mass from 90 to 1890 g (i.e., 2 kg) (calculated (5) from ssamp ∝ 1/xmass). Given this increase in either the sampling mass or the number of sample increments, it is likely that the on-site sampling survey would require an extra 2 days to complete (3 days in total). Therefore, given the sampling cost of £350 per day, the cost per sample location is £29 ((£1050/36). The probable increase in the original cost of chemical analyses (£10) due to the additional sample preparation that would be required has been estimated as an extra £2 per sample. This gives a total cost of £41 per measurement to achieve the optimal uncertainty value. The OCLI method could be used to provide a more costeffective approach (i.e., optimal level of uncertainty) for surveys that are subsequently undertaken at a site. For example, supplementary sampling may have been undertaken at the site using the ex situ method, with the aim of further delineating two Pb hotspots at the site (e.g., as employed by the in situ PXRF method). Unlike the in situ PXRF method, however, where supplementary samples can be taken while still on-site, it is likely that the time that elapsed between the initial and supplementary ex situ survey would be at least 2-3 weeks (assuming a routine 10 day laboratory turnaround time for the chemical analyses, the interpretation of the measurements, and the planning of the supplementary survey). It is likely that 2 days would be required to set up the supplementary sampling designs (2 × 18 sampling VOL. 38, NO. 24, 2004 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
6829
TABLE 4. OCLI Method Indicates that To Achieve the Optimal Uncertainty Value, and Therefore the Optimal Expectation of Financial Loss, an Increase in Measurement Costs Is Requireda uncertainty (µg g-1)
costs (£)
protocol
sanal
ssamp
smeas
Lanal
Lsamp
exp loss
estimated ex situ (AAS) optimal ex situ (AAS) change (%) estimated in situ (PXRF) optimal in situ (PXRF) change (%)
13.8 12.7 -8
310.3 66.8 -78.5
310.6 68.0 -78 532.5 88.0 -84
10 12 20 13 91.0 600
10 29 190
10906 292 -97 3724 597 -84
a £20-£41 per sample for the ex situ method and £13-£91 per sample for the in situ PXRF method. Achieving the optimal uncertainty reduces the expectation of loss value by £10 614 for the ex situ method and £3127 for the in situ PXRF method. b ‘exp loss’ ) expectation of financial loss.
locations, each with 22 sample increments), which gives a sampling cost of £39 per location (£700/18) or £51 per measurement (£12 per chemical analysis). This value does not take into account the possible financial cost associated with a delay in site redevelopment, which might be considerable for some sites. The reduction in potential losses from unnecessary remediation, however, as a result of a reduction of the mass of land removed (£42 000 to £10 500 per sample location), would seem to make supplementary sampling a prudent action in this case. Optimization of the In Situ PXRF Technique. For the in situ (PXRF) technique, the expectation of loss value at the actual uncertainty value (£3724) (Table 4) is approximately 6 times greater than the expectation of loss at the optimal uncertainty (£597). Achieving the optimal uncertainty for the in situ PXRF technique, and thus the optimal expectation of loss value, requires a 5-fold reduction in measurement uncertainty (smeas ) 532-109 µg g-1, i.e., a ratio of 4.89). An increase in the count time of the PXRF measurement would, however, only serve to reduce the instrumental detection limit, and the main component of the measurement uncertainty arises here from the contaminant heterogeneity. The most appropriate method to achieve the optimal value would therefore involve taking composite measurements at each sample location. Theoretically, a composite of 24 measurements at each sample location would be required to increase the sampling precision by the required factor of 4.89 (x24). This is similar to the required improvement for the ex situ method and can be acquired using similar means. Assuming that each in situ PXRF measurement had a count time of 140 s as before, each sampling location would require 60 min to complete (24 × 140 s) or 43 h in total to complete the site survey (n ) 36 + 18 supplementary + 10 sample duplicates). With this increase in sampling time, it is conceivable that 7 days would have been required to complete the optimal in situ PXRF site (assuming a working day of 8.5 h). Given the estimated daily measurement cost of £700 per sample measurement for the in situ PXRF method, the cost per sample for 24 increment composite measurements would cost approximately £91 per sample ((£700 × 7)/54). The shallow gradient of the OCLI curve from the optimal to the actual uncertainty value (Figure 5) demonstrates that a relatively large reduction in uncertainty, and increase in measurement expenditure (£700 to £4900), is required to achieve the optimal uncertainty value. The relatively small reduction in the potential expectation of loss (from £3725 to £597) indicates that the cost and time required to achieve the fully optimized level of uncertainty is therefore probably not warranted. General Discussion. A comparison of the U-shaped OCLI curves (Figure 5) for the in situ and the ex situ techniques illustrates that the actual uncertainty value for the in situ technique has a much lower expectation of loss. The OCLI method indicates that although the in situ technique has higher actual uncertainty, it is more FFP than the ex situ 6830
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 38, NO. 24, 2004
method over most of the range of uncertainty. One explanation for this advantage is because of the 4 times smaller mass of topsoil that might be unnecessarily remediated due to the supplementary samples taken by the in situ technique (480 m3 for the ex situ method and 120 m3 in situ). The ex situ AAS technique is a more precise method of measurement even when sampling uncertainty is included (U ) 83% as compared to 102%). The increase in sampling standard deviation is expected given the smaller sample mass that is analyzed by the PXRF (∼1 g cf ∼100 g) since ssamp ∝ 1/xmass. The extent and spatial distribution of Pb contamination provided by both techniques, however, provides the investigator with a remarkably similar characterization of the site (Figure 4). The higher uncertainty generated by the in situ PXRF can be tolerated because of its ability to provide rapid, real-time measurements of contamination that allows for adaptive sampling to be implemented while still onsite. It has been demonstrated here using the innovative OCLI method that the in situ PXRF technique provided a significant reduction in the overall expectation of loss as compared to the more traditional ex situ approach over a wide range of uncertainty. One potential limitation of the in situ PXRF technique is that it may produce misleading measurements if the contamination is vertically stratified within the soil. A previous published study (24) using in situ PXRF tested this hypothesis by measuring along a vertical soil core that was removed following a measurement taken using the PXRF methodology. The comparison between these measurements showed little systematic spatial trend with depth of core and that surface analysis by PXRF gave results that were not biased due to vertical variation in Pb concentration in the soil. These findings may not be the case at this or other sites and sampling at a variety of depths by PXRF may be required to establish whether there is a change of concentration with depth. For the study presented here, however, the overall agreement between the concentrations taken by both techniques suggests that the contamination does not change substantially with depth. Additional developments to the OCLI method are required if it is to achieve its full potential. Currently, the OCLI method does not directly include the financial savings achieved by the reduction in the time required for the overall site investigation using the in situ PXRF method (Figure 1). These can arise from the delays in land redevelopment caused by waiting for laboratory analysis to be completed and report and also any further delays incurred by the need for supplementary sampling and analysis. Further applications of the OCLI method might be used to consider a range of different consequence costs to reflect different remediation strategies and techniques. Contaminated land often contains a variety of contaminants; therefore, the OCLI should ideally be adapted to consider a multi-element optimization. The OCLI method should also be developed to interpret the measurements for
the whole site collectively (spatial OCLI) rather than the current method that only considers the uncertainty of individual measurements. Future work will also focus upon the choice of contaminant concentration, which should reflect the frequency distribution of the measurements, as the number of measurements, and their proximity to the threshold limit, will also both affect the likelihood of misclassification and thus the expectation of loss. While the input costs employed here for the OCLI method might not accurately represent the true cost values experienced by all commercial environmental investigators, the method itself has been shown to be feasible and useful. It is clear that the use of actual costs paid by commercial consultancies in practical applications will provide realistic input values for the OCLI method. Indeed, it is possible that the OCLI method might be employed prior to the investigation, rather than in the retrospective manner presented here, when investigating a series of sites (e.g., gas works) that might hold similar levels of contaminant concentration and heterogeneity. The values of input parameters from previous investigations would in that case provide useful initial default values for the application of the OCLI method. This study highlights the flaw in the common perception that the quality, or FFP, of the measurements taken as part of a contaminated land investigation should be based purely upon the precision of the analytical technique. It is the inherent short-range heterogeneity of the contaminant concentration (i.e., within the sampling location) that generates the highest proportion of measurement uncertainty (i.e., that from the sampling phase). The quantitative evidence given by the OCLI method demonstrates that despite generating higher uncertainty on each measurement, it can be more cost-effective to employ an in situ measurement technique, such as PXRF, rather than the more traditional, but time-consuming, ex situ approach.
Literature Cited (1) Crumbling, D.; Groenjes, C.; Lesnik, B.; Lynch, K.; Shockley, J.; Van, E. E. J.; Howe, R.; Keith, L.; McKenna, J. Managing uncertainty in environmental decisions. Environ. Sci. Technol. 2001, 35 (19), 404-409. (2) International Standards Organization. Implementation of ISO 3534-1: StatisticssVocabulary and symbolssPart 1: Probability and general statistical terms; ISO: Geneva, 1993. (3) Ramsey, M. H.; Argyraki, A. Estimation of measurement uncertainty from field sampling: the implications for the classification of contaminated land. Sci. Total Environ. 1997, 198, 243-257. (4) De Zorzi, P.; Belli, M.; Barbizzi, S.; Menegon, S.; Deluisa, A. A practical approach for assessment of sampling uncertainty. Acc. Qual. Assur. 2001 7, 182-188. (5) Gy, P. Sampling of particulate materialsstheory and practice; Elsevier: Amsterdam, 1979. (6) Crumbling, D. Applying the concept of effective data to environmental analyses for contaminated sites. Current perspectives in site remediation and monitoring; United States Environmental Protection Agency (USEPA); Washington, DC, 2001; Report EPA/ 542-R-01-013. (7) United States Environmental Protection Agency (USEPA). Guidance for the data quality objectives process; Washington, DC, 2000; Report EPA/600/R-96/055. (8) Crumbling, D.; Griffith, J.; Powell, D. Improving decision quality: making the case for adopting next generation site characterization practices. J. Environ. Cleanup Costs, Technol., Techniques 2003, 13 (2), 91-111. (9) Ramsey, M, H.; Thompson, M.; Hale, M. Objective evaluation of precision requirements for geochemical analysis using robust analysis of variance. J. Geochem. Exp. 1992, 44, 23-26. (10) British Standards Institution (BSI). Investigation of potentially contaminated sitesscode of practice; British Standard 10175, 2001.
(11) United States Environmental Protection Agency (USEPA). Field analytical and site characterization technologies summary of applications; Washington, DC, 1997; pp 1-40; Report EPA-542R-97-011. (12) Potts, P. J.; Ramsey, M. H. Use of portable X-ray fluorescence in the characterization of arsenic contamination associated with industrial buildings at a heritage arsenic works site near Redruth, Cornwall. J. Environ. Mon. 2002, 4, 1-9. (13) Mosier-Boss, P. A.; Lieberman, S. H.; Theriault, G. A. Field demonstrations of a direct push FO-LIBS metal sensor. Environ. Sci. Technol. 2002, 36 (18), 3968-3976. (14) Ramsey, M. H.; Taylor, P. D.; Lee, J. Optimized contaminated land investigation at minimum overall cost to achieve fitnessfor-purpose. J. Environ. Monit. 2002, 4, 5, 809-814. (15) Landsberger, S.; Iskander, F.; Basunia, S.; Barnes, D.; Kaminski, M. Lead and copper contamination of soil from industrial activities and firing ranges. Biol. Trace Element Res. 1999, 7172, 387-396. (16) Department of the Environment, Food and Rural Affairs (DEFRA) and the United Kingdom Environment Agency. Swindon, 2002; SGV Series 1-10. (17) Department of the Environment: Interdepartmental Committee on the Redevelopment of Contaminated Land (ICRCL); London, 1987. (18) Miesch, A. T. Effects of sampling and analytical error in geochemical prospecting. In Computers in the Mineral Industry (Part 1); Parks, G. A., Ed.; Stanford University, Geological Science: Stanford, 1964. (19) Garrett, R. G. The determination of sampling and analytical errors in exploration goechemistry. Econ. Geol. 1969, 64, 568569. (20) United States Environmental Protection Agency (USEPA). A rationale for the assessment of errors in the sampling of soils (ASSESS); Las Vegas, 1990; Report 1-65, EPA/600/4-90/013. (21) Taylor, P. D. Uncertainty of geochemical measurements of contaminated land: causes, estimation, and cost-based optimization. Ph.D. Thesis (unpublished); The University of Sussex, 2003. (22) Ministry of Agriculture, Fisheries and Food (MAFF). The analysis of agricultural materials; London, 1986; ISBN 0112427626. (23) Thompson, M.; Walsh, J. N. A handbook of inductively coupled plasma spectroscopy; Blackie and Son Limited: Glasgow, 1983. (24) Argyraki, A.; Ramsey, M. H.; Potts, P. J. Evaluation of portable X-ray fluorescence instrumentation for in situ measurements of lead on contaminated land. Analyst 1997, 122, 743-749. (25) Potts, P. J. A handbook of silicate rock analysis; Blackie Academic and Professional: London, 1987. (26) International Standards Organization (ISO). Statistics, vocabulary and symbols. Part 1. Probability and general statistical terms; Geneva, 1993. (27) Finnamore, J.; Denton, B.; Nathanail, C. Construction Industry Research and Information Association. Land contaminations management of financial risk; London, 2000. (28) Egerton, A. J. Achieving reliable and cost-effective water treatment. Water Sci. Technol. 1996, 33 (2), 143-149. (29) Lee, S. I.; Kitanidis, P. K. Optimal estimation and scheduling in aquifer remediation with incomplete information. Water Resour. Res. 1991, 27 (9), 2203-2217. (30) James, B. R.; Freeze, R. A. The worth of data in predicting aquitard continuity in hydrological design. Water Resour. Res. 1993, 29 (7), 2049-2065. (31) Thompson, M.; Fearn, T. What exactly is fitness-for-purpose in analytical measurement? Analyst 1996, 121, 275-278. (32) Ramsey, M. H.; Lyn, J.; Wood, R. Optimized uncertainty at minimal overall cost to achieve fitness-for-purpose in food analysis. Analyst 2001, 126, 1777-1783. (33) Fearn, T.; Fisher, S.; Thompson, M.; Ellison, S. A decision theory approach to fitness for purpose in analytical measurement. Analyst 2002, 127 (6), 818-824.
Received for review February 18, 2004. Revised manuscript received August 17, 2004. Accepted August 18, 2004. ES049739P
VOL. 38, NO. 24, 2004 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
6831
