Evaluation of Performance Characteristics of ... - ACS Publications

Aug 30, 2008 - Chemical Science and Technology Laboratory, National Institute of Standards and ... The OAWG evaluation and benchmarking is typically a...
1 downloads 0 Views 1MB Size
Download PDF
Anal. Chem. 2008, 80, 7327–7335

Evaluation of Performance Characteristics of Multistep Analytical Methods from Collaborative Study of Linked Samples David L. Duewer,* Michele M. Schantz, and Reenie M. Parris Chemical Science and Technology Laboratory, National Institute of Standards and Technology (NIST), Gaithersburg, Maryland 20899-8390 Franz Ulberth European Commission, Joint Research Centre Institute for Reference Materials and Measurements (IRMM) Retieseweg 111, 2440 Geel, Belgium Following unexpectedly variable results from an international comparison study of the determination of selected polychlorinated biphenyl (PCB) congeners in shellfish tissue, a group of national metrology institutes collaboratively explored the analytical characteristics of their measurement systems using a designed study with four sample materials. This “Uncertainty Suite” consisted of a 10-congener mixture of PCBs in relatively nonvolatile isooctane, a 5-congener mixture in relatively volatile methylene chloride, a methylene chloride extract of freezedried mussel (Mytilus edulis) tissue, and the (homogenized) mussel tissue itself. These related-but-different samples presented the participants’ measurement processes with a linked series of analytical challenges. Data evaluation tools were developed to combine and visualize measurement results for the different congeners of interest for each material and, exploiting the linkages among the samples, to help identify causes for observed changes in performance. In addition to characterizing individual measurement processes, (1) the limiting sources of measurement uncertainty were found to be chromatographic separation and signal quantification in a natural matrix, (2) the achievable among-participant total measurement uncertainty for PCB calibration solutions is ∼1.9% over the mass fraction range from 40 to 500 ng/g, and (3) the achievable among-participant measurement precision for the determination of PCB congeners in mussel tissue at levels above 0.5 ng/g mass fraction is ∼5.4%. The primary focus of the Organic Analysis Working Group (OAWG), Consultative Committee for Amount of Substances Metrology in Chemistry, is benchmarking the higher order organic chemical measurement capabilities of the national metrology institute and international organization members of the Comite´ * To whom correspondence should be addressed. Tel: (301)-975-3935. Fax: (301)-926-8671. E-mail: [email protected]. 10.1021/ac8009966 CCC: $40.75  2008 American Chemical Society Published on Web 08/30/2008

International des Poids et Mesures (CIPM).1 As the CIPM members often provide reference materials and calibration services with minimal measurement uncertainties and shortest metrological traceability chains to the International System of Units (SI), the benchmark results may limit the capabilities that can be claimed by producers of secondary reference materials and, ultimately, by routine testing laboratories.2 The OAWG evaluation and benchmarking is typically accomplished through collaborative studies involving well-defined organic molecular entities and amount of substance measurements that are traceable to the SI. While few participants in these studies make routine measurements, the OAWG member organizations are all experienced in the selection, development, and validation of organic analysis methods needed for particular determinations. The measurement performance assessed through OAWG studies provide evidence to support international recognition of the validity of the organic chemical measurement services provided by the member organizations. These services include the development and specification of Reference Measurement Procedures and the production of Certified Reference Materials (CRMs).3 In 2003, seven members of the OAWG participated in a preliminary or “pilot” study of the determination of selected polychlorinated biphenyl (PCB) congeners in mussel (Mytilus edulis) tissue. As well as being a human food source, this common shellfish is a well-established sentinel species for evaluating chemical and biological contaminant trends in coastal waters.4 PCBs have been widely used as industrial fluids, flame retardants, diluents, hydraulic fluids, and dielectric fluids for capacitors and transformers. PCBs consist of 209 possible congeners depending on the substitution of the chlorine atoms around the biphenyl moiety; as a chemical class, they are environmentally stable and tend to bioaccumulate. Congener-specific PCB methods are (1) Kaarls, R. Accred. Qual. Assur. 2006, 11, 162–171. (2) Eurachem/CITAC, Traceability in Chemical Measurement, 2003. http:// www.eurachem.org, accessed 3 Mar 2007. (3) BIPM, Calibration and Measurement Capabilities website: http://kcdb. bipm.org/AppendixC/, accessed 3 Mar 2007. (4) NOAA CCMA National Status & Trends Program website: http://ccma. nos.noaa.gov/stressors/pollution/nsandt/, “Mussel Watch Program” link, accessed 3 Mar 2007.

Analytical Chemistry, Vol. 80, No. 19, October 1, 2008

7327

typically based on gas chromatography with electron capture detection or mass spectrometric detection. Even using robust summary estimators,5 the pilot study indicated that the among-participant percent relative standard deviation (RSD) for the PCBs ranged from about 8% at 100 ng/g mass fraction to 45% at 2 ng/g. Given the estimated material homogeneity, nominal similarity of the analytical procedures employed, and measurement uncertainties reported by each participant, these RSDs were considerably larger than the 5% expected from earlier studies of multicomponent PCB solvent solutions. After extensive discussion of possible sources of variability in the rather complex analytical processes required for measuring PCB contamination at endogenous levels, the OAWG decided to further investigate the sources of PCB measurement variability using a set of samples designed to enable assessment of each participant’s performance at several stages of the measurement process. This report presents the design of the suite of samples utilized in the study, the tools developed and used in the evaluation of the reported results, and the analytical conclusions. Properly implemented, measurement systems for PCB contamination in mussel tissue can achieve among-participant RSD of less than 6% for mass fraction levels above ∼0.5 ng/g. EXPERIMENTAL SECTION Target Analytes. Approximately 150 of the possible PCB congeners have been reported in the environment. The OAWG selected five of these as being representative of PCB measurement challenges, including potential interferences and spanning the volatility and concentration ranges typical of environmental samples. PCB 28 (2,4,4′-trichlorobiphenyl) is volatile and may coelute with PCB 31 (2,4′,5-trichlorobiphenyl). PCB 101 (2,2′,4,5,5′pentachlorobiphenyl) may coelute with a minor congener, PCB 90 (2,2′,3,4′,5-pentachlorobiphenyl). PCB 105 (2,3,3′,4,4′-pentachlorobiphenyl) is often found in lower concentrations and may change elution order with PCB 132 (2,2′,3,3′,4,6′-hexachlorobiphenyl) depending on the analytical conditions. PCB 153 (2,2′,4,4′,5,5′hexachlorobiphenyl) is often one of the most abundant congeners and may coelute with PCB 132. PCB 170 (2,2′,3,3′,4,4′,5-heptachlorobiphenyl) is one of the less volatile congeners, is typically found at lower concentrations, and may coelute with PCB 190 (2,3,3′,4,4′,5,6heptachlorobiphenyl). Analytical Methods. The determination of specific PCB congeners in a complex natural matrix typically involves extraction, concentration, separation, selective detection, and calibration. While all participants in this study used gas chromatographyisotope dilution-mass spectrometry (GC/ID-MS) analysis techniques, each participant employed somewhat different methods of extraction, separation columns and conditions, extract cleanup (identification and quantification of the target congeners from coextractants), and choice of primary calibration materials. Participants used the same instruments and calibration solutions for all of the materials they analyzed. Samples. The four materials used in this study, collectively referred to as the “Uncertainty Suite”, were designed collaboratively by OAWG members. In order of anticipated analytical complexity, the materials were as follows: a gravimetrically (5) Analytical Methods Committee. Analyst 1989, 114, 1693-1697.

7328

Analytical Chemistry, Vol. 80, No. 19, October 1, 2008

Table 1. Analytical Challenges samples analytical challenge

iOct

CH2Cl2

Extract

Tissue

separation and quantification solvent handling extract cleanup extraction

modest some none none

some large none none

large large large none

large large large large

formulated mixture of PCBs in isooctane (2,2,4-trimethylpentane) here termed “iOct”, a gravimetrically formulated mixture of PCBs of assessed purity in methylene chloride (dichloromethane) here termed “CH2Cl2”, an extract of freeze-dried mussel tissue in methylene chloride here termed “Extract”, and the freeze-dried mussel tissue itself here termed “Tissue”. The analytical challenges presented by each of these samples are summarized in Table 1. The four materials were prepared at NIST during Summer 2004. The materials were distributed in December 2004. All chemical analyses were completed by April 2005. iOct. This material was designed to enable validation of each participant’s ability to handle and prepare calibration solutions with high accuracy and to analyze PCB congeners in a relatively lowvolatility solvent. iOct was an isooctane solution of four of the five target congeners (PCBs 28, 101, 105, and 153) and six nontarget congeners: PCB 52 (2,2′,5,5′-tetrachlorobiphenyl), 66 (2,3′,4,4′tetrachlorobiphenyl), 118 (2,3′,4,4′,5-pentachlorobiphenyl), 138 (2,2′,3,4,4′,5′-hexachlorobiphenyl), 180 (2,2′,3,4,4′,5,5′-heptachlorobiphenyl), and 196 (2,2′,3,3′,5,5′,6,6′-octachlorobiphenyl). The nontarget PCBs in this material were selected to avoid known separation challenges with the target congeners. The solution was gravimetrically formulated; all concentrations were adjusted for assessed purity of the component materials. The concentrations of the target congeners included in this material ranged from 90 to 360 ng/g. Each solution was dispensed in 2-mL ampules with ∼1.2 mL of solution per ampule. CH2Cl2. This material was included so that any uncertainty component due to solvent volatility could be assessed. CH2Cl2 was a methylene chloride solution containing just the five target PCBs. To help ensure that solvent volatility was the major analytical challenge, the concentrations of the congeners were all higher than those in the iOct material, ranging from 36 to 520 ng/g. As with iOct, the CH2Cl2 material was gravimetrically formulated with adjustment for the assessed purity of the component materials. The solution was dispensed in 2-mL ampules with ∼1.2 mL of solution per ampule. Extract. This material was designed to assess participants’ chromatographic separation and peak integration capabilities given the analytical interferences typical of complex samples without including the extraction efficiency component. Extract was a concentrated methylene chloride extract of the mussel tissue described below. Eleven separate 20-h Soxhlet extractions, each of between 10 and 15 g of freeze-dried mussel tissue in 250 mL of methylene chloride, were combined and evaporated to a volume approximately equal to the total number of grams of mussel tissue extracted. The extract was dispensed in 2-mL ampules with ∼1.2 mL of solution per ampule. Tissue. This material was intended, in combination with Extract, to assess extraction and extract cleanup issues. Tissue was freezedried tissue prepared from mussels collected near an urban area

of the United States. The mussels in this material were harvested from a locale different from that used in the 2004 study. The tissue was frozen, cryogenically ground, mixed, freeze-dried, sieved (60 mesh; 6% rather than discordance with the consensus values (panel L3 in Figure S-3, SI). Extract Measurement Performance. Figure S-5 (SI) displays all of the measurements, the empirical PDF, and the N(MMmedian, sMMiqr) reference distribution for the five target PCBs in the Extract material. Figure S-6 (SI) presents the dispersion and location estimates and the proportional total dispersion function σˆ Extract(x) ) 0.053x. Figure S-7 (SI) displays the concordance and apparent precision analyses for all participants; Figure 6 summarizes these characteristics. The results reported by the majority of participants agree well; however, those reported by participants L1 and L9 are discordant with the majority and with each other.

Figure 6. Target plot summary of extract concordance and apparent precision. The plot format is identical to that of Figure 4. Here, the semicircular lines denote 1, 2, and 3 σˆ Extract(x) ) 0.053x units (5.3, 10.6, and 16.5%).

Two of the four quantitative values reported by participant L1 are considerably larger than the consensus estimate. Participant L9 did not quantify three of the target PCBs and reported much lower values for the other two than did the majority. While the chromatographic components of the measurement systems used by these participants were adequate for simple PCB solutions, they are not adequate when challenged with rather lower measurand levels in the presence of a complex background. While well within the 2 σˆ Extract cutoff, participant L3’s measurements are again less comparable to the consensus values than are those of the majority: In contrast with the pessimistic uncertainties of the CH2Cl2 measurements, the apparent imprecision is dominated by the discordance of PCB 170 target (panel L3 in Figure S-7, SI). This overestimation of the lowest mass fraction measurand suggests that the chromatographic components of this participant’s measurement system are not fully adequate for lower-concentration PCBs in a complex matrix. Tissue Measurement Performance. Figure S-8 (SI) displays all of the measurements, the empirical PDF, and the N(MMmedian, sMMiqr) reference distribution for the five target PCBs in the Tissue material. Figure S-9 (SI) presents the dispersion and location estimates and the proportional total dispersion function, σˆ Tissue(x) ) 0.068x, for the Tissue measurements. Figure S-10 (SI) displays the concordance and apparent precision analyses for all participants; Figure 7 summarizes these characteristics. The performance characteristics for most participants, including L1, L9, and L3, are little different from those in the Extract material. The greatest difference is for participant L2; while the apparent precision values for this participant are less than one total dispersion unit for both materials, the Tissue measurements are ∼1 σˆ Tissue unit (∼7%) less than consensus while the Extract measurements are ∼0.4 σˆ Extract (∼3%) greater than consensus. Given that the levels for all measurands in Tissue are somewhat higher than in the Extract material, the increase in σˆ Tissue over σˆ Extract suggests that the extraction components of the measurement process do impact measurement performance. The consistent relative discordance of participant L2’s measurements suggests a proportional underextraction or loss of measurand (panel L2 in Figure S-10, SI).

Figure 7. Target plot summary of tissue concordance and apparent precision. The plot format is identical to that of Figure 4. Here, the semicircular lines denote 1, 2, and 3 σˆ Tissue(x) ) 0.068x units (6.8, 13.6, and 20.4%).

Figure 8. Tissue/Extract ratios for the five target PCB congeners, The dot and bars represent the MMmedian and sMMiqr location and dispersion estimates of the Tissue/Extract ratios for the five target PCBs, displayed as a function of estimated mass fraction of the targets in the Tissue material. The plot format is otherwise similar to that of Figure 1.

Tissue/Extract Ratio. While Extract was prepared to provide somewhat smaller mass fraction levels than those expected for the participant’s own extracts of the Tissue material, the ratio of results between the two materials should be constant. For each participant, for each of the target measurands, this ratio and its uncertainty are given by

Ratioij )

(

U95(Ratioij) ) Ratioij

xTissue,ij ; xExtract,ij

)(

)

U95(xTissue,ij) U95(xTissue,ij) + . xTissue,ij xExtract,ij

Figure S-11 (SI) displays all of the Tissue/Extract ratios, the empirical PDF, and the N(MMmedian,sMMiqr) reference distribution for the five target PCBs. Figure 8 displays the MMmedian and sMMiqr estimates of the ratios for the five targets, again in dot-and-bar format but with the data spacing proportional to the logarithm of the estimated mass fraction in Tissue. From the MMmedian and sMMiqr of the PDF, the Tissue/Extract proportionality constant and its approximate U95 uncertainty is 1.40 ± 0.08. There appears to be slight but regular decrease in ratio with Analytical Chemistry, Vol. 80, No. 19, October 1, 2008

7333

Figure 9. Measurement performance characteristics for the Tissue/ Extract ratios. The plot format is identical to that of Figure 4, with the exception that the ratios for the five target PCBs are scaled by the individual sMMiqr total dispersion estimates.

mass fraction. Assuming that the Extract material represents effectively complete extraction of the target PCBs, this trend is compatible with systematically greater extraction difficulty with declining mass fraction in the matrix or, more plausibly, with a small but rather constant loss of all the target PCBs during the extraction stage. The impact of such loss should be inversely proportional to the amount of tissue extracted. Figure S-12 (SI) displays the concordance and apparent precision analyses of the ratio values for all participants; Figure 9 summarizes these characteristics. As the sMMiqr for the PCB 170 ratio is much larger than that of the other four target PCBs, each of the five targets are scaled to their observed total dispersion. While the resulting concordance, apparent precision, and comparability estimates therefore have no simple absolute interpretation, the relative performance characteristics of the participants’ measurement processes can still be inferred. As expected from the performance characteristics in Extract and Tissue, the Tissue/Extract ratio characteristics for participants L9 and L3 are not similar to those of the majority population. Also as expected from the observed change in concordance values in the two materials, participant L2’s extraction and cleanup components are strongly influencing the characteristics of the total process. However, participant L1’s ratio characteristics are (unexpectedly) of the majority. The strong discordance, and limited number of quantitative measurements, for L9’s Extract and Tissue measurements suggest a fundamental failure of this participant’s chromatographic process for complex materials. While participant L3′s PCB 170 Extract and Tissue results are discordantly large, the fairly systematic change from a large positive to a large negative ratio value with increasing PCB index (panel L3 in Figure S-12, SI) suggests that the extraction component of their measurement process was sensitive to some physicochemical property of the congeners. However, the pattern is not consistent with volatility; while PCB 28 is relatively volatile, PCB 170 is not significantly more volatile than are the other three targets. It is more likely that participant L3’s performance characteristics are influenced by congenerspecific limitations in both the chromatographic and extraction components of the measurement process. The asymptotic approach to consensus of participant L2’s ratio values with increasing 7334

Analytical Chemistry, Vol. 80, No. 19, October 1, 2008

Figure 10. Majority population measurement standard deviation as functions of mean. This scatterplot displays the standard deviations for selected measurements in the CH2Cl2 (labeled open squares), Extract (solid black circles), and Tissue (labeled open circles) materials as functions of the measurement means. Two lines are shown, the thinner representing the σˆ Calibration(x) ) 0.019x achievable total uncertainty function for measurements of PCBs in calibration solutions and the thicker representing the σˆ matrix(x) ) 0.054x achievable precision function for measurements of PCB contamination in mussel tissue.

PCB index (panel L2 in Figure S-12, SI) is much more plausibly related to congener volatility and may be related to volatile losses during extraction. Participant L1’s very concordant ratio values (panel L1 in Figure S-12, SI) demonstrate that whatever afflicts the Extract and Tissue measurements affects them in proportion to the effective mass of mussel tissue. This suggests that although this participant’s chromatographic process suffers from unrecognized interferences, the extraction process is typical of the majority’s. Achievable Precision and Bias. The measurement characteristics discussed above are based on robust estimation of a mixed population of measurement processes. Having identified the processes that are not of the majority population, it is possible to more directly estimate the achievable precision. Figure 10 presents the simple mean and SD estimates for the CH2Cl2, Extract and Tissue measurements of participants L4 to L8 plus the CH2Cl2 and Extract measurements of participants L2 and L10. As expected from the assessment of total dispersion from the robust estimates, the SD estimates for the selected CH2Cl2 measurements are proportional to the means. Linear regression with a forced zero-intercept estimates the proportionality as σˆ calibration(x) ) 0.019x (RSD ) 1.9%) for the determination of PCB congeners in organic solvent materials over a mass-fraction range of 40-500 ng/g. Given the similarity of the iOct and CH2Cl2 measurement performance characteristics and the excellent agreement between the gravimetric and consensus estimates, these measurements are not significantly biased. Therefore, this precision function also estimates the achievable total measurement uncertainty (the combined precision and bias) for PCB congeners in organic solvents. As the true levels of PCBs in the Extract and Tissue materials are unknown, the absence of bias in the matrix-material measurements is not demonstrated. However, the concordance of the selected measurements argues that the chromatographic separation and quantitation components of the measurement process

for matrix materials can be unbiased. The agreement among the Tissue/Extract ratios for the five target measurands suggests that PCBs can be reliably extracted from freeze-dried mussel tissue. Since the proportionality between the SDs and means are similar for the selected Extract and Tissue measurements, the extraction and isolation components of the measurement process do not appear to intrinsically increase measurement variability. Combining the two sets of five mean and SD values, the zerointercept linear regression estimates the achievable interlaboratory precision as σˆ matrix(x) ) 0.054x (RSD ) 5.4%) for the determination of PCB congeners in tissue matrix materials over a mass-fraction range from 0.5 ng/g to at least 100 ng/g. ACKNOWLEDGMENT We thank Willie May, Chair of the OAWG, for instigating the “Uncertainty Suite” study and all OAWG participants for contributing to its design, analysis, and interpretation. We thank the following individuals for actually making the measurements: Berit Sejeroe-Olsen, EC-JRC Institute for Reference Materials and Measurements, Geel, Belgium; Jorge Espinoza and Veronica Manzano, ANALAB CHILE S.A., Santiago, Chile; Dazhou Chen, National Institute of Metrology, Beijing, China; Be´atrice Lalere,

Laboratoire National d’Essais, Paris, France; Tin Win, Federal Institute for Materials Research and Testing, Berlin, Germany; C.S. Mok, Government Laboratory, Homantin, Kowloon, Hong Kong; Keiichiro Ishikawa and Masahiko Numata, National Metrology Institute of Japan, Ibaraki, Japan; Byungjoo Kim, Korea Research Institute of Standards and Science, Taejon, Korea; Melina Pe´rez Urquiza, Centro Nacional de Metrología, Quere´taro, Mexico; John Kucklick and Dianne Poster, National Institute of Standards and Technology, Gaithersburg, MD, USA; and Gavin O’Connor, LGC, Teddington, Middlesex, United Kingdom. We thank Stephen Wise, Chief, Analytical Chemistry Division, NIST, for supporting the development of the required study materials and data analysis methodologies. SUPPORTING INFORMATION AVAILABLE Additional information as noted in text. This material is available free of charge via the Internet at http://pubs.acs.org.

Received for review May 15, 2008. Accepted July 14, 2008. AC8009966

Analytical Chemistry, Vol. 80, No. 19, October 1, 2008

7335