Chapter 17 Reporting Low-Level Data for Computerized Data Bases 1
2
1
3
Martin W. Brossman , Gerard McKenna , Henry Kahn , Donald King , Robert Kleopfer , and John K. Taylor 4
5
1
U.S. Environmental Protection Agency, Washington, DC 20460 U.S. Environmental Protection Agency, Edison, NJ 08837 Ministry of the Environment, P.O. Box 213, Rexsdale, Ontario M9W 5L1, Canada U.S. Environmental Protection Agency, Kansas City, KS 66115 National Bureau of Standards, Gaithersburg, MD 20899 2
Downloaded by UNIV LAVAL on July 13, 2016 | http://pubs.acs.org Publication Date: December 9, 1987 | doi: 10.1021/bk-1988-0361.ch017
3
4
5
The problems of affectively describing and qualifying low-level data are compounded by the need for utilizing brief codes in combination with numerical measures of the analyte in large computerized data bases. These codes, typically supported by complex definitions, attempt to caution a wide range of data users with diverse applications. In attempting to address the data description problem issues of limit of detection, limit of quantification (alternately called limit of quantitation), criteria of detection, applicable confidence limits, etc. - must a l l be addressed. Frequently, statistical rigor is in conflict with the pragmatic use of the data. Furthermore, the difference between the quantity of analyte and results of the measuring system is often confused. The panel is composed of members of a task force formed to resolve data coding policies for one of the largest environmental data systems, STORET. They discuss statistical and analytical approaches and issues involved in coding data with the accompanying advantages and hazards.
Overview and I n t r o d u c t i o n
( G e r a r d McKenna)
As more and more e n v i r o n m e n t a l measurements a r e g e n e r a t e d , t h e r e becomes an o b v i o u s need t o i n p u t e n v i r o n m e n t a l measurements i n t o computerized databases. T h i s r e p l a c e s t h e a n t i q u a t e d and d i a p p e a r i n g p r a c t i c e o f s t u f f i n g t h e d a t a i n t o desk drawers and f i l e 0097-6156/88/0361 -0317$06.00/0 © 1988 A m e r i c a n C h e m i c a l Society
Currie; Detection in Analytical Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 1987.
Downloaded by UNIV LAVAL on July 13, 2016 | http://pubs.acs.org Publication Date: December 9, 1987 | doi: 10.1021/bk-1988-0361.ch017
318
DETECTION IN ANALYTICAL CHEMISTRY
cabinets. W i t h i n the EPA, t h e r e have a r i s e n a number o f such d a t a systems such a s , f o r example, STOREΤ, SAROAD, and ODES. One t r a d e - o f f o f these systems, which c e r t a i n l y c o n t a i n many p o s i t i v e b e n e f i t s , i s t h a t t h e y d e c r e a s e g r e a t l y the a b i l i t y o f a c h e m i s t t o communi.cate t o a d a t a user a l l the r e s e r v a t i o n s he has about h i s measurements. Most o f t h i s a r e a o f c o n c e r n s u r rounds measurements t h a t l i e v e r y c l o s e t o z e r o . Computer formats t r y t o r e d u c e t h e c h e m i s t ' s words o f warning t o a l p h a b e t i c a l symbols, used as codes f o r d i f f e r e n t c l a s s e s o f l i m i t a t i o n s . A number o f e f f o r t s have t a k e n p l a c e t o e s t a b l i s h c o n v e n t i o n s of r e p o r t i n g codes. Ihese i n c l u d e , f o r example, work done by t h e American C h e m i c a l S o c i e t y Committee on E n v i r o n m e n t a l Improvement, work done by a sub-committee o f t h e American S o c i e t y f o r T e s t i n g and M a t e r i a l s , t h e I n t e r n a t i o n a l J o i n t Commission between the u n i t e d S t a t e s and Canada, c o n v e n t i o n s e s t a b l i s h e d by t h e EPA con t r a c t l a b o r a t o r y program and c o n v e n t i o n s b e i n g e s t a b l i s h e d by t h e STOREΤ low l e v e l work group. In a d d i t i o n , t h e r e have been numerous c o n v e n t i o n s s e t a t g r a s s - r o o t s l e v e l s by i n d i v i d u a l EPA, s t a t e and p r i v a t e water t e s t i n g l a b o r a t o r i e s . Out o f t h e s e e f f o r t s have come a m u l t i t u d e o f terms such as L i m i t o f D e t e c t i o n , Method D e t e c t i o n L i m i t , Instrument D e t e c t i o n L i m i t , L i m i t o f Q u a n t i t a t i o n , C r i t e r i o n o f D e t e c t i o n and a m u l t i tude o f symbols such as the l e s s t h a n s i g n , ND, TR ( f o r t r a c e ) , U, M, J , Τ and W, and K. EPA i s now p r o p o s i n g LTL ( l e s s than lower l i m i t o f d e t e c t i o n ) and LTC ( l e s s than c r i t e r i a o f d e t e c t i o n ) i n computer s t a n d a r d s . Some o f the c o n v e n t i o n s come w i t h r i g i d d e f i n i t i o n s f o r p r e s c r i b e d use and o t h e r s come w i t h vague d e f i n i t i o n s and a l l o w f o r " a n a l y t i c a l judgment" and f l e x i b i l i t y . Lack o f c o n v e n t i o n and s t a n d a r d i z a t i o n o f codes and symbols has caused a g r e a t d e a l o f c o n f u s i o n and i m p a t i e n c e w i t h a n a l y t i c a l c h e m i s t s from t h e o u t s i d e w o r l d . A q u i c k , s i m p l i s t i c view of t h e s i t u a t i o n can e r r o n e o u s l y l e a d one t o t h i n k an a g r e e d upon, s t a n d a r d i z e d c o n v e n t i o n and one t h a t i s u n i v e r s a l l y a c c e p t e d i s n o t that d i f f i c u l t a task. To t h e computer p e r s o n t h e r e p r o b a b l y appears the p r o l i f e r a t i o n o f t o o many c o d e s . To t h e a t t o r n e y , i t appears t h a t a n a l y t i c a l c h e m i s t s a r e t r y i n g t o " w a f f l e " t o o much. To t h e s t a t i s t i c i a n , i t p r o b a b l y appears t h a t t o o much tampering o f d a t a i s g o i n g on and t o the p u b l i c i t appears t h a t t h e r e may be an attempt t o mask o r c o n c e a l i n f o r m a t i o n w i t h the use o f t h e s e c o d e s . Most a n a l y t i c a l c h e m i s t s w i l l q u i c k l y agree on t h e r e a l s o l u t i o n t o the p r o b l e m . That i s t o choose up f r o n t , o r i f need be, d e v e l o p a more s e n s i t i v e and a p p r o p r i a t e methodology f o r the intended use. Such a method s h o u l d get you w e l l up on t h e measurement and q u a n t i t a t i o n range and l e a v e a l l the low l e v e l c o d i n g problems b e h i n d . However, t h i s u s u a l l y i s n o t t h e s o l u t i o n . F i r s t o f a l l , many o f our s t a t e - o f - t h e a r t methods which we have now a r e n o t more s e n s i t i v e t h a n a r e a r e a s o f e n v i r o n m e n t a l c o n c e r n and i t t a k e s c o n s i d e r a b l e time and e f f o r t t o d r i v e down the l e v e l s o f d e t e c t i o n . A l s o , f o r some p a r a m e t e r s , e n v i r o n m e n t a l t o x i c i t y exposure l e v e l s a r e b e i n g d e t e r m i n e d c o n c u r r e n t l y w i t h s u r v e y measurements b e i n g made and, t h e r e f o r e , a l l p o s i t i v e f i n d i n g i s t r e a t e d w i t h g r e a t i n t e r e s t , even i f around z e r o . A l s o , t h e p u b l i c sometimes p e r c e i v e s any f i n d i n g , even i f q u a l i f i e d , as s i g n i f i c a n t even as Avogadro's number i s approached.
Currie; Detection in Analytical Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 1987.
Downloaded by UNIV LAVAL on July 13, 2016 | http://pubs.acs.org Publication Date: December 9, 1987 | doi: 10.1021/bk-1988-0361.ch017
17. BROSSMAN ET AL.
319
Reporting Low-Level Data
C o n s e q u e n t l y , e f f o r t s must c o n t i n u e t o e s t a b l i s h a u n i v e r s a l convention to t r e a t l o w - l e v e l data. Such c o n v e n t i o n must: 1. be s a t i s f a c t o r y t o t h e a n a l y t i c a l c h e m i s t and n o t compromise o r o v e r - g e n e r a l i z e t h e l i m i t a t i o n he i s t r y i n g t o communicate; 2. a l l o w f o r v a r y i n g l e v e l s o f r i s k f o r Type I a n d I I e r r o r s t o accomodate d i f f e r e n t i n t e n d e d uses o f t h e d a t a and d i f f e r e n t management p h i l o s o p h i e s w i t h r e g a r d t o i t s u s e . 3. be c o n s i s t e n t l y u n d e r s t o o d , even by c h e m i s t s n o t w e l l v e r s e d i n s t a t i s t i c s so c o n s i s t e n t use i s f o s t e r e d ; and, 4 . be f r e e from m o d i f y i n g o r a d u l t e r a t i n g the d a t a t o make i t unuseable t o a s t a t i s t i c i a n . In any e v e n t , coming up w i t h such a c o n v e n t i o n w i l l never p r e c l u d e t h e need f o r a s e r i o u s i n v e s t i g a t o r o r d a t a u s e r from g o i n g b e h i n d t h e d a t a i n t h e d a t a base. T h i s i n c l u d e s f u r t h e r communicating with the o r i g i n a t i n g a n a l y t i c a l chemist e i t h e r d i r e c t l y o r i n d i r e c t l y through t h e e x a m i n a t i o n o f a l l s u p p o r t i n g d a t a and q u a l i t y c o n t r o l documentation. Also, i t includes understanding the data i n t h e c o n t e x t o f t h e e x p e r i m e n t a l d e s i g n f o r which i t was g e n e r a t e d . vfe hope t h a t t h e f o l l o w i n g p r e s e n t a t i o n s and your i n p u t p r o v i d e us w i t h some h e l p i n r e s o l u t i o n o f t h e s e p r o b l e m s .
will
A R e t h i n k o f t h e F a c t o r s I n v o l v e d i n R e p o r t i n g R e s u l t s Below t h e Method D e t e c t i o n L i m i t . (Donald E. King) The a p p l i c a t i o n o f t h e c o n c e p t o f " d e t e c t i o n l i m i t " as a c r i t e r i o n by t h e a n a l y t i c a l s c i e n t i s t f o r w i t h h o l d i n g t h e r e s u l t s o f lowl e v e l measurement i s not s u p p o r t e d . T h i s usage may a r i s e from t h e mistaken b e l i e f t h a t the o n e - t a i l e d s t a t i s t i c a l t - t e s t i s a t e s t o f t h e q u a l i t y o f t h e r e s u l t , r a t h e r t h a n t h e e x t e n t t o which t h e r e s u l t i n d i c a t e s t h a t a n a l y t e i s p r e s e n t i n t h e sample. I t i s also t h e r e s u l t o f c o n f u s i n g t h e l i m i t a t i o n s o f d e t e c t i o n and measurement w i t h t h e l i m i t a t i o n s o f t h e a n a l y t i c a l p r o c e s s and t h e impact o f sample m a t r i x e f f e c t s . A f u r t h e r argument a g a i n s t t h e use o f t h i s c o n c e p t as t h e b a s i s f o r not r e p o r t i n g r e s u l t s i s t h e improper h y p o t h e s i s t h a t a low r e s u l t i s n e c e s s a r i l y d e r i v e d from a p o p u l a t i o n o f r e s u l t s w i t h mean z e r o . I n s t r u m e n t a l d e t e c t i o n l i m i t (IDL) p r o t o c o l s a r e used t o d e t e r m i n e when an a n a l o g s i g n a l i s s u f f i c i e n t l y d i f f e r e n t from t h e b a c k g r o u n d n o i s e t o c o n c l u d e t h a t a m e a s u r e a b l e " r e a l " s i g n a l has been o b s e r v e d . But IDL t e r m i n o l o g y and e s t i m a t i o n p r o t o c o l s a r e not standard. More r e c e n t l y d a t a r e p o r t a b i l i t y has been l i n k e d t o t h e r e p e a t a b i l i t y o f measurement o f samples a f t e r a more o r l e s s ext e n s i v e sample p r e p a r a t i o n and a n a l y s i s p r o c e s s . Many a n a l y t i c a l c h e m i s t s c u r r e n t l y c o n s i d e r i t improper t o r e p o r t measurements o b s e r v e d t o be below t h e i r Method D e t e c t i o n L i m i t (MDL). They f a i l t o r e c o g n i z e t h a t t h e measurement was i n f a c t above t h e IDL and i s t h e r e f o r e a r e a l r e s u l t . The a n a l y s t o f t e n tends t o o v e r - e s t i m a t e t h e " d e t e c t i o n l i m i t " b e c a u s e o f c o n c e r n about t h e a c c u r a c y o f s e t t i n g z e r o , t h e p o t e n t i a l f o r b i a s , m i s i d e n t i f i c a t i o n , e t c . , and f e a r t h a t t h e d a t a u s e r w i l l draw u n s u p p o r t a b l e c o n c l u s i o n s .
Currie; Detection in Analytical Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 1987.
Downloaded by UNIV LAVAL on July 13, 2016 | http://pubs.acs.org Publication Date: December 9, 1987 | doi: 10.1021/bk-1988-0361.ch017
320
DETECTION IN ANALYTICAL CHEMISTRY
The symbol "
