Access to Data for Epidemiological Studies - ACS Publications

16 Access to Data for Epidemiological Studies Ralph R. Cook , Sandra L. Tirey , Nanette W. and M a r y A . Woodbury

Downloaded by NORTH CAROLINA STATE UNIV on January 2, 2018 | http://pubs.acs.org Publication Date: May 5, 1994 | doi: 10.1021/ba-1994-0241.ch016

1

2

Spadacene , 3

1

Epidemiology U n i t , D o w C o r n i n g Corporation, M i d l a n d , MI 48686-0994 Chemical Manufacturers Association, 2501 M Street, N.W., Washington, DC 20036 The D o w Chemical Company, 1803 Building, M i d l a n d , M I 48674 1

2

3

Epidemiological

research is dependent on data on exposures, health

outcomes, and confounders

(alternative causes for the specific dis

eases of interest). In addition, it often requires a means to tie these three entities together. This is usually achieved by means of personal data, such as name, birthdate, and social security number. In some projects, epidemiologists can gather the data they need directly from the study subjects; but in many projects, they must extract it from records developed for purposes other than health research.

Various

barriers are associated with access to these secondary sources. Some are related to concerns about privacy and confidentiality. Others in volve litigation, economics, logistics, and even philosophy. This chap ter explores the problems of data access and proposes some solutions.

Ε P I D E M I O L O G I C A L R E S E A R C H IS D E P E N D E N T O N D A T A : data Oil e x p o

sures, o n h e a l t h responses, a n d o n confounders (i.e., o t h e r causes for the h e a l t h response o f interest). T o c o n d u c t m e a n i n g f u l w o r k , the database m u s t b e r e l a t i v e l y c o m p l e t e ; that is, there s h o u l d b e little selection bias. I n a d d i t i o n , the data m u s t b e v a l i d : T h e r e s h o u l d b e m i n i m a l misclassification of exposures, h e a l t h responses, or confounders.

I n particular, because e p i d e

m i o l o g y is a study of comparisons, there s h o u l d be m i n i m a l differential m i s classification; that is, the relative v a l i d i t y o f the data s h o u l d b e e q u i v a l e n t for b o t h the exposed a n d the u n e x p o s e d , o r for b o t h the diseased cases a n d the h e a l t h y controls. O b s e r v a t i o n a l researchers must d e a l w i t h biases w i t h w h i c h most ex p e r i m e n t a l i s t s have little experience. H u m a n s are not exposed to one agent

0065-2393/94/0241-0231$08.00/0

© 1994 American Chemical Society

Draper; Environmental Epidemiology Advances in Chemistry; American Chemical Society: Washington, DC, 1994.


232

ENVIRONMENTAL EPIDEMIOLOGY

at a t i m e at some p r e d e t e r m i n e d c o n t r o l l e d l e v e l , b u t rather to a m u l t i t u d e of agents simultaneously i n an often i l l - d e f i n e d fashion. I n t r y i n g to u n d e r stand the p o t e n t i a l i m p a c t of one agent o n h u m a n h e a l t h , the effects of o t h ers (confounding variables that confuse the interpretations) m u s t b e a d dressed. Similarly, the pressures that lead to i n c l u s i o n i n any g i v e n study b y b o t h cases a n d controls (selection) must be u n d e r s t o o d a n d c o n t r o l l e d i n study d e s i g n , analysis, or i n t e r p r e t a t i o n . F i n a l l y , methods of classification or m e a s u r e m e n t not o n l y m u s t b e v a l i d ; they m u s t b e consistent, or misclassification—particularly differential m i s c l a s s i f i c a t i o n — w i l l result. A n y of these biases can have a significant impact o n the calculated risk estimates a n d consequently o n the interpretations that are d e r i v e d f r o m t h e m . I n some research, epidemiologists have the l u x u r y of g a t h e r i n g data d i rectly from the study subjects v i a questionnaires, c o l l e c t i o n a n d analysis of b l o o d or u r i n e , or tests of physiological function. T h e y can design the c o l l e c t i o n procedures to gather a sufficient a m o u n t of data a n d to manage a l l the phases of data quality. H o w e v e r , most e n v i r o n m e n t a l a n d occupational epidemiologists m u s t use secondary sources of data that are extracted f r o m records d e v e l o p e d b y others for reasons u n r e l a t e d to h u m a n h e a l t h research. F o r example, the data may c o m e f r o m death certificates, outpatient m e d i c a l records, p e r s o n n e l files, r e s i d e n t i a l directories, tax records, i n d u s t r i a l h y g i e n e reports, sales receipts, or process descriptions. F u r t h e r m o r e , e v e n i f the data are v a l i d , occupational a n d e n v i r o n m e n t a l epidemiologists must gain access to it. G a i n i n g access can be difficult. I n some cases, the logistics for o b t a i n i n g data are e x t r e m e l y c u m b e r s o m e . I n other cases, a d m i n i s t r a t i v e barriers are based o n p r i v a c y a n d confidentiality. P r i v a c y of the i n d i v i d u a l a n d c o n f i d e n tiality of data are b o t h e x t r e m e l y i m p o r t a n t issues i n our society. Yet e p i demiology, b y its v e r y nature, r e q u i r e s i n v a s i o n of p r i v a c y a n d c o m p r o m i s e of confidentiality. N e a r l y e v e r y e p i d e m i o l o g i c a l study involves a d y n a m i c conflict that must b e resolved. T h i s p a p e r touches on v a l i d i t y b u t focuses p r i m a r i l y o n a c c e s s — i n p a r ticular, the barriers to access. It also summarizes steps b e i n g taken i n b o t h the p u b l i c and p r i v a t e sectors to o v e r c o m e these barriers a n d t h e r e b y to facilitate the m o r e effective a n d efficient d e v e l o p m e n t of e p i d e m i o l o g i c a l information. T h e r e are differences a m o n g data, i n f o r m a t i o n , a n d k n o w l e d g e . K n o w l edge is a p p l i e d i n f o r m a t i o n a n d i n f o r m a t i o n is a n a l y z e d data. H o w e v e r , it does not follow that any a n d a l l data, e v e n i f v a l i d , can b e c o n v e r t e d to v a l i d information. H o w data are c o l l e c t e d , t h e i r focus, and t h e i r r i g o r are a l l i m portant components that are a l l too often o v e r l o o k e d . F o r example, about 5 years ago i n A l b e r t a , C a n a d a , an evaluation of p o p u l a t i o n h e a l t h statistics i d e n t i f i e d an a l a r m i n g a n d a b r u p t 5 0 % increase i n cancer rates. F o r about 2 m o n t h s , this was h e a d l i n e news w i t h various allegations about the risks b e i n g associated w i t h life-style factors, or waste



16.

COOK ET AL.

Access to Data for Studies

233

d u m p s , or o i l p r o d u c t s — a l l the standard concerns (1). E v e n t u a l l y , it was r e v e a l e d that the cancer rates h a d b e e n m i s c a l c u l a t e d (2). C a n c e r rates are s i m p l y the n u m b e r of p e o p l e (the numerator) i n a g i v e n p o p u l a t i o n (the denominator) w h o d e v e l o p cancer. To make any sense, the n u m e r a t o r m u s t b e a subset of the denominator. I n this situation, the n u merator of the cancer rates, the n u m b e r of cases, h a d b e e n o b t a i n e d f r o m F o r t Saskatchewan as w e l l as the s u r r o u n d i n g c o u n t y b o t h before a n d after the so-called cancer e p i d e m i c ; however, the d e n o m i n a t o r data, the n u m b e r of p e o p l e i n the p o p u l a t i o n u n d e r study, h a d b e e n d e t e r m i n e d i n two different ways. Because of a change i n the p o l i t i c a l boundaries halfway t h r o u g h the study p e r i o d , i n the " b e f o r e " calculation the n u m b e r h a d i n c l u d e d those w h o l i v e d i n F o r t Saskatchewan a n d the s u r r o u n d i n g county. I n the " a f t e r " calculation, it h a d i n c l u d e d o n l y those i n the county. T h e " e p i d e m i c " was not a result of an increase i n cancer. It was s i m p l y a result of u s i n g data inappropriately, i n this case data that w e r e q u i t e v a l i d w h e n a n a l y z e d o n an e l e m e n t - b y - e l e m e n t basis. T h i s p a r t i c u l a r mistake obviously t r i g g e r e d a n d r e i n f o r c e d the p u b l i c s c h e m o p h o b i a . O n e has to ask h o w m a n y other "cancer e p i d e m i c s " have b e e n a result of similar u n i n t e n t i o n a l , or perhaps e v e n i n t e n t i o n a l , misuses of v a l i d data? O b v i o u s l y , epidemiologists n e e d v a l i d data i n o r d e r to do m e a n i n g f u l e n v i r o n m e n t a l h e a l t h research. T h e s e data often c o m e f r o m different sources, each w i t h its o w n idiosyncrasies. I f investigators are not d i r e c t l y i n v o l v e d w i t h the p r i m a r y data c o l l e c t i o n , they m u s t be aware of these differences a n d adjust for t h e m i n study d e s i g n , analysis, or i n t e r p r e t a t i o n . E v e n i f data are k n o w n to b e v a l i d , epidemiologists m u s t have access to those data for analysis. E p i d e m i o l o g i s t s m u s t have access to a lot of data, because epidemiology, as a scientific tool, tends to be a b l u n t i n s t r u m e n t . I n aggregate, it can b e h i g h l y accurate, b u t it is usually not v e r y precise. T h e r e f o r e , interpretations of cause a n d effect i n e p i d e m i o l o g y d e p e n d m o r e o n the consistency of r e sults across m u l t i p l e studies than o n the presence or absence of statistical significance o r the statistical p o w e r of one study. To use the analogy of target shooting, the results of i n d i v i d u a l studies t e n d to scatter l i k e pellets of b u c k shot. W h a t is i m p o r t a n t is the p a t t e r n of results of m u l t i p l e studies a n d not necessarily the findings of any single study. F u r t h e r c o m p l i c a t i n g the p i c t u r e is the fact that m u c h e p i d e m i o l o g i c a l research is a hypothesis-generating exercise. T h e data are e x p l o r e d to i d e n tify clues that n e e d f u r t h e r investigation. D a t a are c u l l e d to d e t e r m i n e the possible. H o w e v e r , m o r e research usually m u s t b e d o n e to identify the h i g h l y probable. D u r i n g the i n i t i a l stages, it may be expedient to identify a p l e t h o r a of associations to be p r o b e d further. D u r i n g subsequent stages, it is m o r e i m p o r t a n t to focus, to test hypotheses, a n d t h e r e b y to d e v e l o p a smaller b o d y of information that e i t h e r suggests r e a l cause a n d effect or c o n -


234



versely, i f insufficiently c o m p e l l i n g e v i d e n c e of a causal association, suggests that f u r t h e r w o r k s h o u l d cease a n d resources s h o u l d be reallocated to a n other issue. Paradoxically, at the same t i m e o u r society is d e m a n d i n g answers to an e v e r - i n c r e a s i n g array o f c o m p l e x h e a l t h concerns, it is b e c o m i n g m o r e difficult to gain access to the data n e e d e d to d e v e l o p the i n f o r m a t i o n r e q u i r e d to address these concerns. Too m a n y of these barriers are of a regulatory or legal nature.

Study Group I n s u r v e y i n g the basic types of data, the e p i d e m i o l o g i s t m u s t first d e c i d e what ^roup s h o u l d be s t u d i e d . T h i s r e q u i r e s answers to the f o l l o w i n g q u e s tions: H o w s h o u l d they be i d e n t i f i e d ? W h a t c o m p a r i s o n g r o u p s h o u l d be used? W h a t are the sources of data? I n the occupational setting, it is possible to use p e r s o n n e l lists, w o r k histories, d e p a r t m e n t assignments, a n d the l i k e to d e v e l o p registries of a l l the i n d i v i d u a l s ever e m p l o y e d . To ensure that some b l o c k of former e m ployees is not m i s s i n g from the master registry, it can be cross-checked w i t h the Social S e c u r i t y A d m i n i s t r a t i o n s 941 files (3). T h e 941 files p r o v i d e an annual q u a r t e r - b y - q u a r t e r s u m m a r y of all those for w h o m the c o m p a n y has m a d e a c o n t r i b u t i o n . H o w e v e r , these sources give data o n l y o n w h e t h e r a n d w h e n someone was e m p l o y e d b y a p a r t i c u l a r company. I f the c o m p a n y has m u l t i p l e p r o d u c t i o n l i n e s , they p r o v i d e no i n d i c a t i o n of what the employees d i d at the c o m p a n y or t h e i r potentials for exposure. A d d i t i o n a l data r e sources are therefore n e e d e d . In c o m m u n i t y epidemiology, the p r o b l e m is e v e n m o r e c o m p l e x . It has b e e n estimated that at least 4 m i l l i o n p e o p l e i n the U n i t e d States l i v e w i t h i n a m i l e o f one o f 725 hazardous waste sites o n the E n v i r o n m e n t a l P r o t e c t i o n Agency's N a t i o n a l P r i o r i t y L i s t (4). I n theory, it is possible to d e v e l o p a registry of w h o lives w h e r e b y g o i n g to the p r o p e r t y tax rolls; b u t these may list the names of absentee landlords a n d not i n c l u d e the names of the r e s i dent tenants. A l t e r n a t i v e l y , one c o u l d use telephone d i r e c t o r i e s , b u t not e v e r y o n e has a telephone. O n e c o u l d go house to house a n d gather names, b u t this is a v e r y labor-intensive endeavor a n d still may not identify e v e r y one. F o r example, some c o m m u n i t i e s i n C a l i f o r n i a c o n t a i n a n u m b e r of i l legal aliens. These p e o p l e are reluctant to participate i n data-gathering exercises, especially i f they must give t h e i r names to a g o v e r n m e n t agency. T h e issue is f u r t h e r c o m p l i c a t e d b y the fact that the focus of the h e a l t h research may be a c h r o n i c disease such as cancer i n w h i c h a t i m e lag of years or decades can occur b e t w e e n the exposure a n d the c l i n i c a l identification of the cancer. I n d e v e l o p i n g the list of those i n a study of c h r o n i c disease, the epidemiologist is less i n t e r e s t e d i n w h o is exposed c u r r e n t l y t h a n i n w h o was


16.

COOK E T AL.


235


exposed historically. U n f o r t u n a t e l y , the p o p u l a t i o n of the U n i t e d States is h i g h l y m o b i l e . It has b e e n estimated that m o r e than 1 0 % o f the p o p u l a t i o n change t h e i r zip-codes e v e r y year. F i n d i n g p e o p l e can be an exercise i n " s h o e - l e a t h e r " epidemiology. G o i n g back to the occupational setting a n d g i v e n a list of the e v e r e m p l o y e d exists, the next p r o b l e m is d e t e r m i n i n g the sublist of a l l w h o w e r e e v e r exposed. S o m e t i m e s it is possible to d e v e l o p these lists (perhaps e v e n to the extent of d e f i n i n g those w i t h low, m e d i u m , a n d h i g h p o t e n t i a l for exposure) b y cross-referencing d e p a r t m e n t lists w i t h j o b d e s c r i p t i o n s , p r o cess i n f o r m a t i o n , a n d i n d u s t r i a l h y g i e n e reports. H o w e v e r , this p r e s u m e s that such i n f o r m a t i o n has b e e n gathered a n d f u r t h e r m o r e r e t a i n e d . I n the U n i t e d States, business tends to focus o n c u r r e n t s u r v i v a l or future p l a n n i n g . R e t e n t i o n a n d a d m i n i s t r a t i o n of h i s t o r i c a l records is expensive. I n these times of " d o w n s i z i n g " , records that serve no obvious business f u n c t i o n are destroyed. W h e n E u l a B i n g h a m was the h e a d of the O c c u p a t i o n a l Safety a n d H e a l t h A d m i n i s t r a t i o n ( O S H A ) , this p r o b l e m was b r o u g h t to h e r attention. She m a n d a t e d that c e r t a i n records be r e t a i n e d for a p e r i o d of 30 years after e m p l o y m e n t (5). T h i s p o l i c y s e e m e d to ensure that data w o u l d b e available to the N a t i o n a l Institute for O c c u p a t i o n a l Safety a n d H e a l t h ( N I O S H ) , the agency charged w i t h d o i n g i n d u s t r i a l e p i d e m i o l o g i c a l studies. S o m e t h i n k the regulation fell short of the mark. Because it tacitly p r o m o t e d the d e s t r u c t i o n of p o s t e m p l o y m e n t records o l d e r than 30 years, those w i t h shorter terms of e m p l o y m e n t a n d therefore shorter durations of exposure w e r e e l i m inated from l o n g - t e r m c h r o n i c disease studies. A s a c o n s e q u e n c e , the e p i demiologists* ability to explore the i m p o r t a n t d i m e n s i o n of d o s e - r e s p o n s e was restricted. I n support of e p i d e m i o l o g i c a l research, some companies have d e v e l o p e d the p o l i c y of k e e p i n g records for 75 years f r o m date of first e m p l o y m e n t ( D o w C h e m i c a l , personal c o m m u n i c a t i o n ) . F o r l o n g - t e r m e m ployees, the O S H A regulation a n d the D o w C h e m i c a l p o l i c y have basically the same effect. H o w e v e r , for s h o r t e r - t e r m e m p l o y e e s , the D o w C h e m i c a l p o l i c y ensures that m o r e useful data are available.

Exposures N o study of cause a n d effect can b e c o m p l e t e w i t h o u t some i m p l i e d or exp l i c i t estimate of exposures. I n the occupational setting, epidemiologists r e l y h e a v i l y o n i n t e g r a t i n g w o r k histories, process records, a n d i n d u s t r i a l h y giene reports. T h e first two a l l o w the specific d e t e r m i n a t i o n of w h o w e r e l i k e l y exposed to what (the agents of exposure) a n d w h e n this was l i k e l y to have o c c u r r e d . T h e t h i r d , the i n d u s t r i a l h y g i e n e records, p r o v i d e an i d e a of the levels of exposure. U n f o r t u n a t e l y , the i n d u s t r i a l h y g i e n e records m u s t be u s e d w i t h some c a u t i o n , because m u c h of the data is c o l l e c t e d for c o m -



236


pliance purposes. A g a i n , a r e g u l a t i o n p r o m u l g a t e d for purposes of p u b l i c h e a l t h garbles the intent. B y law, m a n d a t e d i n d u s t r i a l - h y g i e n e m e a s u r e ments m u s t be taken w i t h o u t regard to the use of p e r s o n a l p r o t e c t i v e e q u i p m e n t . I n other w o r d s , two e m p l o y e e s standing side b y side, one i n a selfc o n t a i n e d " m o o n s u i t " a n d the other i n street clothes, must b y law have the same l e v e l of exposure r e c o r d e d . C o m p l i a n c e - o r i e n t e d i n d u s t r i a l h y g i e n e p r o v i d e s measures of p o t e n t i a l exposure, not of actual exposure. To o b t a i n a measure of actual exposure, to a v o i d the p r o b l e m s of i n t e r p r e t a t i o n , a n d to c o n v e r t i n d u s t r i a l - h y g i e n e data to i n d u s t r i a l - h y g i e n e i n f o r m a t i o n that is usef u l for e p i d e m i o l o g i c a l research, a k n o w l e d g e a b l e i n d u s t r i a l hygienist must actively participate i n the project. O f course, this i n d i v i d u a l s p a r t i c i p a t i o n must be d o n e i n a fashion that m i n i m i z e s differential misclassification, b u t that usually can be h a n d l e d b y a w e l l - d e s i g n e d protocol. M o s t i n d u s t r i a l - h y g i e n e measurements are of the e n v i r o n m e n t , not the b i o l o g i c a l entity. T h e y do not measure w h a t enters the w o r k e r s b o d y or what enters the l i k e l y organ of c o n c e r n . I n the future, data gathered b y b i o m o n i t o r i n g techniques may p r o v e useful i n addressing this s h o r t c o m i n g . I n spite of the l i m i t a t i o n s , exposure measures i n the i n d u s t r i a l sector can be e x t r e m e l y useful, b u t o n l y i f they are gathered w i t h e p i d e m i o l o g y i n m i n d . I n contrast, the exposure measures i n c o m m u n i t y e p i d e m i o l o g y can be nebulous. T h e exposure measures may b e m o d e l e d i n some statistical fashion, or degree of exposure m a y b e d e t e r m i n e d b y distance f r o m the site b u t w i t h o u t consideration for major variables such as topographical features, w i n d d i r e c t i o n , or hydrology. I n m a n y situations, the exposures are d e f i n e d b y the study participants themselves v i a a questionnaire. T h i s approach can be fraught w i t h p r o b l e m s . F o r example, do y o u r e m e m b e r what y o u h a d for l u n c h three weeks ago; do y o u r e m e m b e r w h a t chemicals y o u u s e d i n college? O n the other h a n d , i f y o u w e r e t o l d that w o r k i n g w i t h a c e r t a i n c h e m i c a l w o u l d l i k e l y cause a painful death i n the future unless y o u i m m e d i a t e l y r e c e i v e d an antidote, m i g h t y o u not t r y to r e c a l l w h e t h e r or not y o u h a d u s e d the c h e m i c a l ? S e lective r e m e m b e r i n g is r e f e r r e d to as r e c a l l bias. E x p o s u r e data gathered b y questionnaire are p a r t i c u l a r l y subject to it. F u r t h e r m o r e , i f the situation is p a r t i c u l a r l y controversial, there may be differential r e c a l l bias w h e r e b y those w i t h p r o b l e m s do a better j o b of r e m e m b e r i n g t h e i r past exposures than do those i n the c o m p a r i s o n group. T h e result is gross overestimates of risk. A n u m b e r of e p i d e m i o l o g i c a l studies have a t t e m p t e d to use self-reporti n g of exposures. B o n d et a l . (6) investigated this practice a n d found it to be of questionable validity. F o r example, the a b i l i t y to r e c a l l one s usual a n d customary j o b , the most c o m m o n j o b h e l d d u r i n g the t i m e of e m p l o y m e n t , v a r i e d from 40 to 100%, but the ability to r e c a l l specific chemicals u s e d i n these jobs w e n t d o w n to single-digit percentages. T h e o p p o r t u n i t i e s for


16.

COOK E T AL.


237

u n d e r e s t i m a t i n g risk i f nondifferential r e c a l l exists o r for grossly o v e r e s t i m a t i n g risk i f differential r e c a l l exists are e n o r m o u s .


Because rigorous exposure data o n i n d u s t r i a l cohorts are n o t r e a d i l y available, especially to researchers outside the p r i v a t e sector, a recent effort has b e e n made to c o m p u t e r i z e the occupational i n f o r m a t i o n o n d e a t h certificates. A n evaluation of the u t i l i t y of this approach to d o c u m e n t i n g exposures has y i e l d e d most discouraging results (7). A t best, this i n f o r m a t i o n m i g h t b e used o n a l i m i t e d basis to generate hypotheses, b u t i t has n o u t i l i t y for testi n g hypotheses. F u r t h e r m o r e , such exercises l e a d to t w o other p r o b l e m s : (1) T h e s e types o f studies are often g i v e n m o r e c r e d e n c e than they deserve; (2) they often d e v e l o p hypotheses that cannot b e tested. S o m e t h i n k that i f m o r e rigorous data are n o t available, this approach s h o u l d not b e u s e d ; i f m o r e rigorous data are available, this approach may not b e n e e d e d .

Confounders I n s i m p l i f i e d t e r m s , epidemiologists m u s t evaluate the r e l a t i o n s h i p b e t w e e n exposure a n d h e a l t h response w h i l e simultaneously t a k i n g into consideration the i m p a c t o f c o n f o u n d i n g variables, t h e alternative exposures that m i g h t cause the same h e a l t h response. F o r example, i f a study w e r e b e i n g d o n e o f the potential association b e t w e e n c h e m i c a l X a n d l u n g cancer, i t w o u l d b e necessary to consider the s m o k i n g habits o f those b e i n g s t u d i e d . I f not, any statistical association b e t w e e n c h e m i c a l X a n d l u n g cancer m i g h t b e c o n f o u n d e d , o r confused, b y s m o k i n g . A confounder has b o t h a biological a n d a statistical d i m e n s i o n . F i r s t , it must b e an alternative cause for the h e a l t h o u t c o m e u n d e r study; that is, it may b e a confounder i n any study o f that h e a l t h o u t c o m e . S e c o n d , it is a confounder o n l y i f it is u n e q u a l l y d i s t r i b u t e d i n that p a r t i c u l a r study. T h e r e fore, w h e t h e r o r n o t it is a confounder is study-specific. F u r t h e r m o r e , w h e t h e r the d i s t r i b u t i o n is u n e q u a l i n a m e a n i n g f u l fashion is often a value j u d g e m e n t . M a k i n g those d e t e r m i n a t i o n s is part o f the art o f epidemiology. Stated another way, a confounder is just another type o f exposure, b u t one that is confusing o r c o n f o u n d i n g t h e interpretations o f the possible associations o f interest. C o n f o u n d e r s have a l l o f the p r o b l e m s a n d l i m i t a t i o n s that have b e e n e n u m e r a t e d for exposures, a n d some others. I n t h e occupational setting, t h e exposures o f interest can b e m o n i t o r e d b y t h e i n d u s t r y i n some reasonable fashion. H o w e v e r , the confounders c a n be exposures that take place outside the w o r k e n v i r o n m e n t , such as smoki n g , d r i n k i n g , sexual habits, o r u s i n g i l l i c i t drugs. It m a y b e difficult to o b t a i n data o n these confounders. I n t h e c o m m u n i t y setting, any effort to get i n formation o n e i t h e r exposures o r confounders can b e e v e n m o r e c o m p l i c a t e d a n d usually has to b e done d e nova. Irrespective o f w h e t h e r the e p i d e m i o l ogy is p e r f o r m e d i n the i n d u s t r i a l sector o r i n t h e c o m m u n i t y , there must


238


be some forethought about what m i g h t be a confounder to ensure g a t h e r i n g the p r o p e r data.


Health Outcomes T h e t h i r d type of data involves h e a l t h responses. M a n y occupational m e d i c a l departments gather c l i n i c a l h e a l t h data o n active e m p l o y e e s v i a automated m u l t i p h a s i c h e a l t h testing. I n a d d i t i o n , most companies of any size have h e a l t h insurance for active e m p l o y e e s a n d retirees. A s m a l l e r n u m b e r have set u p surveillance programs based o n m o r t a l i t y data. P r o b l e m s related to access a n d v a l i d i t y can occur, irrespective of source. F o r example, the m u l t i p h a s i c c l i n i c a l data usually are c o l l e c t e d o n l y o n active e m p l o y e e s w h o choose to participate. N o t o n l y does this m e a n data o n former e m p l o y e e s are not available, possibly i n t r o d u c i n g a selection bias, b u t a m o n g those to w h o m the examinations are available, there may b e an additional issue of self-selection—but selection i n w h i c h d i r e c t i o n ? A r e those w i t h concerns about t h e i r h e a l t h selecting themselves into the process to obtain m o r e information so that they can m a k e reasoned life decisions about diet, exercise, or smoking? A r e they a v o i d i n g the c o m p a n y - s p o n s o r e d h e a l t h examinations because t h e y are c o n c e r n e d that identification of a h e a l t h p r o b l e m m i g h t cause t h e m to lose t h e i r jobs? Insurance records can p r o v i d e data o n i n c i d e n c e of disease b u t o n l y for those e n r o l l e d i n the p r o g r a m , usually some subset of the active e m p l o y e e s a n d retirees. Insurance records do not p r o v i d e data o n e m p l o y e e s not e n r o l l e d or o n those w h o have left the company. N o r do t h e y p r o v i d e data of u n i f o r m quality. Insurance records are e c o n o m i c i n s t r u m e n t s . T h e y p r o v i d e a " t r i g g e r " for the p a y m e n t of h e a l t h care p r o v i d e r s . T h e y are not p r i m a r i l y designed for e p i d e m i o l o g i c a l research. To s u r m o u n t the limitations of the h e a l t h insurance records, some i n dustrial epidemiologists have u s e d t h e m as triggers to set u p disease-specific registries (8). A c l a i m for cancer, for example, triggers a c o m m u n i c a t i o n to the h e a l t h care p r o v i d e r to p r o d u c e m o r e i n f o r m a t i o n . I n some situations, it has b e e n possible to d e v e l o p procedures i n c o n j u n c t i o n w i t h hospital t u m o r registries to acquire the sophisticated data. H o w e v e r , i n other situations, hospitals have refused to share any i n f o r m a t i o n , e v e n w h e n the e p i demiologists have h a d signed releases f r o m the patient ( D o w C h e m i c a l , personal c o m m u n i c a t i o n ) . W e suspect that the refusals are p r o m p t e d b y three things: (1) It is i n convenient for the hospitals to cooperate. I n m a n y hospitals, the r e t e n t i o n of records is not an o r d e r l y process, especially for the files f r o m m a n y years ago. A n y effort o n the hospitals part to f i n d these records a n d selectively extract the information r e q u e s t e d can be an expensive operation. (2) A m o n g better-organized hospitals, m a n y s i m p l y destroy any inactive records that are over 7 years o l d . (3) H o s p i t a l s are c o n c e r n e d about the p o t e n t i a l l i a b i l i t y


16.

COOKETAL.


239


associated w i t h sharing data. It is easier for t h e m s i m p l y to refuse to c o o p erate. It w o u l d b e h e l p f u l i f procedures w e r e i m p l e m e n t e d to a l l o w r e searchers i m p r o v e d access to h o s p i t a l records, perhaps b y e x p l i c i t l y exe m p t i n g hospitals from l i a b i l i t y associated w i t h b r e a c h of p r i v a c y i f they share data w i t h professional epidemiologists p e r f o r m i n g h e a l t h research. I n selected sites, it has b e e n possible to negotiate collaborative research w i t h state t u m o r registries (9). T h i s has a l l o w e d access to sophisticated t u m o r - i n c i d e n c e data o n b o t h active a n d f o r m e r e m p l o y e e s , or at least o n those still l i v i n g i n the catchment area of the registry. It w o u l d b e useful i f t r u l y national t u m o r registries w e r e d e v e l o p e d a.nd w e r e available to e p i demiologists, perhaps w i t h access procedures based o n acceptable protocols. M u c h of the w o r k of i n d u s t r i a l epidemiologists has i n v o l v e d m o r t a l i t y as the o u t c o m e of interest. T h e y have u s e d b o t h the Social S e c u r i t y A d m i n istration (SSA) a n d the N a t i o n a l D e a t h Index ( N D I ) to d e t e r m i n e w h o is alive a n d w h o is dead. U n f o r t u n a t e l y , the S S A has lately chosen to r e d u c e severely t h e i r e p i d e m i o l o g i c a l research support activities (10). I n years past, it was possible to obtain i n f o r m a t i o n f r o m the S S A that w o u l d place f o r m e r employees i n three basic groups: l i v i n g , d e a d , a n d lost to f o l l o w - u p . R e sources c o u l d t h e n be allocated to f i n d i n g the death certificates for the d e ceased a n d to t r a c k i n g d o w n those lost to f o l l o w - u p i n o r d e r to d e t e r m i n e t h e i r v i t a l status. T h e S S A n o w provides o n l y two basic categories: d e a d a n d lost to follow-up (which i n c l u d e s a l l the l i v i n g a n d t r u l y lost to follow-up). A s a consequence, m o r e resources are n e e d e d to d e t e r m i n e the v i t a l status, and a major inefficiency has b e e n i n t r o d u c e d into the system. I n theory, the N D I , a f u n c t i o n w i t h i n the N a t i o n a l C e n t e r for H e a l t h Statistics, is specifically d e s i g n e d to support e p i d e m i o l o g i c a l research. A l t h o u g h the service this g o v e r n m e n t agency p r o v i d e s is s u p e r b , u n f o r t u nately it is somewhat constrained i n b o t h its t i m e frame a n d charter, w h i c h constrains its use for chronic-disease research (11). A t present, the N D I has data from as early as 1979. F o r data before that t i m e , it is necessary to q u e r y the S S A records for clues as to w h o is d e a d a n d the state i n w h i c h the death occurred. E v e n w h e n the S S A was p r o v i d i n g m o r e c o m p l e t e disclosure, the data w e r e i n c o m p l e t e . Information o n the state i n w h i c h the last financial transaction took place was g i v e n . H o w e v e r , the state of last financial transaction is not necessarily the state i n w h i c h the death certificate is stored. A f t e r a p e r s o n dies i n N o r t h D a k o t a , the next of k i n c o u l d process the forms i n M i c h i g a n . T h e S S A records w o u l d suggest the death certificate s h o u l d b e o b t a i n e d f r o m M i c h i g a n . H o w e v e r , any q u e r y p l a c e d there w o u l d o b v i o u s l y fail, a n d other search strategies w o u l d have to b e u s e d . F o r those e m p l o y e d i n the n o r t h e r n states, the next step often w o u l d be to q u e r y a n u m b e r of other states, for example, the " s n o w b i r d " states l i k e F l o r i d a a n d Texas, states to w h i c h retirees travel d u r i n g the w i n t e r . T r a c k i n g d o w n the records can b e a t i m e - c o n s u m i n g process. I n this e x a m p l e , however, e v e n that strategy


240



w o u l d fail, a n d a laborious state-by-state search w o u l d b e necessary. T o h e l p streamline this process, it w o u l d b e h e l p f u l i f i n f o r m a t i o n c o u l d b e made available, perhaps t h r o u g h the N D I , to a l l epidemiologists, perhaps i n d i rectly from the I n t e r n a l R e v e n u e S e r v i c e records, as it is to N I O S H . It also w o u l d be h e l p f u l i f the N D I c o u l d change its c u r r e n t l i m i t of 1979 to at least another 10 years earlier. I n this age of cancer p h o b i a a n d interest i n other diseases of l o n g latency, 1 0 - 1 5 years is too short a t i m e span. O f course, w h e t h e r the investigators use data f r o m the S S A or the N D I , they still have to go back to the states for copies of the death certificates. S o m e states, such as Texas a n d M i c h i g a n , are a joy to w o r k w i t h . O t h e r s have b e e n quite the opposite. F o r example, to obtain a d e a t h certificate from the state of S o u t h C a r o l i n a it has b e e n necessary to sign a " w a r r a n t y of c o n f i d e n t i a l i t y " , w h i c h said a m o n g other things that the p r i m a r y investigator a n d his or h e r heirs w e r e fully liable for any damage caused b y i m p r o p e r use of the data, w h e t h e r such i m p r o p e r use r e s u l t e d from t h e i r o w n actions or from the actions of any p e r s o n or i n s t i t u t i o n o b t a i n i n g access to the data (State of S o u t h C a r o l i n a , personal communication). I n S o u t h C a r o l i n a , an i n d u s t r i a l e p i d e m i o l o g i s t was not a l l o w e d to sign as an agent for the company. T h e researcher was to be personally r e s p o n s i b l e . If a court r e q u i r e d sharing the death certificate w i t h the p l a i n t i f f s l a w yers a n d they m i s u s e d the data, the e p i d e m i o l o g i s t c o u l d b e h e l d financially responsible. I n fact, a l i b e r a l i n t e r p r e t a t i o n w o u l d suggest that i f the state of S o u t h C a r o l i n a m i s u s e d the data after the death certificate h a d b e e n p r o v i d e d to an outside researcher, the e p i d e m i o l o g i s t c o u l d be h e l d financially responsible. Needless to say, this release f o r m p o s e d a significant b a r r i e r to access. I n occupational a n d e n v i r o n m e n t a l epidemiology, it is often i m p o r t a n t to do the research q u i c k l y i n o r d e r to place into perspective a h i g h l y e m o tional situation. Various companies have set u p registries d e s i g n e d to a l l o w quality studies to be d o n e quickly. H o w e v e r , some states have policies that specifically p r e v e n t that from h a p p e n i n g . F o r example, P e n n s y l v a n i a r e quires a separate application a n d w r i t t e n a p p r o v a l each a n d e v e r y t i m e a study uses P e n n s y l v a n i a data (State of P e n n s y l v a n i a , personal c o m m u n i c a tion), e v e n i f n e w data are not b e i n g requested. A l t h o u g h this p o l i c y is r e a sonable, it is still one m o r e i m p e d i m e n t to b e addressed before the research can p r o c e e d . I n some situations, it is necessary to obtain a p p r o v a l from an I n s t i t u tional R e v i e w B o a r d (IRB) before the i n f o r m a t i o n on the death certificate can be u s e d to contact the a t t e n d i n g p h y s i c i a n or the next of k i n to get a d d i t i o n a l information. T h e p r o b l e m is that the I R B m u s t b e a p p r o v e d b y the N a t i o n a l Institutes of H e a l t h ( N I H ) . T h i s seems to b e a reasonable r e q u i r e m e n t , but it contains a "catch-22". A n i n d u s t r y - b a s e d I R B can conform to a l l the recommendations of the N I H b u t still be u n a b l e to obtain N I H


16.

COOK ET AL.


241


approval because the institute w i l l certify o n l y research organizations that receive N I H funds. I f a research group is self-sufficient a n d does not want to c o m p l e t e a l l the p a p e r w o r k necessary to obtain such funds, it cannot r e ceive N I H approval. It has b e e n a r g u e d that these procedures are n e e d e d for social reasons, that is, to protect the p r i v a c y of the i n d i v i d u a l a n d that of the next of k i n . T h e paradox is that i n most states, death certificates are p u b l i c d o c u m e n t s , available to anyone w h o walks i n off the street, as l o n g as the d o c u m e n t s are r e q u e s t e d one at a t i m e . O r d e r i n g m o r e than one for h e a l t h research triggers a series of t i m e - c o n s u m i n g p r o c e d u r e s , w h i c h often do not appear i n any regulations or w r i t t e n d e p a r t m e n t policies. F o r example, i n a situation i n v o l v i n g access to copies of n i n e d e a t h c e r tificates, the state of N e w Jersey r e q u i r e d a protocol. T h e r e q u i r e m e n t of a protocol is reasonable, because a m o n g other things, it ensures that p r o c e dures w i l l b e i n place to protect against any u n d u e i n v a s i o n of privacy. H o w ever, the N e w Jersey b u r e a u specifically r e q u i r e d that the p r o t o c o l has to be r e v i e w e d a n d a p p r o v e d b y a u n i o n . I n this p a r t i c u l a r situation, eight of the n i n e p e o p l e h a d b e e n salaried e m p l o y e e s , a n d the n i n t h h a d r e t i r e d before the c u r r e n t u n i o n b e c a m e certified. T h u s , n o n e of the n i n e w e r e " r e p r e s e n t e d " b y the u n i o n . W e w o u l d l i k e to see the w h o l e process s t r e a m l i n e d . F o r e x a m p l e , it w o u l d be m o r e efficient i f access to death certificates for research purposes was u n d e r the c o n t r o l of N D I . P e r h a p s this c o u l d b e a c c o m p l i s h e d , e v e n to the extent of p r o v i d i n g the death certificate a n d the assigned nosology codes, the u n i q u e n u m e r i c a l codes for each specific cause of d e a t h as d e f i n e d i n the I n t e r n a t i o n a l Classification of Diseases. To financially support the data-gathering effort, some fee s h o u l d b e sent back to the o r i g i n a t i n g states, b u t this c o u l d b e part of the N D f s charges for the service.

Discussion and Conclusion A n u m b e r of regulations a n d r e l a t e d g o v e r n m e n t agencies have b e e n d e v e l o p e d w i t h the intent of i m p r o v i n g the i n f o r m a t i o n that is available for r e a soned decisions about h u m a n h e a l t h a n d the e n v i r o n m e n t . I n t h e o r y these regulations are good, but i n practice they often are s u b v e r t e d . T h e Toxic Substances C o n t r o l A c t (12) is an example. It contains what some consider " g o t c h a " legislative r e q u i r e m e n t s . T h e legislation does not r e q u i r e i n d u s t r y to do e p i d e m i o l o g i c a l research. H o w e v e r , i f research is c o n d u c t e d , the l e g islation effectively forces an o v e r i n t e r p r e t a t i o n of the data u n d e r some severe t i m e constraints. F u r t h e m o r e , the regulations or t h e i r interpretations change over t i m e (13). I n d u s t r y epidemiologists r e c e n t l y have b e e n r e q u i r e d to go back a n d report, u n d e r the pressure of n e w g o v e r n m e n t interpretations, findings that had b e e n p u b l i s h e d i n the o p e n scientific l i t e r a t u r e m o r e than 10 years p r e -


242



viously w i t h each report i n v o l v i n g a fine of u p to $15,000 a n d a cap of $1 m i l l i o n . U n d e r such pressures, it s h o u l d be no surprise that m a n y companies are reluctant to set u p e p i d e m i o l o g i c a l programs. O n e other i m p e d i m e n t to e p i d e m i o l o g i c a l research is the l i t i g a t i o n p r o cess a n d the role plaintiff lawyers p l a y i n it. It is the i m p r e s s i o n of m a n y i n the scientific c o m m u n i t y that w h e n a situation becomes controversial, p l a i n tiff lawyers are close at h a n d . T h e y d i s r u p t the research process i n one of two ways. F i r s t , i f the attorneys d e b r i e f t h e i r clients before they are c o n tacted b y the epidemiologists, t h e y exacerbate the r e c a l l bias, w h i c h leads to overestimates of risk. T h i s has b e e n l a b e l e d as the " l i t i g a t i o n b i a s " (14). S e c o n d , some also have the i m p r e s s i o n that plaintiff attorneys f r e q u e n t l y find it easier to argue t h e i r case o n the basis of allegation, i n n u e n d o , a n d vague hypotheses rather than facts, at least facts as scientists u n d e r s t a n d t h e m (15). A t times, it appears that lawyers advise t h e i r clients not to cooperate i n any study that m i g h t d e v e l o p i n f o r m a t i o n p r e j u d i c i a l to t h e i r case (that w o u l d support the lack of cause a n d effect). W i t h i n the ethics of the legal profession, this may be a reasonable p o s i t i o n . E p i d e m i o l o g i s t s f i n d it offensive. It is also troublesome because decisions seem to be made about c o m p l e x scientific issues b y a process that does not seem to place great emphasis o n scientific analysis. A t a recent legal s y m p o s i u m , one attorney e v e n a r g u e d that science has no place i n a court of l a w (16). T h e social i m p l i c a t i o n s of this p o s i t i o n are enormous. F o r example, some p h a r m a c e u t i c a l companies have refused to market efficacious drugs, because the p o t e n t i a l cost of d e fense against litigation, often specious litigation, is greater than the a n t i c i pated profit of the product. To h e l p i n t r o d u c e m o r e objective science into the legal process, the C h e m i c a l M a n u f a c t u r e r s Association ( C M A ) has s u p p o r t e d course w o r k at the N a t i o n a l J u d i c i a l C o l l e g e d e s i g n e d to teach judges about the p r o p e r role of science i n the c o u r t r o o m . It may sound as i f w e are a r g u i n g that it is i m p o s s i b l e to conduct o c c u pational a n d e n v i r o n m e n t a l h e a l t h research. T h a t is not o u r intent. I n most cases, it is possible to f i n d ways to o v e r c o m e these obstacles, b u t they can be v e r y t i m e - c o n s u m i n g a n d r e s o u r c e - d r a i n i n g . M a n y h e a l t h researchers t h i n k it is i m p o r t a n t to make better i n f o r m a t i o n available m o r e r a p i d l y so that better decisions can b e made w i t h i n a reasonable amount of t i m e . T h e A g e n c y for Toxic Substances a n d D i s e a s e Registries has b e e n set u p to address this issue o n a c o m m u n i t y basis (17). G a t h e r i n g the necessary data a n d c o n v e r t i n g these data to useful i n f o r m a t i o n constitute an awesome r e s p o n s i b i l i t y that A T S D R researchers w i l l n e e d h e l p to a c c o m p l i s h . I n the private sector, a n u m b e r of companies have set u p in-house e p i d e m i o l o g i c a l research units. H o w e v e r , obviously m a n y m o r e units are n e e d e d to b u i l d the b o d y of available v a l i d data to a size sufficient to address the i m p o r t a n t questions from the perspective of the consistency of study


16.

COOK E T AL.


243


results. F o r this reason the C M A established a p i l o t project c a l l e d the E p i d e m i o l o g y Resource a n d Information C e n t e r ( E R I C ) (18). T h i s g r o u p is d e v e l o p i n g a n d p u b l i s h i n g a series o f monographs d e s i g n e d to m a k e it easier for i n d i v i d u a l companies to d e v e l o p in-house expertise. O n e of these monographs is the Occupational Epidemiology Resource Manual (19). It provides an o v e r v i e w for the n o n e p i d e m i o l o g i s t manager, a n d some p r a c t i c a l tips for professionals i n t e r e s t e d i n b e g i n n i n g or e x p a n d i n g a p r o g r a m i n industry. A t least one graduate school, the U n i v e r s i t y of Massachusetts at A m h e r s t , is u s i n g it as a resource m a n u a l . It s h o u l d also p r o v e useful to consultants a n d g o v e r n m e n t research organizations. A n o t h e r m o n o g r a p h is the Guidelines for Good Epidemiology Practices for Occupational and Environmental Epidemiologic Research. T h e s e G o o d E p i d e m i o l ogy Practices ( G E P s ) are to e p i d e m i o l o g y what the G o o d L a b o r a t o r y P r a c tices, the G L P s , are to toxicology. B o t h o f these C M A d o c u m e n t s are available to anyone w h o is i n t e r e s t e d , a n d the G E P s have b e e n p u b l i s h e d (20). M o s t of the epidemiologists w h o w o r k f u l l t i m e i n i n d u s t r y , at least those i n the p e t r o l e u m a n d c h e m i c a l i n d u s t r i e s , b e l o n g to a g r o u p c a l l e d the I n dustrial E p i d e m i o l o g y F o r u m ( I E F ) . T h i s g r o u p also has p u b l i s h e d a series of monographs. T h e most recent, d e v e l o p e d i n c o n j u n c t i o n w i t h T o m B e a u c h a m p (a bioethicist at the G e o r g e t o w n U n i v e r s i t y Institute o f E t h i c s ) , has b e e n i n c o r p o r a t e d into the proceedings of an I E F conference c a l l e d " E t h i c s i n E p i d e m i o l o g y " , w h i c h was p u b l i s h e d as a s u p p l e m e n t to the Journal of Clinical Epidemiology (21). It specifies i n some d e t a i l the obligations e p i demiologists have to subjects, society, funders a n d e m p l o y e r s , a n d c o l leagues. I n s u m m a r y , a n u m b e r o f p r o b l e m s confront occupational a n d e n v i r o n m e n t a l epidemiologists as they strive to d e v e l o p m e a n i n g f u l i n f o r m a t i o n about the p o t e n t i a l h e a l t h effects associated w i t h exposures to toxic c h e m i cals. S o m e o f these p r o b l e m s are r e l a t e d to issues of validity, a n d m a n y m o r e i n v o l v e data access. Paradoxically, society is d e m a n d i n g m o r e i n f o r m a t i o n w h i l e , albeit sometimes i n d i r e c t l y , p r e v e n t i n g ready access to the v e r y data n e e d e d to d e v e l o p this i n f o r m a t i o n . E p i d e m i o l o g i s t s have f o u n d ways of acc o m p l i s h i n g t h e i r w o r k i n spite o f the system; however, it w o u l d b e h e l p f u l if the system w e r e m o d i f i e d to make access to v a l i d data m o r e t i m e - a n d resource-efficient.

References 1. Hicks, G.; Saloway, T. The Edmonton Sun 1987, April 15. 2. Roberts, W. The Record 1987, June 10. 3. Marsh, G. M.; Enterline, P. E. J. Occup. Med. 1979, 21, 665-670. 4. Principal Findings about the Linkages between Releases of Hazardous Substances and the Impact on Public Health: Public Health Impact; Biennial Report


244



Volume II, October 1986-December 1988; Agency for Toxic Substances and D i s ease Registry (ATSDR): Atlanta, G A , 1990; pp 57-59. 5. Fed. Regist. 1980, 45, 35212-35228. 6. Bond, G . G.; Bodner, K. M.; Olsen, G. W.; Burchfiel, C. M.; Cook, R. R. Appl. Occup. Environ. Hyg. 1991, 6, 521-527. 7. Olsen, G. W.; Brondum J.; Bodner, K. M.; Kravat, B. A.; Mandel, J. S.; M a n del, J. H.; Bond, G. G. Am. J. Ind. Med. 1990, 17, 465-481. 8. Pell, S.; O'Berg, M.T.; Karrh, B. W. J. Occup. Med. 1978, 20, 725-740. 9. Bond, G . G.; Austin, D . F.; Gondek, M. R.; Chiang, M.; Cook, R. R. J. Occup. Med. 1988, 30, 443-448. 10. Dupree, E. A. Epidemiol. Monitor 1990, 11, 10-12. 11. Bernier, R. H.; Mason V. M. Epidemiol. Monitor 1991, 12, 400. 12. Fed. Regist. 1978, 43, 11110-11148. 13. Fed. Regist. 1991, 56, 19514-19536. 14. Woodside, F. C.; L y d o n , D . R. N. E n g l . J. Med. 1983, 308, 1604. 15. Huber, P. W . Sci. Am. 1992, 266, 132. 16. Dunleavy, D . M . Presented at the Conference on Breast Implant Litigation, New York, June 1, 1992. 17. Baller, J.; Carlo, G . L.; Sund, K . G . Bur. Nat. Affairs Environ. Rep. 1991, 3, 1951-1956. 18. Bernier, R. Epidemiol. Monitor 1989, 10, 1-3. 19. Pastides, H.; Mundt, K. A. Occupational Epidemiology Resource Manual; Chemical Manufacturers Association: Washington, DC, 1991. 20. Chemical Manufacturers Association J. Occup. Med. 1991, 33, 1221-1229. 21. Beauchamp, T. L.; Cook, R. R.; Fayerweather, W. E.; Raabe, G. K.; Thar, W. E.; Cowles, S. R.; Spivey, G . H. J. Clin. Epidemiol. 1991, 44 (Suppl. 1), 151S169S. RECEIVED for review September 3, 1992. ACCEPTED revised manuscript January 27, 1993.


Access to Data for Epidemiological Studies - ACS Publications

Recommend Documents