CLINICAL CHEMISTRY
Statistics
Carl C.Garber,. R. Peter Mallon, and Arlene 5. Swern MetPath, One Malcolm Avenue, Teterboro, New Jersey 07608 Clinical chemistry offers many opportunities for the application of statistics,extending from the medical research that defines the efficacy or relationship of specific test procedures to specific pathologic diseases (or to the absence of disease, or wellness), to the evaluation of the analytical performance of a test, to the selection of sensitive quality control procedures, and to information processing of test results by the clinician. This review will focus primarily on the laboratory testing component of clinical chemistry. This being the first statistical applications review for this Clinical Chemistry review series in Analytical Chemstry, it is appropriate that several classic and serial works be mentioned at the outset of eachtopic. However, for ageneraldiscussion of statistical applications in clinical chemistry, the reader is referred to such reference books in clinical chemistry as refs V1 and V2. An excellent introductory text to the methods of statistics used in the clinical laboratory has been written (V3). In addition, two key references for statistics in quality control are highly recommended (V4, V5). The proceedings of recent conferences dealing with quality control (V6, V7), method validation (V8),proficiency testing (V9),and quality goals (VIO) are also recommended. This review proceeds along the progression of various phases of laboratory testing beginning with research and development, to method evaluation, establishment of reference limits for healthy and disease states, qualitycontrol,proficiencytesting, with abrief discussion on diagnostic interpretation of results, and finally, statistical aspects of total quality management that tie all these together. This review is intended to present key issues and best practices that have been reported in the literature over the past several years, and as such is not intended to be an exhaustive review.
RESEARCH AND DEVELOPMENT One of the prime objectives of a research and development program in clinical chemistry is to convert theoretical principles of chemistry into exceedingly robust analytical procedures (optimal accuracy, precision, stability, specificity, and so on) that are suitable for routine use in the clinical laboratory, wherein operational factors like short turnaround time for the assay, 24-h per day availability, ease of use, and low cost are primary concerns. Historically, components affecting variation in test performance had been explored one at a time, and standard descriptive statistics were used toanalyze the data. Moresophisticated methodsfor designing experiments were developed which allow for construction of efficient experiments which enable the assewment of multiple componentssimultaneously in acooptimization process (VIIV13). One of the first published examples of this approach dealt with the development of an automated assay for total cholesterol in serum (V14).and a more recent examnle used this technique to optimize antigen to antibody ratios for competitive RIA assays (VZ5). Once test parameters have been defined, it is imponant to determine t he performance characteristics of the assay. The analysis of variance technique (ANOVA) has become a standard technique. not only in the R&D phase, but in the method evaluation phase described below. Advanced applications of ANOVA techniques, applied to well-designed experiments, provide for the determination of many effects in one experiment, such as within-run imprecision, betweenrun imprecision, nonlinearity, carryover of reagent or sample, effect of calibration stability, and contributions of other important factors. Several modelsofthisapproach have been proposed(l'ffi-V21). Anexcellent introductory text to these methods has been written (\'22). Most importantly this approach also provides for estimates of interactive effectn. The idealassay designshodd haveminimalinteractiveeffecm, so that each component of the design can he individually controlled. 48OR
ANALYTICAL CHEMISTRY. VOL. 65. NO. 12. JUNE 15. 1993
Carl C. (iarber 1s tha Director, Quallty
Assurance. at MetPam's main reference laboratory in Teterboro. NJ, since April 1992. Previously. he was wlth Du Pont Diagnostics in Wllmington. DE, and also me University of Wisconsin Hospltai and Medical School, Madison, WI. Dr. Garber received his B.S. in chemlsby from the Universwof Alberta. Canada, and his M.S. and PhD. (1976) degrees in analytical chemistry from the University of Wisconsin-Madison. HIS interests lie in theapplicatkr of statistics intheevaluation of method performanceand intheselection of quality control procedures. He has coauthoredseveral chapters in this area in cllnical chemlsbyreference textbooks and has presented workshopsthroughoutthe Unlted States, Canada, and Mexico.
am
R. Peter Mallon is Directw 01 Quallty at Corporate Headquarlers for MetPam Inc. Prior to joinlng MetPath Inc. he worked as a research assistant at th-3 New Yo& University Medical Center. Dr. Mallon received hls Ph.D. in EnvironmentalHealth Sciences from New York University. Dr. Mallon has written several mmrs and has presented workshopsand seminars about the application of "Total Quaiky Management Principles" In heam care.
_ ~ - ~ ~~
h.A ~ l e n eS. Swern Is currently a con- p*? ~~Mngstattstlclan at MetPamand hasbeen associated wkh MetPath slnce 1979. ' ,-
Some of her past clients have been AmerlcanCyanamid.the Sbang Clink.and lhaNewYukBoardofHeal. Sheeamed a B.A. degree In mathematics and an M.A. degree in applied mathematics. both from Hunter College and a Ph.D. in biostatistlcs from Yale University in 1968. Before becoming associated with MetPath. she workedasa biostatisticlanonclinicalblab. at Mount Sinai School of Medicine. Her current research interests are in the area of muliiple end-point modeling of survival data
METHOD EVALUATION I t has been a general laboratory practice for most clinical laboratories to perform some degree of experimental testing of a new test procedure in the laboratory before adopting if for routine use. This practice has been formalized recently in the federal regulations, the Clinical Laboratory Improvement Amendments of 1988 (V23). Method characteristics to he assessed include precision, accuracy, analytical sensitivity (detection limit), analytical specificity (interferenceeffects), assay range (function of linear or linearized response), reference range, and the efficacyofquality controlprocedurea. In demonstrating acceptable performance, the manufacturer of a test system should perform extensive and statistically valid studies to establish product claims, while the clinical laboratory may perform more limited studies sufficient in scope to validate the product claims (V24). One of the first recommendations as t o the appropriate types of studies to be included in a method evaluation investigation was described in 1970 (V2.5). A problem that laboratorians faced a t that time was the question of how best to summarize the results of a specific experiment in a way that leads to unambi uous interpretationandcorrectiveaction,ifnecessary.Alanfmark paper in 1973 clarified the sensitivity of different statistics to different types of analytic error (V2fi). One of the key conclusions of that paper was the point that the correlation
CLINICAL CHEMISTRY
coefficient (Pearson product-moment coefficient, r ) had no sensitivit to the systematic differences of two methods, that it was m&ly sensitive to the random variation between two methods, and that it was primarily sensitive to the range of the data included in the study. Merely by collecting data over a wide range, one could obtain a hi h correlation coefficient, ap roaching 1.0o0. This was furt%er reinforced (V27-V29). &fortunately, we still find the correlation coefficient is the most commonly employed statistic to summarize the relationship between two assays (V30, V31). A series of five articles were published in 1978 (V32-V36) that provided detailed recommendations not only on experimentalprotocols but also on the estimation of specific errors which could be related to medically allowed maximum errors. Simultaneously, the National Committee for Clinical Laboratory Standards (NCCLS)organized expert subcommittees to address the need for a consensus agreement on appropriate design of ex eriments and statistical analysis for method evaluation. Fhese resulted in a series of uidelines for the estimation of precision based on ANOVX jV37), linearity (V38),and interference (V39),for a com arison of methods study (V40), and for a preliminary ev uation based on a multifactorial experimental design (V41). A 10-part series on laboratory statistics defined many of the fundamental concepts in statistical analysis of laboratory data with exam les (V42-V51). Yet with this background, incorrect use otfeast squares statistics continues as a common problem, due to the fact that laboratory data typically violate the ke assumptions for least squares regression analysis (V52). regression a proach that allows variance of both the deendent a n t i n d e endent variables was first proposed by beming ( V53)and as been shown to provide a moreaccurate estimate of the slope and intercept of the re ession line ( V27, V52, V54). Another "re easion" techniqueras been pro (V55, V56) which seEcta the median slope of al the combinations of lines that can be drawn connecting any two points in a X-Y scatterplot. The advantage of this approach is that it is non arametric; it places no requirements on the distribution ofe!t data. A number of other papers addressing specific statistical applications are worth noting: interpretation of least squares statistics (V57),comparison of means of data (V58), estimation of within-day and between-day components of imprecision( V59),and develo ment of a model that assesaes both independent and depengnt interference effects (V60). There remains considerable discussion as to ap ropriate statistical tests for linearity (V61-V63). Calc d t i o n of total analytic error that includes random interferencesand protocol-specificbiases estimated from a method comparison study was proposed, using a multifactor protocol (V64). With all this attention given to the design of test protocols, it is important to keep in mind the prlmary purpose for performin method evaluation experiments: to determine whether t i e analytical performance of a test method is sufficiently accurate and precise for the medical application of the test. Historically, the approach taken was to base decisions on the statistics themselves, as in the correlation coefficient, or the standard deviation, or the regression equation, or on statistical tests of significance, such as the x2 test, the F test, or the Student's t test. While these forms of statistical anal sis may be interestin , they do not address the clinicalappkxtion of theresulta. $he Westgard approach to method evaluation (V65) involves the estimation of the itude of specifictypes of errors (random and systematic) % : hen making decisions as to whether the individual errors and their combined effects (total error) are sufficiently small to provide useful test results for the medical application. Hence, the concept of decision-making based on "clinical significance" rather than "statistical significance" came into practice.
9
x
K
Yd
QUALITY GOALS Obviously, to make decisionsbased on"c1inicalsignificance" one must have definitions for the maximum amount of error that can be tolerated and not adversely affect medicalpractice. This has been the subject of much debate, of many uestionnaires, surveys, interviews, and conferences (V66-$71), highlighting the need to continue to develo models and strategies to define these standards or targets ofperformance.
While one may be fairly rigorous in the experimental design, statistical summaries, and estimation of errors, the weak link in the whole approach to decision making is the definition of the erformance standard for a iven test. The CLIA 88 reglgations (V23) provide stand8ards of performance for proficiencytesting,but these may or may not reflect standards of performance for medical practice. At the very least, it must be concluded that the combined effects of all errors present in a test s stem must be at least as small as these performance stan1ards.
ESTABLISHMENT OF REFERENCE VALUES The International Federation of Clinical Chemistry (IFCC) issued a series of recommendations on the determination of reference intervals, addressin the concept of reference intervals as opposed to "normrf rangen (V72),the selection of donors (V73),the collectionof samples (V74),the statistical analysisof the data ( V75),and the presentation of the observed values related to the reference range (V76). The NCCLS has also issued a guideline on the determination of reference intervals (V77),which was further reviewed in (V78). Types of variables that need to be controlled or specified include exercise, posture, diurnal variation, diet, medication, graphic location, race, hereditary factors, sex, and importance of standardized sampling conditions a n x h e use of repeated measurements was emphasized in ref V79 to addreas transient changes in a given analyte. A comprehensive review of both the sampling and statistical issues has been published (V80). From a statistical analysis point of view, one of the key issues regardin the estimation of a reference interval is the distribution of t t e data (V81). The x2 test, the Kolmogrov-Smirnov test, and the assessment of a normal distribution plot (result versus cumulative percentage of result) are a proaches that are commonly used. Transformations of &ta using logarithms, reci rocals, square roots, etc., have been used to obtain predictabye distributions (V82) and identification of reference interval limits. A nonparametric approach where the results are listed in ranking order enables the selection of certain percentiles as the appropriate cutoffs (2.5% and 97.5%, for example, to. define the lower and upper reference limits). This is particularly robust in large populations because no assumptions are made about the distribution of the data (V83). The disadvantages of nonparametric approaches, is that they ignore the major portion of the data base, and they focus on the extremities of the data. Thus, they may be more sensitive to mar inally healthy donors that were not screened out in the se ection process. An analysis on the minimum number of data that should be included in the determination has been presented (V84). Many of the statistical issues in the determination of reference limits have been reviewed ( V85). A review of many of the practical and clinical issues was presented (V86) for estimation reference ranges from both selected PO ulations and from individuals serving as their own reference ase. Yet with the extensive development of both experimental and statistical theory of reference ranges there remains a concern for the slow implementation of such knowledge in the laboratory and clinical environment (V87).
Pi;
P
\
INTERNAL QUALITY CONTROL We have seen significant advances in Statistical analysis and design of quality control rocedures over the past decade. Yet we cannot lose sight oPthe problems associated with quality control materials in that their properties may not reflect the characteristics of patient samples in terms of their analytical response. A classic review of these issues was presented (V88) and most still prevail today (V89). From a statistical point of view, a key paper a pearing in 1981 launched the advance in the practice of qu&ty control (V90). This paper provided the transition from Levey-Jennings quality control charts, fiist published in 1950 (V91) and reprinted in 1992 ( V92)to multirule quality control procedures that could be selected or combined on the basis of the performance of both the assay and the quality control procedure. Key factors to be considered in the development of a quality control program are the probability for error detection, probability for false rejection, number of quality control measures in a given run, and frequency of errors. This
CLINICAL CHEMISTRY
was reinforced in a subsequent aper on a redictive value model to quality control ( V93). $any other egecta and factors have been examined such as effectsof between-run and withinrun com onents of variation on quality control (V94, V95), effects ofskewness (V96),and effects of persistent error (V97, V98). The concept of selecting QC rules to detect maximum clinically allowable error was presented in ref V99 and also on a test-specific basis within a single multitest analyzer (V100). With larger numbers of control measurements per run, QC rules based on mean and variance were shown to be more powerfulor selective than those rules based on individual observations (V101). Other special approaches to quality control include the use of movin slope charts (V102), regression analysis (V103), and CU8UM (cumulative sum) techniques (V104),which are particularly sensitive to drift in the mean. Given a certain performance level defined by the method standard deviation (or process capability), one approach that has been presented to achieve process improvements is the concept of evolutionary operation which allowsfor refinement of a production process during operation (V105) and has become a basis for quality improvement strategies, discussed later in this review. Several approaches have been described that incorporate statistical assessment of the mean and distribution of the results of patient samples as additional measures to monitor method performance (V106). However method performance is monitored, the combined and interactive effects of both the method performance characteristics and the quality control performance characteristics should be considered together for a true total error assessment of a process (V107).
EXTERNAL QUALITY CONTROL (PROFICIENCY TESTING) This subset of quality control (external) has received considerable attention in the United States, especially since the passage of CLIA 88 regulations mandating successful participation in proficiency testing programs by all laboratories that erform moderate1 and highly com lex tests [ratings are [sted in the Federa Register (V23)l. fuccessful participation means reporting test results on specificsamples that are within certain defined limits of the defined target or group mean for at least 80% of the samples, in at least two of every three testing events. The very Fist proficiency testing program, more than 40 years ago, provided the first documentation of between-laboratory consistency (V108). Perhaps the greatest contribution of this first interlaboratory study was that the data provided a forum upon which to mount an educational program to advance the analytical science of clinical laboratory testing. The issue of sample cornmutability may be an even greater issue with external quality control, where different test systems are compared, relative to a single target value (VIOS). With the advent of the 1988 federal legislation the focus has changed from education to quantitative performance. Simulation studies showed the necessary accuracy and precision required for a given test system to achieve a high probability of acceptable performance (VIIO-V112). But previous to this was a report on a very extensive statistical anal is to define clinically relevant proficiency testing limits ( E 1 3 ) . From this, fixed limits were proposed. This conept was incorporated in the CLIA 88 regulations, in contrast to the traditional peer group statistical limits. If an external quality control pro am is based on criteria determined from group statistics, t g n the detection of outlying results may be an important factor (V114). On the other hand, a statistical analysis of programs based on fixed limits, such as the CLIA 88 program, may show a samplin bias, such that laboratories that offer a larger test menu maye! at a greater risk of failin proficiencytestmg contrast to this than the smaller laboratory (V115). rojection, one work re orted the actual performance of Lboratories relative to R I A 88 standards for therapeutic drug monitoring and toxicology tests and found that limited service laboratories were more susceptible to sanctions than full service laboratories (V116). One of the key ur oses for CLIA 88 was to extend standards of quality to lagoratory sites, including for the first time, regulatory oversight of hysician office laboratories, the performance of which has !,en summarized (V117). In general, the CLIA 88 performance standards were compatible with the current state of
f
h
482R
ANALYTICAL CHEMISTRY, VOL. 65, NO. 12, JUNE 15, 1993
practice for most laboratories for most tests evaluated (VI 181, with possible exceptions being free thyroxin, digoxin, and triiodothyronine, and sodium, where failure rates were reported, ranging from a high of 5.2 5% for the former to 2.4% for the latter. Of the external quality control programs for immunoassays reviewed (V7),a novel statistical analysis of qualitative tests (i.e., for binary or yes/no results) was presented (V119). Positive rates as a function of analyte concentration or dilution factor may offer a means of comparing different qualitative assays, particularly in the equivocal region at the boundary of positive and negative results. Yet, the authors point out, that as these assays progress from strictly binary, to semiquantitative, to quantitative with advancing technology, the understanding of relative performance and correctness of result is enhanced considerably by the numeric result or strength of signal.
DIAGNOSTIC INTERPRETATION For the clinician, the interpretation of a laboratory result ultimately relates to the differential diagnosisof disease, which is more complicated than simply determining whether the result is within the reference limits. The authors of Beyond Normality: The Predictiue Value and Efficiencyof Medical Diagnosis in 1975 provided a comprehensive review of the value of a test based on diagnostic sensitivity, diagnostic specificity, and the predictive value of the test, based on the prevalence of disease (V120). This process required classifying results in either a “negative”or ‘positive” category,even though most laboratory tests exhibit a continuum of values. Receiver operator characteristic curves (ROC) rovide a graphical means of simultaneous assessment of the & n o s t i c sensitivity and specificity for candidate decision levels or cutoffs (a plot of sensitivity on the Y-axis versus s ecificity on the X-axis). Hence, ROC curves may also be Lased as “relative operating characteristic” curves. The CCLS has recommended guidelines on the use of ROC curves (V121). Recent examples in the literature include the report of the determination of a decision point for serum pro esterone as an indicator of abnormal pregnancies early in t l e gestation period (V122). In another recent report (V123),the authors determined a urine calcium threshold of less than 12 mg/dL for those pregnant women at risk for preeclampsia with a sensitivity of 85%, specificity of 91%, and positive and negative predictive values of 85% and 91% , respectively. As mentioned above, the use of such terms as diagnostic sensitivity and specificity and such tools as ROC curves dichotomizes the data, while laboratory results vary continuously and clinical diagnosis is often dependent on the magnitude of the result. The likelihood ratio (L-value) generalizes the concept of predictive value to quantitative tests (V124) so that the quantitative information is not lost in a binary code. An excellentintroductory text on this sub‘ect has been written (V125). The likelihood ratio has another very important benefit. It provides a structure for evaluating the effectiveness of multiple tests in the interpretation of a disease-orientedpanel of tests. Making different assum tions about the multivariate distributions of these groups o f tests leads to an assortment of analyses including discriminant analysis and logistic regression (V126, V127). For example, the likelihood ratio is currently being used in the assessment of the risk of Down syndrome using three tests: a-fetoprotein, @-humanchorionic gonadotropin, and free estriol (VI%), where the assumption for the likelihood ratio of the combined tests is that of two multivariate normal distributions.
J
TOTAL QUALITY MANAGEMENT The future success of clinical laboratories is tied not only to advancements in technology and intelligent statistical applications, but to the laboratory’s ability to provide high quality servicesto the clinician and patient. Thus, it behooves the clinical chemist to expand the boundaries of the laboratory from the analytical bench to encompass the entirety of the process (from test ordering to test reporting to test interpretation) with a totalquality man ement perspective. This is an area that sofar has received litTe attention in the clinical laboratory compared to other service or manufacturing environments. This expanded horizon must also be done with a sense for continual quality improvement.
CLINICAL CHEMISTRY
The College of American Pathologists Conference XVII provided a forum in 1990 to foster an expanded commitment to excellence,quality assurance, and quality improvement in the laborato and the clinical setting (V129). More recent publicationsxave described the application 0.f total qualit management principles to the processes of delivery of healti care, generally, and in the laboratory, more specifically. The distinction between quality control (QC)and uality assurance (QA) and total quality management is best escribed in one work (V130),based on what is commonly referred to as “The Juran Trilogy”. defines three phases to total quality man ement The(TQ ) quality planning, quality control, and qu&y improvement. The re latory and certification requirements of the laboratory ingstry do not always direct the energiesof the laboratorytoward continuous quality improvement (CQI),but rather toward assurance that quality is controlled (V131). Most training programs of health care professionals have been directed toward the response to problems, Le., to return events back toward a previously controlled level. Few training programs in health care provide training on how to prevent problems; rather these programs train in problem detection and response (V132). Several recent works have described the application of Deming’s “fourteen points” (V133) to health care delivery and clinical laboratory process. One key outcome is the recognition that there is a customer (either internal or external) included in these processes (V134-VI36). The application of TQM principles to systematically reduce the numbers of lost samples within a large reference laboratory has been presented (V137). Similarly,the applicationof TQM principles to reduce cycle time for information response for rece tionist and for medical information nurses (specialists) resufted in a reduction in the number of rejected Medicare
8
trilP
While the 1andmtykQCpaper (V90)advanced the practice of quality control wlth multirule rocedures at the analytical site, .quality improvement has een achieved using seven quality control tools (five of which are statistically based) to reduce variation and waste in pre- and postanalytical processes (V141). Opportunities for quality improvementproceed from a data base of process measurements 0‘134). According to Juran (V130),twojourneys are required for systematicquality improvement: the diagnostic journey and the remedial ourney. The diagnosticjourney is the journey from symptom back to cause, the remedial journey from cause to remedy. Juran stresses the critical importance in distinguishing between these two activities. This difference should be intuitive to those in health care (V141). Without the correct diagnosis, a correct and permanent remedy could not be offered to the patient. Intuition notwithstanding,the authors go on to demonstrate the value of using these seven quality control tools in the analysis of proficiency test errors. Other tools for quality improvement include benchmarking (V142, V143),the use of cross-functional quality improvement teams ( V140),and root cause analysis (VIM). The “kaizen”principle of gradual, incremental, and continuous improvement has been roposed as the force that will drive our industry toward wordclass health care standards (VI&), while others maintain that significant step-function improvements in the laborato are within reach when cycle time is measured, controlley, and improved (V146). Just-in-time (JIT) management of preanalytical processes is discussed (V147),which improves efficiency because the reduced waste and lower inventory direct processes toward zero defects and eliminate nonvalue-added activities.
E
SUMMARY The ap lication of statistics in the clinical laboratory has facilitatef improvements in processes in all phases of ?alytical testing, from research and development to evaluation of analytical performance, to selection of quality control procedures,and to interpretation of results. Two key elements to continued pro ess in clinical laboratory services include, first, an increase&nderstanding of the proper use of statistics. This will facilitate more appropriate application and inter-
pretation of their use not only in the anal ical hase but also in the pre- and postanal ical steps. &on$ such a total system perspective is neec?ed to drive toward error-freetesting services. LITERATURE CITED (VI)Tbtz,N. W. T e ~ o f W.B.SaundersCo.: ~ i ~Phlleddphle, ~ 1986; Chapters 2A-2E. (V2) Kaplan. L. A.; Pesce, A. J. CHnicai chemkrby: 7hory, A n a m end correlebbn; c. v. Mosby: st. LOUIS. Mo, 1989 chapters 16-19. (V3) Welsbrot, I. M. Stablsbrcs for ihe CHn~lLaborerOryrJ. P. Llpphcott and Co.: Philadelphia, PA, 1985. (V4) Westgard, J. 0.; Bany, P. L. Cmt EtTecthro OueNtyConid: Manag@ the OuelHy end p r o d u c of ~ Analytlcel Rocmseq AACC: Washlngttm DC, 1986. (V5) Cembrowskl, 0. S.; Carey, R. N. Laboretory OUeNtyknagement, Wand QA; ASCP Chicago IL, 1989. (V6) Proceedlngsof Ouelity Controlhthe 1000s; Park Nlcollet MedlcalCenter, MlnneapoiIS, MN; Leb. M.1989, 20, 375-436. (V7) Franzini. C., Fraser, C. Q., Mahrano, R., Ede. Analytical qualityanddlagroetlc performance In immunoassays. Ann. Ist. Super. Sen& 1991,27,357-553 (English). (V8) Proceedingof the CAP Conference XXI VeMadeMonofLeboretory MUmds and ~nsbuments,oct 21-23, I S ~ I ; m.pew.Lab. M. 1992, 116, 701-803. (V9) Proceedk.1g.sof the AACC ClhlcalChemlstty Forum C)OwmmentR~&dbn: Can It guarantee quality In the cHnical leborerwy?, Nov I8- 19, 1991; CWn. Chem. 1992, 36, 1203-1288. (VIO) Prowedlngeof the AACC Cllnicaichemistry F m Ac#xecyandpu&&n goalsin dhicalchemlsby: can theybedeflnedbymedJcelrekvance?~Nov 16-17, 1992 a n . chem., In press. ( V l l ) Box, G. E. P. In &s&n andAna&sk of I n d u s l r & i ~ n t s Davls, ; 0. L., Ed.; Hafner: New York, 1954. (V12)Myers, R. H. RapponseSdceMAilyn &Bacon, Inc.: Boston, 1971. (V13) Box, G. E. P.; Hunter, W. G.; Hunter, J. S. Statlsbrcs for€x@ments. An Intfcdwtkm to Deta Analysis, and Model 8u.Whg John Wlley & Sons, Inc.: New York, 1978. (V14) Rautela, G. S.; uedtke. R. J. CHn. Chem. 1978, 24, 108-114. (V15) Ezan, E.; Tlberghlen, C.; Dray, F. CHn. Chem. 1991, 37. 226-230. (V16) Danlel. C. J. &ai. Technd. 1975, 7, 103-108. (V17)Feldmann, U.; Schnelder, B.; Kllnkers, H.; Haeckel, R. J. CHn. Chem. CHn. 8bcbtn. 1981, 10, 121-137. (V18) Krouwer, J. S.; Stewart, W. N.; Schlaln, B. Ciin. Chem. 1988,33, 19841986. (Vl9) Schlaln, B.; Krouwer, J. S. CNn. chem.1989, 34, 2118-2120. (V20) Krouwer, J. S. CHn. Chem. 1991, 37, 26-29. (V21)Goldschmldt, H. M. J.; Krouwer, J. S. I n EvekredeMonMethodsinLaboretory hknWne Haeckel, R., Ed.; VCH Verlag: Weinhem, Germany, 1992. (V22) Morrison, D. F. Mulllverlete StablstlcelMthods Waw-Hlll: New York, 1976. (V23) Health Care Financing Admlnlstratkm (42 CFR Part 405, et ai.), the Publlc Health Service, U.S. Deparbnent of Health and Human Services, C//nkal LebOreW Improvement Amendnents of 1088, Final Ru(e; F d . Regist. 1992, 57 (Feb 28), 7003-7288. (V24) Hartmann, A. A h . FaW. Lab. W .1992, 116, 714-717. (V25) Barnett, R. N.; Youden W. J. Am. J. Ciin. PaM. 1970 54 (Suppl.), 454462. Hunt, M. R. Clin. chem.1873, 10, 49-57. (V28) Westgard, J. 0.; (V27) Waakers, P. J. M.; Hekndoom H. B. A.; OpDeWeagh, 0.J.; Heersplnk, W. clin. m.Acta 1 9 7 5 , ~d3-184. , (V28) Conrbleet, P. J.; Shee, M. C. C i h Chem. 1978, 24, 857-861. (V29) Bookbinder, M. J.: Panoslan, K. J. CUn. chem.1987, 33,1170-1176. (V30) Hackney, J. R.; Cembrowski, G. S. Am. J. Cih. Pathd. 1988. 86,391-
393. (V31) Bland, J. M.; Anman, D. G. Lancet 1986, 8, 307-310. deVos, D. J.; Hunt, M. R.; et ai. Am. J. M. Techno/. (V32) Westgard, J. 0.; 1978, 44, 290-300. (V33) Westaard, J. 0.; deVos, D. J.; Hunt, M. R.; et ai. Am. J. M,T e c h d . . 1978, U, 420-430. (V34) Westgard, J. 0.; deVos, D. J.; Hunt, M. R.; et ai. Am. J. M. T & I ~ o / . 1978, 44, 552-57. (V35) Westgard, J. 0.; deVos, D. J.; Hunt, M. R.; et ai. Am. J. W .Tech&. 1978. 44. 727-742. (V36) WestGrd, J. 0.; deVos, D. J.; Hunt, M. R.; et el. Am. J. W .Techno/. 1978. 44, 803-813. (V37) Kennedy, J. W. Recishm pertcwmence of clinical chendsby devlces: EPS-TZ NatlonalCommittee for Cllnlcal Laboratory Standards: Vlllanova, PA, 1992. (V38) Passey, R. 8. Evakretkmofihell~rttyofquantltettveanalytlcelmebkde: E P W Natlonal Committee for CllnlcalLaboratory Standards: Vlllanova, PA, 1986.
(V39) Powers, 0. W. Interference testhgin clinicaichernkiby: .€We, Natlonai Committee for Cllnlcal Laboratory Standards Vlllanova, PA, 1988. (V40)Kennedy,J. W. Clscwwq&sonofquantmPilvechkaileboretcwym~ using pMent samples: €PO+? National Committee for Clinlcal Laboratory Standards: Vlllanova, PA, 1985. (V41)Passey, R. B. Preilmineryevakretionofcllnlcel~~bymebhods: €PI& r; Natlonal Committee for CUnlcai Laboratory Standards: Vlllanova, PA, 1989. (V42) BaW, S.; Kennedy, J. W. J. CUn. Lab. Auto. 1981, 1, 128-132. (V43) BaUW. S.; Kennedy, J. W. J. Clin. Lab. Auto. 1981, 1. 197-201. (V44) BaW, S.; Kennedy, J. W. J. CHn. Lab. Auto. 1982, 2, 35-40. (V45) Bauer, S.; Kennedy, J. W. J. Cih. Lab. Auto. 1982. 2, 129-133. ANALYTICAL CHEMISTRY, VOL. 65, NO. 12, JUNE 15, 1993
488R
CLINICAL CHEMISTRY (V46) Bauer, S.; Kennedy, J. W. J. CHn. Lab. Auto. 1982, 2, 209-215. (V47) Baw,S.; Kennedy, J. W. J. CHn. Lab. Auto. 1982, 2, 281-283. (V48) Bauer, S.; Kennedy, J. W. J. CNn. Lab. Auto. 1982, 2. 354-357. (V49) Bauer, S.; Kennedy, J. W. J. Clln. Lab. Auto. 1982, 3, 46-48. (V50) Bauer, S.; Kennedy, J. W. J. Clln. Lab. Auto. 1982, 3, 183-187. (V51) Bauer, S.; Kennedy, J. W. J. Clln. Lab. Auto. 1982. 2, 340-347. (V52) Combleet, P. J.; Gochman, N. Clh. Chem. 1979. 25, 432-438. (V53) Mandel, J. The StaLtlcal Analysls of Experimental Deta; Interscience: New York, 1964; Chapter 12. ('454) Mendel, J. J. OM/. Techno/. 1084, 76, 1-14. (V55)Passlng, H.; Bablok, W. J. Clln. Chem. Clln. M m . 1989.21,709-720. (V58) Passlng, H.; Bablok, W. J. Clln. Chem. CHn. 6bctmm. 1984,22,431-445. (V57)Davls, R. 8.;Thompson, J. E.; Perdue, H. L. C h . Chem. 1978,24,811620. (V58) Godfrey, K. New €nd. J. M.1985, 373, 1450-1456. (V59) Bodtblnder, M. J.; Panoslan, K. J. CNn. Chem. 1986, 32, 1734-1737. (V80) Kroll, M. H.; Ruddel, M.; Blank, D. W.; Elln, R. J. Clln. Chem. 1987, 33, 1121-1123. (V61) Tholen, D. W. Arch. P a m . Lab. Med. 1992, 776, 746-756. (V82) Passey, R. B.; Maluf, K. D. Arch. PaW.Lab. M.1992, 7 76,757-760. (V63) Shlres, G. W. Arch. falhd. Lab. M. 1092, 776, 761-764. (V64) Krouwer, J. S. Arch. Pathol. Lab. M.1992, 776, 726-731. (V65) Westgard, J. 0.; Carey, R. N.; Wold, S. CIVn. Chem. 1974,20, 825-833. (V66) Barnett, R. N. Am. J. Clln. Pathol, 1968, 50, 671-678. (V67)Ebvlt~h F. R., Ed. Procwdhgsofthe 1978 AspenConferenceon Analytlc Goals In Cllnlcal Chemlstry, Calk@ of Amerlcan Pathologists, Skokle, IL, 1877. (V68) Ellon-Gerrltzen, W. E. Am. J. C h . Pathol. 1980, 73, 183-185. (V89) Fraser, C. 0. Ann. Clln. Blochem., 1989, 26, 220-226. (V70) Fraser, C. 0. Ann. Ist. Super. Sanlta 1991. 27. 369-378. (V71) Fraser, C. 0.;Peterson, P. H.; Rlcos, C.; Haeckel, R. Ew. J. Clln. Chem. Clh.Blochem. 1992, 30, 311-317. (V72) Solberg, H. E. CHn. Chem. Acta 1987, 765, 111-118. (V73) Petitclerc, C.; Solberg, H. E. C h . Chim. Acta 1987, 170. S1-S12. lV741 Solbera. H. E.: PetltClerc. C. Clln. CMm. Acta 1088. 777. S1-S12. iv75j so~beri;H. E.'C//n. mim: ~ ~ 1987, t a170, si3-s3i. (V76) Dybkaer, R.; Solberg, H. E. Clln. Chlm. Acta 1987, 770, S33-S42. (V77) Sesse, E. A. How to deffne, detsnnlne. andubrllrs reference htwvals In tho cHnlcal labcratory: C 2 l W National Committee for Cllnlcal Laboratory Standards: Villanova, PA, 1992. (V78) Sew, E. A. Arch. Pathol. Lab. Med. 1992, 176, 710-713. (V79) Muttl, A.; Allnovl, R.; Bergamaschi, E.; Franclnl, T. Scl. Total Envkon. 1992, 720, 7-15. (VBO)Graesbeck, R. Scand. J. Clln. Lab. Invest 1990, 5O(Suppl. 201), 45-53. (V81) Solberg. H. E. Sand. J. Clln. Lab. Invest. 1986, 46(Suppl. 184), 125132. (V82) Herrls, E. K.; Wong, E. T.; Shaw, S. T., Jr. Clln. Chem. 1991, 37, 15801582. (V63) Elveback, L. R.; Gulllier, C. L.; Keatlng, F. R. JAMA, J. Am. M.Assoc. 1979, 271, 69. (V84) Lott, J. A.; Mitchell, L. C.; Moeschberger, M. L.; Sutherland. D. E. Clh. them. 1992, 36, 648-650. (V85) Duca. P. Scl. Total Envlion. 1992, 120, 155-171. (V86) Young, D. S. Arch. Palhd. Lab M.1992, 716, 704-709. (V67) Montalbettl, N. Ann. Ist. Sipsf. Sanlta 1991, 27, 365-368. (WE) Fraser, C. G.; Peake, M. J. CRC Cr/t. Rev. Clln. Lab. Scl. 1980, (June), 59-86. (V88) Uldall, A. Ann. Ist. Super. Sanlta 1091, 27, 411-417 (English). (WO) Westgard. J. 0.; Barry, P. L.; Hunt, M. R. C/h. Chem. 1981,27,483-501. (V91) Levey, S.; Jennlngs, E. R. Am. J. Clln. Pathol. 1950, 20. 1058-1066. (V92)Leveyy,S.; Jennlngs, E. R. Arch. Paw.Lab. Med. 1992, 776, 791-798. (V93) Westgerd, J. 0.; Oroth, T. J. Clln. P a w . 1989, 80, 49-56. (V84) Dowille, P.; Cembrowskl, G. S.;,&rauss, J. F. J. Autom. Chem. 1086, 8, 85-88. (V95) Llnnet, K. Clh.Chem. 1989, 35, 1416-1422. (V96) Wood, R. Clh. Chem. 1990, 36,462-465. (V97) Parvin, C. A. Clln. Chem. 1901, 37. 1720-1724. (V98) Parvin, C. A., Clln. Chem. 1992, 38, 384-369. (V89) Llnnet, K. CHn, Chem. 1989, 35, 284-288. (V100) Koch, D. D.; Oryall, J. J.; Quam, E. F.; et ai. Clln. Chem. 1990, 36, 230-233. (V101) Unnet, K., Eur. J. Clln. Chem. Clln. Blochem. 1991, 29, 417-424. (V102) Smlth, S. J.; Myers, 0. L. C / h Chem. 1901, 37, 341-346. (V103) Tillyer. C. R.; Gobln, P. T.; Ray, A. R.; Rlmanova, H. Ann. Clln. Blochem. 1992, 29, 454-460. (V104) Wllllams, S. M. 6r.Med. J. 1992, 302, 1359-1361.
484R
ANALYTICAL CHEMISTRY, VOL. 65, NO. 12, JUNE 15, 1993
(V105) Box, G. E. P.; Draper, N. EvdUMonery OpCKeMon: A S&dlrtlur/Mslhod for Frocess Improvemsnt, Wlley: New York, 1969. (V106) Hoffman, R. G.; Wald, M. E. Am. J. W. pen. 1965, 43, 134. (V107) Bwnett. R. W.; Westgard, J. 0. Arch. faW. Lab. M. 1992. 776, 777-780. (VlO8) Sunderman, F. W. C h . Chem. 1992, 36, 1205-1208. (VlOS)RejR.;Drake, P., Scand. J. clh.Lab. Invest. 1901,205(Suppl.),47-54. (V 110)Ehrmeyer, S. S.; Laessb,R. H.; Lelnweber, J. E.; Oryall, J. J. clh.Chem. 1990, 36, 1736-1740. (Vlll)La~lg,R.H.;Ehrmeyer,S.S.;Lelnweber,J.E.Arch,Pathol,Lab.M. 1092, 776, 770-776. (Vi 12) Laesslg. R. H.; Ehrmeyer, S. S.; Lelnweber, J. E. Clkr. Chem. 1992, 38, 885-903. (V113) Ross, J. W. Arch. Palhd. Leb. M.1988, 772, 421-434. (V114) Thlenpont, L. M. R.; Steyaert, H. L. C.; De Lwnhwr. A. P. C/ln. Chbn. Acta 1987. 168, 337-346, (V115) CastenadaMendez, K. Clln. Chem. 1992, 38, 615-616. (V116) Jenny, R. W.; Jackson, K. Y. W. Chem. 1992, 38, 486-500. (V117) Erlckson, R.; Drlscoll, C.; Dvorak, L.; et ai. J. Fam. Prect. 1891, 33, 457-461. (V118) Rei, R; Jenny, R. W. clin. Chm, 1992, 38, 1210-1217. (V119) Malvano, R.; Pllloton. E.; Plumb, G.; Tanzl, E.; Slgnorlnl, C. Ann. Ist. Super. Sen& 1991, 27, 437-441 (English). (V120) Gamblno, S. R.; Galen, R. S. EeywnfkmaYty: The Vakrs and Efi7ciency of M d h l abgnosk John Wky: New York, 1975. (V121) ZwM, M. H. Asseavnentof& k a l s e n s / t l v l l y a n d ~ o f ~ ~ tests: W l W Natlonal Committee for CHnlcal Laboratory StandaM Villanova, PA, 1987. (V 122) Cowan, B. D.; Vandermoien, D. I.; Long, C. A,; Whitworth, N. S. Am. J. Obstet. Q y n d . 1992, 766, 1728-1734. (V123) Sanchez. R. L.; Jones, D. C.; Cullen, M. 1. ObstetqmecOr. 1991, 77. 5 10-513. (V124) Albert A. Clin. Chem. 1982, 28, 1113-1 118. (V125) Albert A.; Harrls, E. K. hfuMva&te Interpretatbn ofClh&alLabomrcWy &ta; Marcel Mer: New York, 1987. (V126) Comfleld, J. Fed. Roc., Fed. Am. Soc. Exp. W. 1962, 21, 56-61. (V127) Press, S. J.; Wilson, S. J. Am. Stat. Assoc. 1078, 73, 698, (V128) Wald, N. J.; Cuckle, H. S.; et al Br. Med. J. 1988, 207, 883. (V129) Arch. Pathol. Lab. M.1990, B (compkte issue). (V130) Jwan, J. M. Jwan h Pwannhg for The F r w Press, Mecmlllan: New York, 1988. (V131)Westgsrd, J. 0.P w a n n i n g Q u a Y t y ~ P r o c e d v s s l o r A n a l ) a r c e l T ~ Recesses in HealthcamLabomtodm WestQnrd OgunqUn, ME, 1992. (V132) Berwlck, D. M.; Godfrey, A. B.; Roessner, J. C u h g Wllh&m: hbw StrateoJes for QuaYty Inqprovement Josey-Bass: San Franckco, 1990. (V133)bmlng, W. E. Out of fhe CWs& M.I.T. Center for AdvancedEnginedng Study: Cambrldge, MA, 1986. (V134) Stevens, 0. H. The StrategkHeelthCarehfanagw: hfastedngEssentIa1 Lea&mh/p Skl& Josey-Bass: San Francleoo, 1991. (V135) Umiker, W. The Customer CMentedLabomBboretory,ASCP Press: Chlcago, 1991. (V136)Gamblno. S. R.; Malbn, R. P. TotalQuabYyhfannpmenthMwllhCa~ Juran Impro Conference, Atlanta, GA, 1891. (V137)Omachonu, V. K. TotalQuaYtyandProducbMIyhfanepmenthMwllh Care C%gan&atbns; Inst. Indust. Eng. and Am. Soc. Qual. control:Mkaukee, WI, 1891. (V138) Francis, D. P.;Peddecwd, M.; Fewran, K. L.; et al. CM. Lab. hfan. Rev. 1992, 6. 537. J 75(Speclal Issue). (V139) Quallty in Health Care. Qual. F ~ 1992, (V14O)Lwbov. W.;Ersoz, C. J. The~llhCerehfanap'sGukbtoConmu~us QuaYty Improvement, Amerlcan Hospital Aasoclatkn: Chicago, IL, 1991. (V141) Gamblno, S. R.; Malbn, R. P.; Woobow, 0. Arch. Palhd. Lab. M. 1990, 174, 1145-1148. (V142)Camp,R.C. Benctnnadrlng Thesaerchforlndrwbykwtpractbsthet bad to supenbrperfonnenos;Quallty Press: MUwaukw, WI, 1969. (V143)Blnns,G. S.;Early, J.F.Haspltelcamhontkrshmenn~quelltyJ~uan Report, No. 10, Juran Institute, Wilton, CT, 1889. (V144) Witson, P.F.; Dell, L. D.; Andsrson. G. F. ffoot Cause A n e w : A Tool for Total OUeUty hfanagemsnt; QuaUty Pr-: Mlkaukee. WI, 1983. (V145)Sloan,M.D.;Chmel, M. TheQuaYtyRevDkrtknandHaellhCereQuallty Press: Mllwaukee, WI, 1991. (V146) Gamblno, S. R. Lab Rsport Nemrletter; G. 8 R. Publlurtkns: Boston, 1891; Vol. 13(4). (V 147) Womack. J. P.; Jones, D. T.; R m , D. The hfachlne thet Changed tho W W , Macmlllan: New York. 1990.
Quam
ac:
