Foundations for Experimental Design JOHN C. MINTERMAIER1, Vanity Fair Milk, Reading, Pa. The wnclusion. drawn from analytiwl work are frequently subject to doubt unleu the analyst has controlled conditions, adequate replication, and a properly desiwned experiment. The maintenance of oontrolled condition. is dimurmed frnm the standpoints of plant, equipment, reagent., procedure, and peraonnel. A cornparimon of the clnsmicrl method of investigating an analytical procedure with a factorially derigned method rhows that more pertinent information can hr,secured with only a amaU increase in effort.
A
(’HEMIST’S recommendation6 are no more reliable than the data he observes. Although he contributea personal skill, imagination, and scientific knowledge, the validity of his cwnclueioae b in large part dependent on his experimental deaign. It is the purpose of this paper to explore the potentialiti- of experimental deaign from the analyst’s standpoint. CONTROLLEDCONDITIONS I N EXPERIMENTAL WORK
Many yecull ago the accepted method for performing Soxhlet extractions on wool called for 2 hours’ treatment with ethyl ether. Extensive disagreement among laboratories on the same standard materiel indicated that variation in the number of times the apparatus siphoned WIN the factor ( d g n a b l e cause) responsible for the trouble. Standardizing this at #) extractions removed tbe major source of vui(which takes about 2 h) ability and permitted a much &mer approch to ‘%ontrolled cqnditions.” For great& efficiency, therefore, each d y s t should strive for controlled C O I I ~ ~ ~ ~ Ofirst, M ; in his &ut deeign, m n d l y , in his operating conditions, and kst, in himself. Under this ideal system, the influenae of all the major factors on the fins1 resulb are in a state of constancy. Their effete have been isolated, perhaps meanued, but at least placed under control so as to Becure optimum results-i.e., reliible data. We achieve controlled conditions when: A etandard previously set under controlled conditions (such as an analytical standard prepared by the Kational Bureau of Standards) is approsched 80 closely a~ to be within the tolerances set for experimental error. A necleeesry qualification should be that the analyst demonstrah that he is able to m e t thia re uirementreptedly. ?n the absence of a sfsLpdlvd, a given homogeneous material is analyzed repeatedly, gmng results 80 close to each other that the satisfy the Shewhart control chart tmt (13). Several obher au&ors (8, @,Id, 16) have shown that this is both poeeibie and practical. MAINTENANCE OF CONTROLLED CONDITIONS
There b a serious problem in maintaining controlled conditions over the long period of time nectxsary for an involved experiment. Special consideration in the design of the experiment should be given to five major subdivisions: plant, equipment, reagents, procedure, and pereonnel. Pknt. Under this heading falb the type of physical and c*hemical testing where atmospheric temperature and humidity af€ectthe material being analyzed. Use of constant temperature room or cabineb has been found absolutely necessary in textile and plastics laboratories. Routine checks on the controlling equipment or d j u a t m e n t ehould be a part of every day’s operations. The within-room distribution should be explored for padble poeketa with normtandad conditions. I
P m n t rddraa, lkadudr Tatin# hbolrtory, Ramrob Bud Dewlap N. Y.
& Mvldm, Cluett, Peabody & Ca,Troy,
1111
Equipment. I t is fairly d e to ehte that for very acLceumte work all analyste d i b r a t o their equipment. h a w the UIIcertainties in calibrating contribute to the over-all method variability, it i e m n t i a l to use well established methods of calibrating, and to specify one method when a n u m b r of malysts are participating. Reagents. Critical examination of alternative methods uf purifying or stsndardixing will often yield information regarding differenm in results which might have been wrongly attributed to the materialsW i g analyxed. Procedures. Strict adherence to the specified procedure is indispeneable to securing comparable reeulta. Personnel. It i s important to be alert to detect differences bt wren operators. REPETITION IN EXPERIMBNTAL WORK
In the simplest experiment we have two methods to compare. It is desired to provide duplicate analyseg. .The experimenter weighs off two’portiom for method. A m d proceeds with the analysis. He then weighs off two portions for method B and secureshis reault. The time, temperature, humidity, and sample condition for method B may very well have slight1 aince carrying out method A. Can he say that the ’fference e observea is due wholly to the difference in methods? A much better rocedure would be to weigh off all four samplea uired in ayternate sequence. He would then perform m e t h a and method B simultaneously on two samples. After Beveral hours had elapsed he would repeat method A and method Ron the remaining two samplea in reverse order.
cb8Y
i
Every analyst accepts the fact that duplicate or triplicate anslysea are required. Moran (10) gives a detailed discuaaion of the d a t i o n between replication and precision. His limit of uncertainty is an application of the statistic called the standud error of averages. (Increasing N, the number of obeervstions in ~ F Javerage, reduces the .standard deviation of the‘average but does not dirninieb the b h or constant error in the method.) Thie reduction in the fluctuations in magnitude of averages around their mean can be made as smell as desired, for
W P TESTS
~
P'P'T';~
ANALYST1 F2
1 ;1 1
F3 P'P'T'N A*
PI
~3
* *
PI
*
~2
~3
m
DRYING AV3
1145 The results are reported in strength per standard number of yarns as measured in the specimens to be broken (to remove A V E ~ E S ~ weaving and shrinkage variations in yarn count per inch.) The control cloth went through the same treatment omitting the active ingredients but including the wetting, drying, heating, or extraction done on the treated sample. The difference in strength levels is greater than could be atF3 tributed to chance as expressed by the remainder of the experimental error. A repetition of the procedure using freshly prepared treating solutions verified the direction and extent of the strength difference.
-
A sketch of the cloth divisions is shown in Figure 1. DRYING P'PT'N
DI PI
P2
D2
P3
PI
FILTRATION
P2
P3
AVERAGES
SIMPLE FACTORIAL EXPERUMENTATION
A m
Let us explore the problem of investigating an analytical method which has been shown to yield different results between analysts. h series of trials is to be planned to reveal the cause of deviation. Assume precipitation, filtration, and drying to be the factors to be studied first. 411 weighings are to be done on the same balance. The "classical" method of investigation requires varying only one factor a t a time, keeping the others as constant as possible. We will perform each analysis in duplicate for checking purposes on a known standard. The first analyst would proceed to investigate the effect of conditions of precipitations. He might try three temperatures for the reaction, PI, Pn, and Pa. The remainder of the procedure -filtration and drying-would follow the method as given. In the meantime, the second analyst would also be studying p r e cipitation a t the same three levels as the first.
FI ANALYST2 F2
R
F3 P?"r" AVS DRYING A*
* pi
it
*
it
m Figure 2.
1
63!
OBTAINED BY SUMMING ALL PI DATA FROM GI
a GZ;
SIMILARLY FOR
W
a
Factorial Design
d$ = Standard error
(1)
where uc is the standard deviation of the data assembled in the average. The calculation of uC is given by Fisher ( 4 )for small amounts of data:
Table I. Probabilities where s - 2 represents the difference between each observation and the average. Brownlee ( 3 ) illustrates various row and column arrangements for securing the greatest amount of information by the use of spe :ially designed replications. The foundations for this type of experimental design xere laid by Gossett ( 7 ) , Fisher ( 4 ) , Snedecor (14), and others. For planning extensive interlaboratory test programs this row and column arrangement may be carried out as shown by the A.S.T.M. ( 1 ) . It is very reassuring to know how much confidence should be placed in a conclusion from a given amount of data. We often wish to know with what probability of being correct we make the statement: "the material contains xqc * EYc of component," having made five analyses whose coefficient of variation Y is 2.2% and E (the sampling error felt to be practical) is 1%. Substituting in t*v2 =-
E2
(3)
t is found to be approximately 1.0. Then from the table of probabilities associated with values of t it can be seen that there is a low probability (2 out of 3) that we are correct in stating that the average of 5 analyses will be within 1% of the true value of the material. PLANNING EXPERIMENTS
For an example of proper design, consider that the effect of a chemical treatment on cloth is to be studied, similar to one described by Hopkins and Weatherburn (8). A length of cloth, more than enough for the actual test program is reserved. Part is retained as control and the remainder is treated. Tensile tests indicate the treated parts to be weaker. Is the treatment really reducing the strength? Such a conclusion is justified only if: Tensile tests are made on a continuation of the same yarns in both treated and adjacent untreated material.
Probability, % 68.3 90 95.0 95.5 99.0 99.7
t
1.00 1.65 1.96 2.00 2.58 3.00
The outcome of the first part of the study would be the choiae of one of the methods of precipitation. Then filtration would be studied, let us assume through three types of filter media giving rise to data F1, Fz,and Ft. The outcome of these analyses, all made with the precipitation instruction above, would vary only in filtering; drying would be carried out as before. Two types of drying are available for comparison. With the improved precipitation and filtration another series for each type of drying, Di and Dz, is analyzed. (The assumption that we now have the best combination of precipitation and filtration may be unwarranted.) As each step has been compared on standard material in duplicate by two analysts, we can now pick out the best combination to reach the analysis of the known standard. To compare precipitation method variability there will be three pairs of observations from each analyst. Filtration could be compared by three pairs, but perhaps only two will be available (the filtration method used first may not happen to be the best). Drying methods might be compared with either one or three pairs, depending again on the outcome of the last part of the experiment. Each analyst has performed 14 analyses resulting in one combination of the three "best" processes. In the factorial design each analyst would carry out all parts of the investigation to fill out a block design as follows: one analysis for each combination of conditions (Figure 2). We can now study the factors simply from the averages of the various rows and columns: 1. There are six observations for each estimate of precipitation procedure for each analyst: PI F I DI; P1 FZ DI;PI FI DI; PI FI D2; Pi FPDz; Pi FI D2.
1146
ANALYTICAL CHEMISTRY
2. There are six observations for each estimate of filtration procedure for each analyst: Fl P I D1; Fl P2 D1;Fl POD i ; Fl Pi D2; Fi Pz D2; Fi PBD2. 3. There are nine observations for each estimate of drying procedure for each analyst: Dl Pi P i ; DlPI P?;DI Pi P B ;D1 FZ PI.. .D2 F3 Pa. 4. There are eighteen observations to compare one analyst with the other.
For the slight additional cost of doing only four more analyses the confidence in the comparisons is greatly increased by reason of using each datum in each comparison instead of just once. However, the absolute relations of the factors are still not free from influencing each other. For example, is a combination of PI and F P better than any of the others? Such interaction effects can be separated from the influences of the other factors through the procedure known as analysis of variance. This is one of the major advantages of using factorial methods in place of the classical method. Procedures for application have been given (4,6, 11, 14). The questions which may be answered are as follows: 1. Do the analysts differ with relation to the precipitation method they are using? 2. Do the analysts differ with relation to the filtering method? 3. Do the analysts differ with relation to the drying method? 4. Is there an association between precipitation method and filtering method? P, may give much better-Le., closer to the standard-results with Fz than any other P. 5. Is there a relation between filtering methods and drying methods? 6. Is there a relation between precipitation and drying? 7. Do certain combinations of analyst and methods differ significantly? 8. Is there one best combination of precipitation, filtration, and drying? 9. Is precipitation a major factor? 10. Is filtration a major factor? 11. Is drying a major factor? Even if the additional calculations needed to answer these questions are not carried out each time such an experimental design is used, the form permits more information to be gained from approximately the same amount of labor and material. The importance of these inbrrelations cannot be overemphasized,
as here in the hidden combinations lies the cause of much disagreement between analysts. CONCLUSIONS
The validity of our conclusions from the results of n-ell-designed experiments rests on two major assumptions: (1) that the material analyzed truly represents the lot from which it was taken, and (2) that enough replications were made to guarantee a high probability that the analysis is correct. No amount of practical experience, intuitive reasoning, or statistical analysis will permit correct conclusions from a badly designed experiment. LITERATURE CITED
(1) American Society for Testing Materials, Philadelphia, Pa.,
“A.S.T.M. Standards for Textile Materials,” 1948. ( 2 ) Bicking, C. A , , Gore, W. L., Gross, M. D., Mitchell, J. A., and
R‘ernimont, G., Industrial Quality Control, “Quality Control in the Chemical Industry,” Vols. I11 and IV, New York, American Soci:ty for Quality Control, 1947. (3) Brownlee, K. A., Industrial Experimentation,” Brooklyn, N. Y . , Chemical Publishing Co., 1947. (4) Fisher, R. A,, “Design of Experiments,” Xew York, Hafner Publishine Co. 1947. (5) Fisher, R. I.,“Statistical Methods for Research Workers,” Edinburgh, Oliver & Boyd, 1937. (6) Freeman, H. A., “Industrial Statistics,” Ken. York, John Wiley & Sons, 1942. (71 Gossett, W. S.. (Student) “Student’s Collected Papers,” Cambridge, University Press, 1942. (8) Hopkins, J. W.,and Weatherburn, M. W.,Can. J . Research, F25, 264-72 (1947). (9) Mitchell, J. A., Ax*L. CHEM.,19, 961 (1947). (10) Moran, R. F., IND.ENG.CHEM.,ANAL.ED.,15, 361 (1943). (11) Peters and Van T’oorhis, “Statistical Procedures and Their Mathematical Bases,” New York, McGraw-Hill Book Co., 1946. (12) Powers, F. W., IND. EXG.CHEM.,ANAL.ED.,11, 660 (1939). (13) Shewhart, W. A,, “Economic Control of Manufactured Product ” New York. D. Van Nostrand Co.. 1931. (14) Snede’cor,-G.-W.,’ “Statistical Methods,” Ames, Iowa, Iowa State College Press, 1940. (15) Wernimont, G., IND.ENG.CHEM.,ANAL.ED., 18, 587 (1946). RECEIVED September 10,1948.
[End of Symposium]
Determination of Polymer in Furfural Vacuum Distillation Method J. C. HILLYER AND A. J. DEUTSCHMAN, JR.’ Phillips Petroleum Company, Research Department, Bartlesville, Okla.
F
URFURAL has come into extensive use in industry as a solvent in extractive distillations, such as the separation of petroleum fractions and vegetable oil fractionation. It is commonly regarded as a stable material, and the decomposition rate of refined furfural is low ( 2 ) . Kevertheless, under the influence of heat and certain process conditions, furfural tends to polymerize slowly. It deteriorates in storage, as indicated by an increase in color and acidity; the change is in part a result of autoxidation to polymeric acids, which are left behind as a residue on diqtillation (3).
The word “polymer” is used rather loosely to describe the various high boiling residues that form in furfural under exposure to elevated temperature, oxygen, or other degradative conditions. Resinous polymers are present, varying from nonvolatile viscous 1
Present address, Spencer Chemical Companv, Pittsburg, Kan.
oils to solid, carbonaceous particles. In addition, materials of boiling points intermediate between the essentially nondistilling tars and the readily volatile furfural are often present. Recently, it has been demonstrated that under the conditions existing in butadiene purification systems, furfural and butadiene interact to form the condensation product 2,3,4,5-bis(A2-buteny1ene)tetrahydrofurfural (4). This material, n?-hichis a high boiling but distillable liquid (boiling point 115’ C. at 1 mm. of mercury), is present in appreciable quantities in furfural samples withdrawn from circulating streams in the plants. Inasmuch as these intermediate products are eliminated along w ith the resinous polymer in the continuous purification of a portion of the circulating stream, they represent furfural decomposed and their inclusion in the polymer determined is therefore desirable. The determination of the polymer content of furfural has as-
