V O L U M E 19, NO. 1 2
950 The last t\\o columns of Table V I shon the values calculated from these two equations. The observed values now agree so closely with the computed values that s estimated from their differences is entirely consistent with Sd computed from the duplicates. This is also evident from an inspection of Figure 1. The fitting of empirical curves for which no physical interpretation can be made is in general avoided. The empirical curve does serve a purpose in this instance. The literature on the antimony electrode is to the effect that a constant voltage will be found between an antimony and hydrogen electrode in solutions up to pH 8.0. The constant is reported to change at pH 8.0. Yet using data up to only pH 6.80 it has been possible to fit an empirical curve which predicts in an astonishing manner the value found a t pH 8.00. The observed value is 171.5 and the predicted 170.9. I t is an almost inevitable conclusion that the precision of this piece of work has revealed a real departure from linearity over the whole iange of pH. For all practical purposes such as measuring the pH of a solution t o 0.01 pH, the deviations are unimportant. From the vieyyoint of investigating the behavior of this electrode, the statistical examination prompts a number of remarks. The equation predicts a value of 74.2 for pH 1.40. I f this value were found there would bc no question about the general form of the curve. The constant is at a minimum for pH 4.01. I t would be extremely interesting to ascertain whether this minimum is associated with the electrode or the buffer solution, by trying some other buffer. The measurements were made at 25” C. and it would not be difficult to determine whether the location of the minimum is independent of the temperature. There is a question here for the mathematical statisticians: Based upon these data, what are the confidence limits for the location of this minimum on the pH scale? All the above examples make use of the same basic technique of fitting a straight line by least squares and computing standard.deviations for the constants in the linear equation. The essential advantage in this technique is the use that is made of the sample weights, the knovm compositions, or pH values as correlated information. Far too often attention is directed only to the actual observations. I n the last example the several values obtained €or the voltage could be replaced by an average and a standard deviation computed from the individual values. By making use of the as-
sociated pE1 variable additional information n as obtained. The examples have been chosen to exemplify various possible situations which may be encountered. The actual number of measurements ranged from ten to twenty for all the lines fitted and yet these limited numbers were sufficient to carry the interpretation of the data beyond that found in the published reports from which the data were taken. ilpart from the normal errors of manipulation, observations are subject to various errors that may be inherent in the chemical procedure. These errors may be of the so-called constant type, such as the error arising through impure reagents or a displaced end point in a titration. These are often combated by running a blank on the reagents. If the range of samples taken permits fitting a straight line, a constant error, if present, will be revealed by making use of the slope of the line. If the error is directly proportional to the magnitude of the sample, it will not be revealed by this technique. If the error varies in some manner not proportional to the size of the sample, it may be possible to demonstrate its presence. The work with the antimony electrode is a good example of this. In this case the errors do not arise from deficiencies in the electromotive measurements but result when the electrode potential itself departs from a simple linear relationship. There are cases where such departures are of immense significance. Statistical techniques provide a nice measure as to whether apparent departures need be taken seriously. LITERATURE CITED
(1) Baxter, G. P., Guichard, M., Henigschmid, O., and WhytlawGrav. R.. J . Am. Chem. Soc.. 63.845-50 (1941). (2) Fishery R. A., “Statistical Methods for Research Workers,” 7th ed., London, Oliver 83 Boyd, 1938. (3) Fisher, R. A,, and Yates, Frank, “Statistical Tables for Biological, Agricultural and Medical Research,” 2nd ed., London, Oliver & Boyd, 1943. (4) Hazel, W. M., and Eglof, W.K., IKD. EXG.CHEM.,AsAL.-ED., 18. 759-60 (1946). (5) Hovorka, Frank, and Chapman, G. H., J . .4m. C h e h . Soc., 63, 955-7 (1941). (6) Oesting, R. B., and Kaufman, I. P., IKD. ENG.CHEM.,r i ~ a ~ . E ~ . , - l 7125 , (1946). (7) Snedecor, G. W., “Statistical Methods iZpplied to Experiments in Agriculture and Biology,” 4th ed., dmes, Iowa, Iowa State College Press, 1946. .
I
RECEIVED September 6 , 1 9 4 7 .
Design of Experiments in Industrial Research HUGH M. SMALLWOOD, U.S. Rubber Company, Passaic, N . J .
T
HIS discussion concerns the place of a relatively unfamiliar
technique, design of experiments, in a field which is familiar t o most chemists-industrial research. Academic research is characterized by the ability to refine the experiment in order to emphasize known causes of variability. Unknown, or uninteresting, causes of variability can be removed by improvement of apparatus, isolation of the experiment from disturbing influences, and other methods. This exhaustive elimination of unwanted disturbances enables concentration on the selected topic of interest to the exclusion of all other considerations. Modern industries support a sizable amount of research of an academic nature, but, in addition, they require work in fields in which undesirable sources of variability cannot be eliminated. These fields of industrial research fall into two clasaes. On the one hand, industry frequently requires laboratory work with poorly defined tests. Determinations of fatigue life, adhesive-
ness, resistance to accelerated aging, processability in the factory, etc., are important in numerous fields of industrial research. Tests of this type cannot always be refined t o such an extent ?hat the desired differences are clearly distinguishable from experimental error. This inability is, no doubt, a consequence of our imperfect knowledge of the factors underlying such tests but, regardless of its cause, it makes the conclusions of the industrial chemist insecure and troubles him about the best way of setting up his experiments. Another field of industrial research is occupied by factoryscale or pilot-plant euperiments. An attempt to refine these experiments might result in eliminating the most interesting and important features of the problems. Vagaries of operating personnel must sometimes be included, in order to make the final conclusions realistic. Even though the measurements to be made on the final product may be.entirely precise, complications of
D E C E M B E R 1947
951
interpretation are tremendously multiplied in going from the test tube to the factory. These features of industrial research are just those which require careful design of experiments. For many years chemists have been inclined to look down their collective nose a t the biologist and the agriculturist. Since research in these lines is particularly aggravated by the type of difficulty described above, biologists and agriculturists have become accustomed to the use of new methods in planning their experiments. These methods, for the most part, were originated bv the British mathematician, R. A. Fisher, and his co-workers. The chemist is now in a position where he must venture beyond the safe fields of refined experiment and enter upon problems that demand the utmost skill in designing and interpreting the appropriate experiments. Under these circumstances, it is best for him to swallow hi> pride and to look for guidance in his present difficulties to fields in which these methods have been provrd.
Table I.
Block Experiment 1
Pup Pup Pup Pup
I I1 111 IV
B”C
D
Litter 2 A
B C
D
3 A
B
C
D
What is meant by the “design of experiments’“! Simply the planning of a group of experiments so that their combined result will yield a maximum amount of information for the work which has been expended. I t is important that this planning take place before the experiments are carried out. In addition, i t is necessary to consider the analysis which will be applied to the data after they have been obtained, so that there is reasonable certainty of drawing valid conclusions. A number of standard designs have been invented, one or more of which are capable of handling almost any problem, BLOCK EXPERIMENT
The simplest type of experimental design is called the “block experiment.” I t s object is to eliminate, as far as possible, variability of the experimental medium. For example, in rubber technology it is n-ell recognized that the physical properties of a test piece depend on the detailed procedure by which the test piece was mixed and cured. I n order to obtain as great uniformity as possible, i t has been customary to specify that comparisons be made only among test pieces mixed by the same man on the same day. This is a very rough and inefficient way of eliminating variability of the experimental medium. The problem is more efficiently handled if i t is recognized that items of the experiment can be arranged in groups, either in space or in time, so that the variation of the medium will be small within the groups in comparison with the variation expected between groups. (In agriculture, these groups are called blocks. This is the origin of the name of this type of experiment.) Suppose a nutrition chemist is attempting to compound a ratio for a particular breed of dog, and wants to determine which yf four experimental formulas will best nourish growing pups. Differences in rate of growth due to heredity may be as large as or larger than differences due to diet. I t is, however, generally admitted that pups from a single litter will be more uniform than pups in general. The block experiment takes advantage of this proposition, a t the same time recognizing that the number of puppies in a litter is not great enough to allow for all the desired replications within a single litter. The experiment would therefore be set up.as follows: three litters of four pups each are taken for the experiment; diet A is fed to one pup of each litter; diet B to one pup of each litter, etc. This is expressed symbolically in Table I. The experimenter records weight at a definite age, rate of increase in weight, or some other suitable measurement of the ef-
fectiveness of each diet for e ~ c l ie\peIiriierital animal The important feature of the experiment is that each diet is represented in each litter. Because of this, the variation between litters can be allowed for in analyzing the experimental results, with a very material increase in accuracy of the experiment. A further advantage of this type of experiment is that the results are more general, since they have been obtained from a more representative sample of the experimental medium than if, in this illustration, it had been possible to obtain the wholcl e\periment from a single litter. In general, the block experiment is advisable wherever some feature of the experiment is unique in the sense that it cannot tie repeated identically in subsequent experiments. It is not always desirable to set up experiments with the amount of repetition shown in Table I. This objection can be avoided by more sophisticated types of design. FACTORIAL E X P E R I M E h T
Of even greater general utility than the block experiment is the design known as the “factorial experiment.” This design is generally applicable to any experiment R here we wish to observe the effects of more than one independent variable. Briefly, it involves selecting a number of independent variables to be. studied and the number of values to be assigned to each of these variables, and carrying out a complete experiment for each combination of the different values of all the variables.
For example, the yield of a certain chemical is to be studied 111 the pilot plant. Preliminary experiments on a laboratory scale have shown that best results !Till be obtained when the reaction mixture is diluted with approximately 507, of an inert solvent, when the temperature is maintained at 50” or 60” C., and, further, that either catalyst h or B will give substantial amounts of the desired chemical. If the reaction time is rather long, the investigator will wish to begin his search for optimum conditions with a relatively small number of combinations. Suppose he plans to investigate three concentrations of the inert solvent, say, 40, 50, and SOY0,and only tn-o temperatures, 50“ and 60” C. In addition, he wishes t o compare catalyst A with catalyst B. Elementary science courses teach uniformly that the scientific method involves holding all variables but one constant, and allowing that variable to take a number of values. The observed results will then teach us the effect of that variable. If the investigator has Been trained in this school of thought, he may decide to vary the concentration of the inert solvent a t a single temperature and with a single catalyst. On completing these three experiments he will pick the optimum concentration of inert solvent and study first the effect of high temperature, and then (at a single temperature) the effect of catalyst B. He might obtain the results shown in Table 11.
Table 11. Yield of Chemical Concentration of Inert Solvent 50% 60% Yields Lb. Lb. Lb. 45,l 45.7 44.9
40%
Temperature
= Catalyst I Catalyst B
c.
50 60
50
60
...
45.8
...
...
...
45.7
.
...
... ...
It is fair to assume that the investigator is experienced in this type of experiment. He knows that there is an error of 0.1 or 0.2 pound in weighing the product, and so places little confidence in the third digit recorded in the figures in the table. As a consequence of his five experiments, he will probably conclude that the yields are all substantially the same, so that i t makes little difference what are the actual operating conditions of the process within the range - here covered. This is a very unsafe conclusion to draw from the meager
952
V O L U M E 19, NO. 1 2
data shown. This type of experiment calls for the factorial design. In this case, use of th- factorial design means that experiments would have been carried out so as to fill in all of the blanks shown in the table. It is entirely possible that the results might be as shown in Table 111.
Table 111. Factorial Experiment Concentration of Inert Solvent 50% 60% Yields Lb. Lb. Lb.
40%
Teniperat itre
c.
Catalyst A Catalyst B
-
50
60 50 60
45.1 44.8 33.0 32.6
45.7 46.8 46.7 46.5
44.9 44.7 53.8 54.2
I t is a t once apparent that, although the amount of inert solvent is unimportant when catalyst A is used, this factor becomes extremely important when catalyst B is used. Results of this experiment clearly indicate the necessity for further investigation before specifying optimum operating conditions. Admittedly, the data of Table I11 are fictitious and have been chosen to illustrate the importance of a complete investigation. It is not infrequent, however, in actual experimental work to obtain unexpected results of just this type. A major advantage of the factorial experiment is that it is our best safeguard against missing important information. A further feature of the data given above must be emphasized. It will be noticed that the effect of temperature is insignificant. The uniformity of results obtained a t the two temperatures gives us confidence in the reliability of the data. More than this, however, this design alloas an estimate of the reliability of the conclusions without carrying out any duplicate experiments, regardless of the presence of an effect due to temperature. Kherever experimentation is costly, this feature is of extremely great value. 9 further point should be noticed. In comparing the effect of high temperature with that of low, the average of the six figures obtained a t 60” C. should be compared with the average of the six figures obtained a t 50” C. This’gives great accuracy to our final conclusions. This method of analysis is justified by the symmetry of the factorial design. To a first approximation, the effects of concentration of inert solvent and the nature of the catalyst cancel out in this comparison, Similar statements can be made for each of the variables involved in the factorial experiment. In consequence, the total information yielded by the factorial experiment per unit of the experiment is very much greater than in any other design. ANALYSIS OF VARIAIYCE
In describing these experiments the fact that detailed analysis This analysis is rather more elaborate than that ordinarily used. Techniques are available for separating the total variability exhibited by a set of data into items associated with each of the sources of the variability. In addition, an analysis of this type furnishes an estimate of the error in the experiment and consequently allows a statistical test of the significance of the conclusions. This type of analysis is called an analysis of variance. A complication arises in many applications of this analysis, whose avoidance requires considerable experience with the experimental medium. I n order that the error estimate be valid, i t must include all pertinent sources of error; otherwise, the replication is incomplete and serious mistakes can be made.
of. the results is contemplated has been mentioned.
This situation is most readily explained by a simple illustration. In the outline of the block experiment, it was pointed out that appreciable errors are involved in mixing the recipes
from which rubber tcst pieces are cured. This fact is not always recognized with sufficient clarity in drawing conclusions from experimental data. Suppose an investigator is studying tensile strength of a rubber vulcanizate as affected by three different accelerators. Preliminary experiments have shown him the proper quantity of each accelerator t o use. He then makes one mix of each accelerator and, according t o customary practice, obtains a range of cures from each mix. The tensile data might look somewhat as shown in Table IV. By inspection, it can be seen that the differences are not large, but it appears that accelerator ’B gives a higher tensile than does C and perhaps also has some advantage over A. This conclusion may well be fallacious. All the cures containing accelerator B were made from a single mix. If any mistakes were made in the preparation of the mix, they may affect all the data obtained from that mix. The apparent replication of this experiment is incomplete, in that any errors due top reparation of the original mix are not replicated. There is no doubt but that the tensile strength of the compound containing accelerator B is greater than that containing accelerator 8,but it has not been proved that this difference is due to the difference in accelerator.
Table IV.
Tensile D a t a A4ccelerators B C Tensile Strength, Lb./Sq. Inch
.i
Cure a t 140’ C. Min .
3900 4100 4000
40 60 80
4300 4200 4300
3700 3900 3600
Under these cii cumstancrs definite conclusions can only be drawn when the mixes are replicated. The appropriate design is shown in Table V. This evtended experiment shows that in fact there is no difference betvieen the accelerators. These data are fictitious and correspond to a very poor level *of control on the mixing procedure. There are, however, many cases of practical importance where incomplete replication can have verv serious consequences.
Table 1’. Yariance Analysis Accelerators B B C Tensile Strength, Lb./Sq. Inch
-1
.1 Cure a t 140’ C. Mtn. 40 60 80
3900 4100 4000
3600 3500 3800
4300 4200 4300
3700 3900 3600
3700 3900 3600
C
4100 4000 3800
It will be noted in Table V that the amount of replication given to answering the question, “Is there any difference between these accelerators?” differs from the amount given to the question, “How does tensile strength depend upon cure?” Under these circumstances, a special type of variance analysis is required in which different error estimates are used, appropriate to the amount of replication given to each question. CONCLUSIONS
An attempt has been made to describe what can be done by the design of experiments, and also some of the circumstances under which experimental design becomes important. Only a few of the features of this subject have been covered. A number of other tools are available and have been found to be of extreme utility. The subject is a large one, but its study is rewarding. Whenever all the information contained in a set of data must be extracted, or whenever results must be +stained as efficiently as possible, the design of experiments can be counted on to bring real benefits to industrial research. RECEIVED September
4 , 1947.
