ADVANTAGES OF STATISTICALLY DESIGNED EXPERIMENTS ARE ILLUSTRATED, AND SEVERAL EXAMPLES SHOW THAT IT I S 4 RELATIVELY SIMPLE MATTER, WITH FACTORIAL AND FRACTIONAL FACTORIAL DESIGNS, TO DETEFMNE THE EFFECTS OF SEPARATE VARIABLES EVEN THOUGH MANY VARIABLES ARE CHANGED AT THE SAME TIME.
W. G. HUNTER AND M. E. HOFF
PLANNING EXPERIMENTS TO RESEARCH EFFICIENCY he purpose of experimentation is to obtain informaT t i o n . Research efficiency might be defined as the amount of useful information obtained per unit cost. Statistically designed experiments are one means available to individual experimenters and research laboratories to increase research efficiency. This paper illustrates the point with some real examples, showing the basic simplicity and relative efficiency of a statistical approach to planning experiments. It is not necessary to be a statistician or mathematician to use these ideas; experience has shown that chemists and engineers can easily learn the fundamental principles of experimental design. Some small, simple experimental plans will be illustrated--e.g., a design for studying u p to seven variables in only eight runs. By using these experimental plans it has been possible, for instance, to discover variables that are important even though, a t the outset of experimentation, those same variables were thought to be unimportant. If these efficient statistical designs had not been used, these variables might have remained unexamined and hence undiscovered because of a lack of time and money. Statisticians, beginning with Si Ronald Fisher in the 1920’s, have been developing various experimental plans for different situations in agriculture and biology, as well as in chemistry and chemical engineering. This paper is concerned with experimental designs found valuable, particularly in industrial research and development laboratories in the latter two fields.
There are several advantages of statistically designed programs: Many variables may be studied a t one time making it possible to gain a n insight into their simultaneous effect on responses of interest. Interactions between variables can be determined. This is impossible if the more usual one-variable-at-atime procedure is used where, in turn, each variable is changed with all the remaining variables being held constant. Notice especially Example 2 in this regard. Useful directions of experimentation are often indicated which can be explored. See, in particular, Example 4. The experimeter can proceed sequentially, constantly adapting his experimentation in the light of the most current information, including that which he himself has just collected. For this purpose fractional factorial designs act as flexible building blocks and have, in fact, been successfully employed a t all stages of experimentation from start to finish. I n particular, factorial designs and fractional factorial designs are useful even a t the very outset of experimentation when very little may be known about the system. Designed experimental programs are efficient when the purpose is to screen variables-to select out of a large set of variables those that are most important (2,8). Because of the careful balance and special patterning present in statistically designed experiments, interpretation of the results is often easy. Note that the first three examples require no mathematical analysis. VOL 5 9
NO. 3
MARCH 1967
43
Figure 7.
EXAMPLES
Effects of three variables on a castjilm (Example 7)
I
I
I
The examples that follow illustrate the use of sound design principles. No complicated mathematics is involved in the analysis of the data in these examples. I n fact, just looking carefully a t the data themselves yields considerable information in many instances. All examples represent actual experimental programs.
I
Example 1.
-t -
+ (1) Ernulrifer A
IMPORTANCE OF EXPERIMENTAL DESIGN Two broad areas of statistics are the design of experiments and the analysis of data: Design is concerned with how experiments are planned and analysis is concerned with methods of extracting all relevant information from the data once they are collected. Of the two, design is undoubtedly of greater importance. The damage of poor design is irreparable; no matter how ingenious the analysis, little information can be salvaged from poorly planned experiments. On the other hand, if the design is sound, then even quick methods of analysis can yield a great deal of information. In fact, sometimes no mathematical analysis of the data is necessary to extract virtually all the information of value. As mentioned above, with factorial designs and fractional factorial designs, because of the special patterning that exists, it is often possible to do this merely by looking a t the data as they have been collected-Le., in many cases, a visual examination of the data suffices. But even if a mathematical analysis is required, the point remains: Of the two areas of design and analysis, the more important one is design. As a way of appreciating this fact, the reader is encouraged to ask himself for each of the examples in this paper: What would have happened if a statistically designed experiment had not been employed? Would it have been possible without a design to have obtained so much information with the same amount of work? TABLE
I.
EXAMPLE 1-THE EFFECTS O F THREE VARIABLES O N CAST F I L M S
A 23Factorial Design, Single Response
Table I shows the results of a study on the effect of three variables in a polymer latex-the amount of eniulsifier A , the amount of emulsifier B , and the catalyst concentration-on the properties of cast films. Eight polymer formulations were prepared following the 23 factorial design presented in Table I-e.g., the sixth formulation listed consisted of 3y0 emulsifier A , no emsulfier B , and lY0 catalyst. In the notation 23, 3 is the number of variables, 2 is the number of levels used for each variable, and 23 = 8 is the number of formulations prepared. The order of preparation of these eight formulations was determined in this example, as well as all the others, by randomization. Each formulation was used to cast a film on a microscope slide. Four of these films dried clear and four dried cloudy-e.g., the film from the sixth formulation was clear. The data (as presented either in Table I or, from a geometrical point of view, in Figure 1) suggest that variable 2, the amount of emulsifier B, is the variable that most strongly influences cloudiness. The interpretation of the data in this example is straightforward because, for one reason, only one response was observed. In practice it is more common, however, that a number of responses are observed. Is there a clear interpretation of data in these circumstances? Let us now look a t the next example which involves three responses. Example 2.
A Z 3 Factorial Design, Three Responses
Eight polymer solutions were prepared once again following a 23 factorial design but this time the three variables were the amount of reactive monomer, the type of chain length regulator, and the amount of chain length regulator. The results are shown in Table 11. TABLE II. EXAMPLE 2-THE EFFECTS O F THREE VARIABLES ON POLYMER SOLUTIONS
Variables Run
a
44
1'
Variable identification: Variables 1. Emulsifier A 2. Emulsifier B 3. Catalyst
25
3'
2% 0% 1/2%
Observations
Run
Clear Clear Cloudy Cloudy Clear Clear Cloudy Cloudy
+ 3% 2% 1%
INDUSTRIAL AND E N G I N E E R I N G CHEMISTRY
7"
2a
3'
Milky?
1 2
+
-
-
3 4 5
+ + +
-
Yes No Yes No Yes
6
7 8
-
-
-
+
+ -
+ +
-
+ + + +
No Yes No
Variable identification: Variables 1. Amount of reactive monomer 2. Type of chain length regulator 3. Amount of chain length regulator
Observations Viscous? Yellow? Yes Yes Yes Yes No No No No
a
No No No Slightly No No No Slightly
+
'
%$3i%
1%
3%
The sixth polymer solution listed consisted of 30% reactive monomer and 3y0 of type-A chain length regulator; this polymer solution was clear (not milky) and neither viscous nor yellow. -
TABLE
111.
EXAMPLE 3-THE EFFECTS O F VARIABLES O N CAST F I L M S
FIVE
k
- - - + + - - + + - + - - + + + - - - - + - + + - + - - + + + -
1
No No No No Yes Yes Yes + + + + - I -Yes
2 3 4
5 6 7 8
No Yes NO Yes No Yes No Yes
Variable identification: Variables 1. Catalyst 2. Additive 3 . Emulsifier A 4. Emulsifier B 5. Emulsifier C
a
TABLE
IV.
Run
Yes Yes No No NO No Yes Yes
No Yes Yes
Slightly Slightly No
Yes Yes No
NO
h-o
NO
Yes No No Yes
No No
Slightly Slightly
+, %
%
-3
Slightly No Yes Yes
+
l'/z
1 l/4
3 2 2
2 1 1
l/2
EXAMPLE 4-THE EFFECTS O F FOUR VARIABLES O N S T A B I L I T Y
2"
1"
3"
Stability
40
11
+ +-
+ + + +
+ + a
Variable identification: Variables
V.
5 16 = worst
+
-
20 % 2yo 150 C.
30%
1. Acid 2. Catalyst 3. Temperature 4. Monomer TABLE
16 8 12 6 15 11 = bat
+ + +-
-
-
',"I.
100 c. 25 %
50%
EXAMPLE 5-A DESIGN FOR VARIABLES I N E I G H T RUNS
Run
10
2s
3a
40
5a
6a
1
-
-
-
+-
+ +-
+ + + +
++
+ +++
+ +
2 3 4
5
6 7
8
+++ +
-
+ ++ +
-
Variable identiJcation: Variables 1. Type I monomer 2. Type I1 monomer 3. Ratio of Type I to Type I1 4. Anionic surfactant 5. Nonionic surfactant Y 6. Nonionic surfactant Z 7 . Catalyst a
Example 3.
5
-
-
-
-
+ A
SEVEN
7a
-
+ +-
-
+-
+
+ B
c
D
40-60 2% 0% 0%
60-40
'/2%
Figure 2 shows the eight polymer solutions in bottles numbered from 1 to 8 corresponding to the run numbers listed in Table 11. The analysis of the data consisted mainly in this case of examining these bottles. The most obvious result as seen in Figure 2 is that there is a regular alternation between milky and clear solutions. This phenomenon is directly correlated with variable 1, the amount of reactive monomer present. So, what this cursory analysis tells us is that variable 1 seems to control whether the polymer solution is milky or not. By examining the bottles themselves the additional results given in Table I1 were observed. Just as variable 1, the amount of reactive monomer, seems to control whether the polymer solution is milky or not, so variable 3, the amount of chain length regulator, seems to control viscosity. We say this because there is perfect correlation between this particular variable and viscosity. Both of these conclusions are consistent with elementary chemical considerations. The yellowness apparently results from an interaction between variables 1 and 2 ; that is, neither variable by itself produces the yellow color, but when they are both at their levels (that is, 30y0 reactive monomer used with type-B chain length regulator), this effect is produced. If the variables had been studied in the more customary one-variable-at-atime fashion, this interaction may have remained undetected.
4% 3%
3% 1%
A 25-2 Fractional Factorial Design
This example is slightly more complicated than the previous two since more variables and more responses are involved. However, the interpretation of the results is basically the same. Five variables were involved -the catalyst concentration, the amount of a certain additive, and the amounts of three emulsifiers A , B, and C. Eight polymer solutions were prepared, each was spread as a film on a microscopic slide, and the properties of the films were recorded after they dried. The experimental design used here was a 25-2 fractional factorial design. The results are shown in Table 111. The notation 2k-* indicates that there are k variables, each a t 2 levels, and that there are 2"-P runs. Therefore, in this example with a 25-2 fractional factorial design, five variables are studied in 25-2 = Z3 = 8 runs, each variable being studied at two levels. Surprisingly, many conclusions can be drawn from these results by a mere visual inspection even though only eight runs have been performed and five factors have been varied. The important variable with respect to haziness is variable 3, emulsifier A . The important variable with respect to adhesion is variable 1, catalyst concentration. The important variables with respect to the remaining responses of grease on top of the film, grease under the film, dullness of the film when the pH has been adjusted, and dullness of the film when the original pH is used are variables 4, 5, 4, and 4, respectively. These are the most obvious conclusions that could be drawn a t this stage although alternatives are possible. VOL. 5 9
NO. 3 M A R C H 1 9 6 7
45
Figure 2. Effects of three variables on apolymer solution (Example 21, run numbers shown on bottles
8,ll
116,lS
I
+
-
(I) Acid Figure 3. Effects of the amount of acid and catalyst on stability (Example 4)
These three examples illustrate the point that, if experiments are carefully planned, a great deal of information can often be obtained even though no elaborate mathematical analysis is performed on the results. The converse of this statement is not true, namely that it is possible to obtain a great deal of information if the experiments were not carefully planned but an extensive and careful mathematical analysis were performed on the results. I n the three examples, because of the balanced systematic experimental plans that were employed, visual inspection of the data yielded much valuable information. For reasons such as these, then, we can say that design is more important than analysis. Example 4.
A 24-1 Fractional Factorial Design
As part of a comprehensive investigation, four factors were varied in an attempt to obtain a formulation with satisfactory stability characteristics. The four factors were the amount of acid, the concentration of the catalyst, the temperature, and the amount of monoiner. Many experiments, not statistically designed, had been run on this system without satisfactory results. I t was decided to try the Z4-' fractional factorial design shown in Table IV. These eight runs were made and the stability for each formulation was observed, as recorded 46
INDUSTRIAL A N D ENGINEERING CHEMISTRY
in Table I V . Low stability figures are better than high ones. A simple mathematical analysis of these figures [see (2) for details of this calculation procedure] suggests that only two of these four variables are important, namely, variables 1 and 2, the acid concentration and the catalyst concentration, respectively. The stability figures can therefore be displayed as shown in Figure 3 to examine this tentative conclusion from a somewhat different viewpoint. This tentative conclusion appears reasonable since the pairs of stability results (16, 15), (8, l l ) , (11, 12), and ( 5 , 6) associated with the four distinct combinations of variables 1 and 2 can be regarded as duplicate runs differing from one another only because of experimental error. I n other words, we are saying that variables 3 and 4 are dummy variablesthat is, they have no effect on the results over the ranges of the experimental conditions that have been studied. The best way to verify these tentative conclusions is to manipulate the variables 1 and 2 in an attempt to improve stability. (These conclusions must be regarded as tentative since other interpretations, which involve interactions in addition to main effects, are possible.) Recall that we desire low stability readings. Examination of Figure 3 indicates that it might be profitable to explore in a southeasterly direction-Le., to the lower right. Thus, in terms of the variables themselves, what was done was the following: Variables 3 and 4 were maintained a t fixed levels, while variable 1 was increased and variable 2 was simultaneously decreased. A few exploratory runs were performed in that direction, and, for the first time since the beginning of the investigation some months previously, product formulations with good stability were obtained. This example illustrates one of the values of statistically designed experiments-that directions to pursue in future experimentation are often clearly indicated. Faster progress can sometimes be achieved if the direction of steepest descent (or ascent) is calculated. See (5) for details. Example 5.
A 27-4Fractional Factorial Design
One useful class of designs consists of the fully saturated two-level fractional factorial designs, an example of which is a 2Yd4 fractional factorial design shown in Table V. The sixth run, for example, consisted of using the Type I monomer B and the Type I1 monomer C in a ratio of 60/40 with 2y0 of the anionic surfactant, 3y0 of the nonionic surfactant Y , no nonionic surfactant 2,and 1 / 2 7 0 of catalyst. Notice that seven variables are studied in only eight runs. This type of design has found repeated use a t S. C. Johnson & Son, Inc. A fully saturated two-level fractional factorial design is one that permits - 1 variables to be studied in only AV runs. Such designs, incidentally, do not exist for all values of -V. The experimenter has to exercise some care in analyzing data from such designs because interactions are confounded with main effects. But interactions are confounded with main effects even when unsaturated fractional factorial designs are employed so that there will always be some ambiguity in the results
~~
TABLE VI.
DESIGN
Run
la
2"
35
4"
1
-
-
++++
-
-
++-
2 3 4
5 6 7 8 a
EXAMPLE 6-FIRST
+ +-
+ +
Variable identi'catlon: Variables 1, Catalyst (0.47,) 2. Reducing agent (0.4Yo) 3. Emulsifier (1YC) 4. Emulsifier (17 0 ) 5. Emulsifier 2
-
-
+ + + +
-
++ -
TABLE V I I .
5" -
+
++-
+ +
70
-
Trace
96
-
None Trace Pione
97 95 98
2"
1
-
++
4
+
+
Variable identifrcation:
+
-
Variables 1. Catalyst 2 . Temperature
I/:%
3/a%
45 c .
55O
A L V
B M W
X
Y
Run
I"
2"
Coagulum
0%
2%
1 2 3 4
-
-
None None None None
from fractional factorial designs. Confounding will be more extensive, however, with a fully saturated design and, consequently, more care must be exercised in interpreting results. In either case, additional experiments can usually be devised to remove any serious ambiguities that might exist. I t should perhaps be pointed out that even with a full factorial design, or any design for that matter, there will always be some uncertainty in the interpretation of the results. One source of uncertainty, of course, is experimental error. Some knowledge of the experimental error is necessary if sound conclusions are to be drawn. The investigator will often have some idea of the magnitude of the experimental error, as we are tacitly assuming in all these examples. Otherwise it must be estimated from replicate runs. Example 6.
Conuersion, 1"
3
The Sequential Use of Designs
I n actual practice experimental investigations rarely consist of only a single experimental design but rather are usually composed of a number of stages of experimentation. After each stage the experimenter has the opportunity of analyzing all the available data and, in the light of the tentative conclusions he reaches and partial insights he gains, he can then plan a further set of runs. Such a flexible attack on research problems has obvious advantages over a more rigid one in which the complete program of runs that is set up a t the outset of experimentation is exactly followed with no modifications or adaptations of strategy being made along the way. As information is collected, more fruitful avenues of approach often suggest themselves. The experimeter should be constantly alert to these ideas and should be ready, in particular, to sacrifice finishing a complete design if the experimental effort that would be needed to complete the design could be better spent in exploring some promising new ideas that have come to light since experimentation started. AUTHORS W . G. Hunter is an Associate Professor in the Department of Statistics and the Engineering Experiment Station, University of Wisconsin. M . E. Hof is a Chemist for the S. C. Johnson C3 Son, Inc., Racine, Wis. T h i s research was supported in part by the h'SF Grant No, NSF-GP 2755.
DESIGN
Coagulum
Run 2
a
E X A M P L E 6-SECOND
TABLE VIII.
EXAMPLE 6-THIRD
c.
DESIGN
Conversion,
+-
+
+
+
Variable identification: Variables 1. Emulsifier Y 2. Emulsifier (17,)
70 98
99
99 100
-
+
0%
2% W
V
There is a danger, of course, of abandoning designs too soon because sometimes, even if one would like to depart from the design to follow up an interesting hunch, it will be best to complete the design first. .4s with so many aspects of experimental work, judgment and common sense play a major role in deciding the best course of action to take in a particular situation. Let us illustrate these ideas by means of an example. The development of a n emulsion polymer had progressed to the point of having the monomer ratios fairly well determined. There were two major problems yet to be solved: to reduce the coagulum which occurs during polymerization to as low a level as possible and to increase the conversion from monomer to polymer to as high a level as possible. There was another problem in that the existing preparation procedure was complicated; it was hoped to find a simpler procedure. What variables should be examined for this system? Three main categories included the catalyst, the reducing agents, and the emulsifiers. Should catalyst A be used (which presents no problem for production) or catalyst B (which is more difficult to use but believed necessary for this particular latex)? At least two reducing agents ( L and M ) should be studied but the best levels are not known. As far as emulsifiers were concerned, since the one used for the development of the latex thus far had become impossible to reorder and buy for production, there were at least six others ( U , V , W , X, Y , and Z ) to be screened. In addition to the three main categories just mentioned, of course, there were the monomer ratio, the theoretical nonvolatile, the method of preparation, and others. Thus, there was an extremely large number of variables of possible importance. I n fact, there were far too many to be handled in only eight runs-the size of the experimental design desired, based on various considerations. Therefore, after careful thought, some of the VOL. 5 9
NO. 3
MARCH 1967
47
variables were selected to be held constant in the first design. They included monomer ratios, emulsifier Lr, and the method of preparation (a modification of the existing complicated procedure). A 25-2 fractional factorial design as shown in Table VI was set up with the following as variables: catalysts A us. B , reducing agents L us. hl, emulsifiers V LS. W and X us. Y , and emulsifier Z studied at two levels, 0% and 2%. The first four runs were prepared. The coagulum was present a t about equal levels for all these runs. Runs 1 and 2 gave about 97Oj, conversion and 3 to 4 gave only about 57y0. During the preparation it was noted that reducing agent M seemed to start the polymerization very quickly. I t was postulated that 1LI inactivated the catalyst before it had a chance to initiate the polymerization of all the monomer. T o check this, runs 5 through 8 were prepared with the following change in the original plan. When the reducing agent was M (runs 7 and 8) the amount was halved from 0.4y0 to 0.27,. Runs 5 and 6 gave about 97y0 conversion and 7 and 8 were 91y0 and 74Yc, respectively. This indicated that the reducing agent was detrimental to good conversion. Kow the following question arose: Should runs 3 and 4 be prepared with 0.270 of reducing agent M to complete the modified design or had enough been learned so that the experimenter could proceed to a new design? Among the tentative conclusions reached at this stage were the following: There did not seem to be much difference between catalysts A and B. There was no dramatic difference between emulsifiers V and W; there was a slight hunch, however, that W was better. Also, with regard to emulsifiers X a n d Y , there was only a small difference; there was a hunch here that Y was the better of the two. There was an apparent interaction between emulsifier Z and reducing agent M which gave plating on the flask; it was decided, therefore, not to use emulsifier 2 in further formulations. The question also arose as to whether a reducing agent was really needed for this system. I t was noted that run 1 had no creaming and run 6 had the least amount of coagulum. Taking all this information into account together with the fact that the initiation system seemed to have much more effect than the emulsifier, a second design (much simpler than the first) was constructed. I t was decided to hold the following factors constant: monomer ratio, emulsifier L‘ level, emulsifier V level (I%), reducing agent L level, and the method of preparation (a relatively simple procedure). The design constructed was a 22 factorial design involving only two variables, catalyst A level and temperature, as shown in Table V I I . The main result obtained from the second design, as shown in Table VII, was the clear indication that catalyst level was the key variable and it should be increased above 3 / 4 7 0 , the upper limit employed in the second design, to minimize the coagulum and maximize the conversion. Now it was possible to go back and check the hunches from the first design by means of a third design. The constants in this design were chosen to be the monomer ratio, the emulsifier U level, the catalyst A 48
I N D U S T R I A L A N D E N G I N E E R I N G CHEMISTRY
level (lajO),a reaction temperature of 50’ C. (a compromise between 45’ and 55’ C.), and the method of preparation (the simple procedure again). The variables were the emulsifier Y at two levels (0% and 2%) plus the emulsifiers V and W each at a single level (l’%), and the design employed once again was a 22 factorial. The results, which are shown in Table VIII, clearly show that the problem had now been solved. The two major objectives had been accomplished with the help of experimental designs, conditions having been established which gave no coagulum and total theoretical conversion. In addition, it was found that this could be achieved using a method of preparation much simpler than the original complicated one. In less than a week’s time the chemist had proceeded from a state of considerable ignorance about the system (not knowing which factors were important or how they affected the responses of interest) to having the difficulties well under control.
D I SCUSSION The preceding examples illustrate several advantages of statistically designed experiments. Unfortunately, many experimenters still resort to a one-variable-at-atime procedure because they think, incorrectly, that is the only approach that will enable them to determine clearly the effect of each separate variable. In fact, of course, there are serious disadvantages associated with this approach-e.g., any interactions that exist between variables will go undetected; for a particular variable, a main effect that is evaluated is valid only for one fixed set of conditions for all the other variables. Statistically designed experiments overcome these and other disadvantages. With factorial designs, for example, interactions can be evaluated quantitatively; all main effects are valid for the entire experimental region-results, once obtained, are more generally applicable. Furthermore, as the above examples illustrate, it is a relatively simple matter with factorial and fractional factorial designs to determine the effects of separate variables even though many variables are changed at the same time-even though a “many-variables-at-a-time” procedure is actually employed. The paper by G. E. P. Box and J. S. Hunter (2) about fractional factorial designs has proved itself to be a most valuable practical reference on this topic. Going beyond fractional factorial designs, readers might also be interested in learning about other related statistical techniques that have likewise proved useful to chemists and chemical engineers-response surface methodology ( 4 , 6 ) , evolutionary operation (7, 7), and mechanistic modeling (3)-although they are not discussed further here. LITERATURE C I T E D (1) B
~ G .~E. ,P., ,ippi. stotiJt. 6, a i (1957).
( 2 ) Box, G . E. P., Hunter, J. S., Technornrtrtcs 3, 331, 449 (1961).
(3) Box, G. E. P., Hunter, W. G., Ibid., 7, 23 (1965). (4) Box, G. E. P., \Vilson, K. B., J . Roy. Statist. SOC. B13, 1 (1951). (5) Davirs, 0. L.. Ed,, “ T h e Design and Analysis of Industrial Experiments,” Chap. 11, Hafner, New York, 1954. (6) Hill, 7.V. J., Hunter, W. G., Technornefrics 8, 571 (1966). (7) Hunter, \V. G., Kirtrell, J. R., Ibid., p. 389. (8) Stowe, R. A,, biayer, R. P., IND. END. CHEY.58, 36 (1966).
