successfully to determine intermediate steady state temperatures. Predicted behavior was found to be in reasonable agreement with experimental results. The initial rate of change of reactor temperature due to disturbances in inlet temperature and/or concentration was faster than the change following a perturbation in flow rate, causing difficulties in bringing the reactor back to its original condition. The closeness of the operating mean residence time to the corresponding critical value was found to be an important factor in this regard. The reactor performance was very sensitive to disturbances near the critical mean residence time. Acknowledgment B. K. Guha wishes to thank Monash University for the awarding of a Monash Graduate Scholarship which enabled him to carry out this investigation. Nomenclature C = concentration, M cp = specific heat, cal/(g)("C) q = flowrate,l./sec R = reaction rate, mol/l. sec Rg = gas constant, cal/(mol)(K) T = temperature, "C t = time, sec V = volume, 1.
x = conversion r = mean residence time, sec p = density, g/l. N = stoichiometric coefficient AH = heat of reaction, cal/mol
Subscript A = hydrogen peroxide B = sodium thiosulfate f = feed j = reaction component 0 = a t the initial condition s = steady state Literature Cited Aris, R., "Introduction to the Analysis of Chemical Reactors," p 156, Prentice-Hall, Englewood Cliffs, N.J.. 1965. Aris, R., Amundson. N. R., Chem. Eng. Sci.. 1, 121 (1958). Bilous, O., Amundson. N . R.,A.I.Ch.E. J., 1, 518 (1955) Furusawa, T., Nisimura, N., J . Chem. Eng. Jap., 1, 180 (1968). Griegel, W. B., Ph.D. Thesis, University of Pennsylvania, 1965. Guha, B. K . , Ph.D. Thesis, Monash University, Melbourne, 1973. Kramers, H., Westerterp, K . R . , "Chemical Reactor Design and Operation,'' p 118, Netherland University Press, Amsterdam, 1963. Longwell, J. P., Weiss, M. A,, Ind. Eng. Chem., 47, 1634 (1955). Root, R . B., Schmitz, R. A..A.l.Ch.E. J.. 15, 670 (1969). Spencer, J. L., Ph.D. Thesis, University of Pennsylvania, 1961 Van Heerden, C.. Ind. Eng. Chem., 45, 1242 (1953). Vejtasa, S. A,, Schmitz, R. A.,A.J.Ch.E. J., 16,410 (1970).
Received f o r review April 5, 1974 Accepted November 4,1974
A Response Surface Method for Experimental Optimization of
Multi-Response Processes William E. Biles Department of Aerospace and Mechanical Engmeerfng, Unfversity of Notre Dame, Notre Dame, lndfana 46556
A procedure is described which employs an experimental adaptation of the gradient projection method to find conditions on a set of process design variables which optimize a primary process response, subject to maintaining a set of secondary process responses within specified ranges. This approach consists of performing experiments to establish an improving direction, either the gradient or gradientprojection direction, and then performing experiments along the chosen direction to determine a stopping point. This procedure is repeated until optimum process conditions are found. Several factors affect the progress of this experimental search, including the type of experimental design, the spacing of design points in the experimental region, the number of replicates employed, and the magnitude of random error inherent in the process under study. Properly utilized, the proposed technique offers a viable strategy for process design and development experimentation when multiple process responses are encountered.
The scientist or engineer involved with process design and development is often confronted with employing experimentation to establish the best conditions for operating a process. This typically entails performing designed experiments to determine the optimum value q* for some functional relationship of unknown form
. ..,x,)
(1) where q is some measure of process effectiveness and ( x , , i = 1, 2, . . . , n ) are values for n controllable process variables. For example, q might be process yield (Yo),which is 9 = F(x,, x2,
dependent upon three controllable variables, reactor temperature xl, reactor pressure x2, and agitation speed x g . 152
Ind. Eng. Chem., Process Des.
Dev., Vol. 14, No. 2,1975
Each observation of the process response 7 is made up of a component from each of the process variables XI, x 2 , and x 3 , as well as a contribution from "process error," denoted by c . Thus, an experimental observation y of the process response q can be stated as y = F ( x , , x2,
. . . ,x,) + E
(la)
The quantity t is the inherent error in the process and arises due to the accumulated effects of various parameters outside the control of the experimentor. Many processes require simultaneous consideration of several responses ( q j , j = 0, 1, . . . , m ) . For instance, in the above example, the purity of the reaction product as well
as certain of its chemical and physical properties might be required to meet a set of specifications. Adherence to these specifications will likely affect process yield. The objective of a design and development program with such a prwess might be to determine those process conditions (x,*, i = 1, 2, . . . , n) which maximize the primary response, process yield qo, subject to holding the m remaining responses ( q J , j = 1, 2, . . . , m ) within specifications. Thus, process design can be viewed as a “constrained” optimization procedure. This paper describes a procedure for constrained process optimization uza experimentation. This procedure is essentially the so-called “response surface method” described bji Box and Wilson (1951), with additional features to provide for experimentation on or near binding process constraints. The proposed approach employs a sequence of experimental blocks to find conditions on a set of independent or “process design” variables which optimize the primary process response, subject to maintaining a set of secondary process responses within specified ranges. This technique is basically an experimental version of the gradient projection method developed by Rosen (1961). The Optimization Problem An expedient approach to multiple-response process experimentation is that of constrained optimization. In this approach, one process respmse 7 0 is designated a primary or “objective” response. The remaining m responses (q,, J = 1, 2, . . . , m ) become constraining conditions by placing restrictions on the values they may assume. The mathematical statement of the multiple-response process optimization problem is as follows. Maximize (or minimize) qo = F ( x , , x 2 ,
. . ., x,)
(2)
subject to a . 5 x. 5 c I.
...,Y z )
(i = 1,2,
(3 )
The interval c,-a, represents the controllable range of the ith process variable x L . The limits a, and C~ can either be those permitted by process equipment, or known bounds arising from the nature of the process under study. The quantity d j represents the specification limit on the j t h process or product response q j . As stated in ( l a ) , an experimental observation y , of the j t h process response variable qJ contains process error; that is
= F ( x , , x2,
. . . , x,)
+
€0
..., x “ ) + E j
y j = Gj(xi,x:?,
(2a) (4a)
This error term cJ represents the accumulated effects of various factors outside the experimentor’s control. Inasmuch as the proposed procedure makes use of leastsquares regression, however, it is necessary to state the standard least-squares assumption that for each process response q j the process error t i is a normally distributed random variable with mean zero and variance cJ2. That is, if (2a) and (4a) are expressed as
y j = qj
+cj
( j = 0, I , , . . , m )
(5)
in order that the expected value of the observed response y J be given by ~ ( y= ~ ‘ q) j
( j = 0,1,.
. , ,m )
(sa)
with variance v J 2 ,as required in least-squares regression,
it is necessary that
=0
Var(Ej) = u j 2
( j = 0, 1,
.. . , m ) .
( j = 0, I , . . , m )
(5b) (5c)
The response functions F(x1, x2, . . , x,) and [G,(xl, X Z , . . . , x,), j = 1, . . . , m ] are generally of an unknown form which can only be estimated through experimentation. A variation of this situation is that the functional forms for F ( X ) and G , ( X ) are known, but experimental data must be used to estimate certain coefficients in these models. Often there are other process functions [G,, j = m + 1, , p ] which are of a known form which can be stated deterministically without requiring experimentation. Such functions can be handled computationally and will not be considered further in this paper. Before describing the specifics of the proposed response surface method, it might be instructive to examine just what multiple-response, sequential-block experimentation is. An “experimental block” is a set of experimental design points conducted simultaneously, or in a randomized sequence of experiments. At each design point, an observation is recorded for each of the m + 1 process responses. For example, a t a specified set of values for reactor temreactor pressure x2, and agitator speed x3, one perature XI, would measure process yield yo, product purity y1, and tensile strength y2. A least-squares regression equation is then computed and an analysis of variance performed separately for each process response variable qJ as a function of the n process variables. Then an inference is drawn as to the best location for conducting the next block of experiments. This procedure is repeated until the most beneficial conditions for operating the process ( x i * , i = 1, n ) are established; thus, a “sequence” of experime blocks is generated. Box and Wilson (1951) have presented a lucid treatment of this concept for a single process response. The proposed approach to multiple-response, sequential-block experimentation builds on the “response surface” concept, but offers a strategy to be followed when process-generated constraints are encountered. This approach consists of alternating gradient-determining experimental blocks with step-determining experimental blocks, in moving from some starting conditions XO to “constrained optimum” conditions X*.The gradient-determining block consists essentially in performing a designed experiment, such as a 2” factorial or 2‘I-p fractional factorial design, as described by Box (1952) and Box and Hunter (1957). This designed experiment is placed around the current base point Xk.The results of this experimental block are used to estimate the most favorable direction of movement away from Xk.Then in a step-determining block, a set of t experimental design points are conducted simultaneously along this gradient direction, over the permissible range of the independent variables ( x i , i = 1, n ) , to estimate the location of the new base point Xk++l. This procedure is repeated until some point X* a t whirh the process is considered to be “optimized.” As long as there are no constraints acting on the process, the most.favorable direction away from X k is the gradient direction or, as it is commonly called, the “direction of steepest ascent.” In this situation, the procedure by Box and Wilson (1951) is applied without modification. When constraints become operative, however, it is necessary to adopt a somewhat different strategy. One such strategy is to take a “gradient-projection” direction away from a point X k which lies a t a boundary of the feasible region. This concept is similar to that developed by Rosen (1961) for solving nonlinear programming problems inInd. Eng. Chem., Process Des. Dev., Vol. 14, No. 2, 1975
153
volving functions of known algebraic form, without statistical variation. Although the gradient-projection method is intended as a computational procedure, it is possible to extend the concept to multiple-response process experimentation. A sequential-block method, then, could employ the gradient direction when experimenting in the interior of the feasible region, that is, away from any of the process boundaries, and the gradient-projection direction when exploring along the bounds of the feasible region. The following discussion treats these two segments separately.
50r
’
IO’
0
I
03
Given the problem of finding the maximum of a known function F ( X ) , which has no statistical variation, gradient search proceeds from a current base point X k to a new base point X k + l according to the relation x k
+ Ak[VF(Xk)]
(6 )
where V F ( X @ )= [ a F / a x , , aF/axs,
. . ., a ~ / a x , ] ’
(7)
That is, C F ( X k )is the n-component column vector of first partial derivatives of the function F ( X ) , evaluated a t the point X k . For maximization, this “gradient vector” T F ( X k ) describes the optimal improving direction away from the point X k , while for minimization, the negative gradient - C F ( X k ) is the optimal direction. Since the gradient direction is a local property of the function F ( X ) , there is some point along this direction a t which the function obtains its maximum; that is, there is a step Ak for which
F(X‘
+
A ~ [ v F ( x=~ max{F(Xk )]) x
+
A [ v F ( X ~ ) ] ) } (8)
Hence, gradient search consists of alternately determining a gradient direction TF(XA) and an optimal step hk along this gradient direction. This procedure is iterated until an optimal or near-optimal solution is found. Box and Wilson (1951) developed an experimental procedure which invokes the principles of gradient search. Called “response surface methodology,” their procedure employs a first-order designed experiment around a current base point X k to develop an estimate of the gradient direction. Using a 2 n factorial design and least-squares regression analysis, it is possible to estimate the tangent hyperplane a t point X k by the linear equation
(9) where y = an estimate of the process response 7, x i = value of the ith controllable process variable, and b, = least-squares regression coefficient. Multiple linear regression is used to estimate the n 1 coefficients (bi,i = 0, 1, . . . , n ) . Therefore, any experimental design employed for this purpose must have a t least n + 1 design points. Box (1952) and Box and Hunter (1957) have described experimental designs for this purpose. The regression coefficients ( b l , . . . , b n ) provide the estimate of the gradient direction T F ( X k ) ;that is, the coefficient b , is an experimental estimate of the first partial derivative dF/dx, in (7). The response surface method proceeds by experimentally determining the optimum step Ak away from the current base point X k . In effect, this is a univariable exploration with a new controllable variable A, which is the distance away from the base point X k along the gradient direction. This search can be performed in a single experimental block by employing t experimental trials a t points
+
154 Ind. Eng. C h e m . ,
Process D e s . Dev., Vol. 14, No. 2, 1975
I2
A
Gradient Search in Process Experimentation
Xk“ =
1
09
06
Figure 1. Curvilinear regression in a step-determining experimental block.
A I , Az, . . . , At along the gradient direction. T h e values of the controllable variables X , corresponding t o these points (A,, j = 1, . . . , t ) are given by
x,= P +
hj[VFW)]
(10)
which for each process variable ( x i , i = 1, . . . , n) is x i j = xik
+
Xjbi
(11)
Hence, x i / is the level of the Lth variable x l at the j t h distance A, along the gradient direction. ’Vow, these points (A,, j = 1, . . . , t ) must be chosen so that the t experimental trials span the region of interest, J t h o u g h they need not be spaced uniformly. The t obscwations of the response variable (y,, j = 1, . . . , t ) are recorded and curvilinear regression is used to fit the most statistically significant polynomial equation of the form
(12) where y = an estimate of the process response 7, X = distance along the gradient direction, w, = regression coefficient, and r 5 t - 1. Thus, if a second-degree equation best fits three or more data points, the approximating polynomial is y = wg A
+
wih
+
u’2A 2
Figure 1 illustrates a typical polynomial regression of the responsey on the step-size variable A. The polynomial expression in (12) is solved either analytically or numerically for the value A * which maximizes 5. Then the new base point in the exploration is given by xk+i = x k -I-
A*[VF(Xk)]
(13)
Hence, two sequential, experimental blocks are used in moving from point X k to an improved point X k L 1 : the first block establishes the direction T F ( X k ) ,while the second determines the step A*‘. This procedure is iterated in moving from a starting point XO to a solution X p . Termination of experimentation a t X p usually takes place according to a criterion such as
where 6 is an arbitrarily small increment in the objective process response. Figure 2 illustrates a hypothetical gradient search performed in sequential experimental blocks. However, we have not yet considered the constrained problem in any formal fashion. Gradient-Projection Search in Process Experimentation Given the Constrained optimization problem represented by relations (2)-(4), but without the error components
If ‘ T F ( X k + I )is the gradient direction for the objective function given by (2), then the gradient-projection direction S k * 1 is given by Sktl = [VF(XR+’)] - B,(Bq’Bq)-’Bq’[VF(Xk+’)] (18) where B , is as defined in (l’i),B,’ denotes the transpose of B,, and (B,’B,)-1 is the inverse of the matrix product of B,’ and B,. Hence, the expression B,(B,’B,)-lB’Q is an n x n matrix and S k + l is an n-component column vector. The search continues in this fashion until a termination criterion such as (14) is satisfied. The extension of the gradient-projection method to the experimental realm is accomplished in much the same fashion as that for the unconstrained gradient search described earlier. At an interior point X k , a first-order del or more design signed experiment consisting of n points is conducted to develop the estimate of the objective function F ( X ) , given by the least-squares multiple regression
+
XI
Figure 2. Unconstrained gradient search.
(e,, j = 0, 1,
gradient-projection method described by Rosen (1961) is one of gradient search coupled with orthogonal projection of the gradient onto a linear sub-manifold of any binding constraints. The algorithm begins a t a feasible point X k . A feasible direction S k is defined and a step of length X k is taken in this feasible direction that maximizes the function F ( X ) and yields a new feasible point Xk+I
=
p -+
+
Xk+l
-
xk +
pkXkSk
(16)
where Xk+1 lies on a boundary of the feasible region. That is, a t least one of the constraints (3) and (4) is satisfied a t the equality. Such a constraint is said to be “active.” At the boundary point X k + I , the gradient direction T F ( X k - l ) is determined. If this direction points back into the feasible region, the gradient search procedure is again invoked. If the gradient direction points away from the feasible region, however, it is necessary to proceed in a direction which lies along the intersection of the linearized forms of the active constraints. An appropriate direction S k + l is “projected” onto this intersection of linearized constraints, which is called the “linear sub-manifold .” Suppose that a t point X k + I , q constraints are satisfied as equalities. Let B , be the n x q matrix of first partial derivatives of these q active constraints [ G , ( X ) ,j = 1, . . . , q ] evaluated a t X k + l . That is, B , consists of the q gradient vectors [‘TG,(Xk+l).J = 1, . . . , q ] , or
[
aG, /axl, . . . ,
B, =
aG,/ax,
The coefficients (bo.1, . . . , bo,n) provide the estimate of the gradient direction TF(X) at X k . Then t experimental trials are conducted along the gradient direction, but now all m + 1 responses (q,, j = 0 , 1, . . . , m ) are observed and recorded a t each design point. Likewise, m + 1 curvilinear regressions are computed to yield
(15)
XkSk
As long as X k lies in the interior of the feasible region enclosed within (3) and (4), the direction Sk is the gradient direction given by ( 7 ) . As with gradient search, a step X k is sought so as to maximize the function F ( X k ASk). If Xk causes one or more of the constraints (3) and (4) to be violated, however, it is necessary to determine a quantity p k , 0 < pk 5 1,such that
]
These m + 1 equations need not be of the same degree. The polynomial function for the objective process response (21)
is solved for A * which optimizes j,. This A * is used to check the remaining m functions given in (20) to ascertain whether constraint violation has occurred. If not, (13) holds and the gradient search procedure is again invoked. If constraint violation has occurred, however, p k is determined so that (3) and (4) hold; hence, the new base point XkS-1 is determined from (16). A gradient-projection direction is then estimated by performing a first-order designed experiment about the new base point X k L 1 , employing multiple linear regression to find expressions for the objective function F ( X ) and the q active constraint functions [ G , ( X ) ,j = 1, 2, . . . , 91. These regression models are given by
We then solve for S k f l in eq 18, where
.
V F ( X k ” ) = ( b o , l ,. . , bo,n)’
(17)
(23)
and VGj(x’i+’) = ( b j , j , . . . , hj,,,)’
aG,/ax,, , . . , aGq/ax, Note that we have arbitrarily designated the active constraints as being the first q constraints, but we can reorder the constraints (3) and (4) as we like. There are up to 212 constraints in (3) and m constraints in (4).At most n of the 2n constraints in (3) can be active; hence, y 5 m +
n.
(19)
. . . , m ) , the
(.i = 1, . . . . q ) (24)
Once this gradient-projection direction Sk+I is established, t experiments are executed along this direction just as in the gradient search procedure. It is important to observe that a single experimental design yields the necessary information to employ leastsquares regression with each of the m + 1 process responses (q,, j = 0. 1. . . . , m ) . For instance, if a 2nfactorial or Y - P fractional design is used to estimate the gradient-projection direction given in eq 18, each design point yields observations (y,, j = 0 , 1, . . . , m ) for all m + 1 Ind. Eng. Chem.. Process Des. Dev., Vol. 14. No. 2 , 1975
155
Table I. First Direction-Determining Experimental Block Design point 1 2 3
4 x2
4
3
xi
x2
YO
0.8 0.8 1.2 1.2
0.8 1.2 1.2 0.8
3.73 4.58 7.00 5.71
?'I
1.03 2.67 3.09 2.19
Y2 5.57 5.19 9.50 10.30
2
subject to
Figure 3. Constrained gradient-projection search
process responses. Least-squares multiple regression is employed to compute the q 1 expressions given in (22), which in turn yield the gradient vectors (23) and (24). Figure 3 illustrates a gradient-projection exploration for a constrained process having two controllable process variables XI and x ~ where , a primary process response 70 is optimized subject to constraining conditions on two other process responses, 71 and 7 2 . The criterion for terminating experimentation in the face of constraining conditions will depend on whether the optimum solution X * is encountered within the feasible region or on a boundary. If X * lies interior to the feasible region, the stopping criterion given in (14) is suitable. If X* lies along one of the constraints (3) or ( 4 ) , one or both of the following conditions will result: (a) the gradient direction 'TF(X*) will point away from the feasible region, and (b) the gradient-projection direction S* will have the value zero. Finally, if X * lies at an intersection of two or more of the constraints in (3) and (4), the exploration will converge to such an intersection and a solution will be found there. It should be pointed out, however, that the gradient-projection method does not guarantee an optimal solution when employed in a purely computational procedure. The presence of experimental error creates an even more hazardous situation. Hence, the usual safeguards should be practiced before concluding that a solution has been found. Moreover, the experimentor must recognize that a solution obtained by such a procedure is only an estimate of the true solution.
+
Example Problem
To illustrate how this modified version of the gradient projection method can be applied to experimentation with constrained processes, let us consider a hypothetical process involving two controllable variables XI and x 2 , in which the objective process response 70 is restricted by two other process responses 71 and 172. Suppose the process is one in which observed process responses follow the relations
Suppose that the search is initiated a t the point X o = (1,l). The first step in optimizing this process is to perform a designed experiment about the point XO to estimate the search direction away from XO. Table I gives the design points and observed responses for a 22 factorial experiment about the point XO. The observed process responses y1 and y z clearly indicate that we are in the interior of the feasible region given by (29)-(31). Therefore, least-squares multiple regression can be used to estimate a gradient direction away from X o as in eq 19. This regression equation is = -2.92
+
5 . 5 ~+ ~2 . 6 7 5 ~ ~
(32)
Thus, the estimated gradient direction is V F ( X o ) = (5.5,2.675)'
The first step-determining block involves performing several experiments, in this case four, along this estimated gradient direction. Since we could reasonably ascertain from the first block results that constraint violation would occur if xl goes much beyond 5 and x 2 beyond 3, the range for X is chosen to be 0 IX 5 1.2. Table I1 gives the design points and process responses for the four trials along the gradient direction, as computed by eq 11. The polynomial regression equations computed from these results, as given in (20) are as follows.
go = 5.51 + 4 1 . 4 ~+ 1 0 6 . 7 ~ ' 91 = 1.82 + 18.3X + 35.9X2 o2 = 6.95 + 46.0X - 7.83h2
(33) (34) (35)
Equation 33 suggests that X O should be made as large as possible to maximize 70; however, it is obvious that X o must be so chosen that the constraints (30) and (31) are not violated. Equating the right side of (34) to its boundary value 25 and solving for A0 yields the solution Xo = 0.588. Equating the right side of (35) to 27 and solving yields X O = 0.472. Hence, the smaller value must be chosen. Therefore
Xi
x1 = xo + X ~ [ V F ( X 0 ) ] = ( 1 , l ) ' + 0.472(5.5,2.675)' X' = (3.70, 2.26)'
The terms 6 , (0.0.25) indicate that process error is normally distributed with mean zero and variance uJZ = 0.25. The constrained optimization problem for this hypothetical process is as follows. '
156
Ind. Eng. Chem., Process Des. Dev., Vol. 1 4 , No. 2, 1975
The point X I lies on the estimate of the boundary GAX). This does not mean that (3.60, 2.26)' necessarily lies on the actual boundary Gz(X). Experimental error will generally cause the point to fall to either side of the baundarY.
Table 11. First Step-Determining Experimental Block Design point X 1 2 3 4
0 0.4 0.8 1.2
YO
x2
X1
1.00 3.20 5.40 7.60
1.00 2.07 3.14 4.21
5.53 39.07 107.00 208.84
Table IV. Second Step-Determining Experimental Block
3‘1
Y2
1.75 15.11 39.23 75.59
7.00 23.94 38.88 50.81
Table 111. Second Direction-Determining Experimental Block
Design point h 1 2 3 4
0 0.05 0.10
x2
Yo
Y1
Y2
2.06 2.46 2.06 2.46
43.79 46.54 51.73 55.54
15.69 16.48 18.56 20.41
27.29 24.80 29.06 29.14
Performing a second direction-determining block, a second 2 2 factorial experiment is employed about the estimated boundary point XI to estimate a new search direction. Table In gives the design points and responses for this third block of experiments. The multiple linear regression equation representing the objective function F ( X )is
c0 =
-45.8
+
2.26 2.97 3.68 4.39
v2
2’1
49.06 62.10 78.82 97.21
18.25 22.95 30.03 39.48
27.66 26.76 23.57 20.09
Table V. Third Direction-DeterminingExperimental Block
1
3.40 3.40 3.80 3.80
so
.y 2
3.60 3.87 4.14 4.41
0.15
Design point
Design point x1
1 2 3 4
x1
2 1 . 3 ~ 1+ 8 . 2 5 ~ 2
(36)
2 3
4
x1
X2
3.85 3.85 4.05 4.05
3.10 3.30 3.10 3.30
1’0
64.05 66.10 69.06 71.17
?’ 1
Y2
24.26 25.63 26.30 27.23
25.37 23.77 26.31 24.77
Table VI. Third Step-Determining Experimental Block Design point X 1 2 3 4
0 0.02 0.04 0.06
Xi
N2
YO
3.95 4.00 4.05 4.10
3.20 3.12 3.04 2.96
67.05 67.08 67.23 68.03
Y?
\‘1
25.71 24.98 25.35 25.85
24.63 26.99 26.93 28.95
Then the estimated gradient direction is ing constraint. The estimated solution a t this point is
V F ( X 1 ) = (21.3, 8.25)’ As this gradient direction points away from the feasible
region given by (29)-(31), it is necessary to estimate the gradient-projection direction S1, given by eq 18. Since the matrix B , is simply the vector T G 2 ( X 1 ) here, that is, the gradiect vector for the single violated constraint, the only additional information needed is the multiple linear regression equation representing G2(X1). From the data in Table 111, this regression equation is
Dz =
6.57 t 7 . 6 3 ~-~2 . 8 8 ~ ~
(37)
Therefore VG,(X‘) = (7.63, -2.88)’
From eq 18
-
= [2;:;5]
[ 7 . 6 3 1 [( 7.63)‘ ( 7.63)I-l [ 7.63]’[21.3 ] -2.88 -2.88 -2.88 -2.88 8.25 = [ 1;:;5] Then in the second step-determining block, four design points are conducted along this gradient-projection direction in the range 0 5 h 5 0.15. Table IV gives the design points and process responses for this fourth experimental block. The curvilinear regression equations computed from these results are as follows.
91
= 18.3
+ 242X + 5 3 5 ~ ‘ + 70.3X + 4 7 5 ~ ’
(39)
92
= 27.8
- 13.1X - 258X2
(4 0)
% = 49
(38)
Equating (39) and (40) to 25 and 27, respectively, and solving for A 1 yields X 1 = 0.0345 with G 2 ( X ) as the bind-
X 2 = (3.79, 2.75)’; y o = 58:
1’1
= 21.2:
\’?
= 27
However. if h1 = 0.066 with G 1 ( X ) binding, a better s o h tion is obtained without violating the G z ( X ) constraint. This estimated solution is
X2 = (3.95, 3.20)’:
y o = 67.2: 1’1 = 25: y2 = 25.8
These results suggest that we are nearing the intersection of the constraint functions G 1 ( X ) and G 2 ( X ) , and that a constrained optimum solution might be encountered a t such an intersection. Performing a third direction-determining block using a 2 2 factorial experimental design, the results given in Table V were obtained. Because the search has neared the intersection of constraints, the design points are placed closer toget her. The regression equation for the objective function F ( X ) obtained from the data in Table V is
70
= -65.2
I 2 5 . 2 ~ 1+ 1 0 . 4 ~ ~
(41)
The gradient direction (25.2. 10.4)’ points away from the feasible region. Hence, it is necessary to compute the gradient-projection direction S’2 with G 1 ( X ) as the binding constraint. From the data in Table V, the regression equation for G l ( X )is i.1
= -28.5
+
9.1~1
+
5.75~2
(42)
with the estimate of T G 1 ( X 2 )as (9.1. 5.75)’. Hence, S2 is (2.6, -3.9)’ from ey 18. Note that B I is simply TGI(X), the gradient vector for the violated constraint. Performing the third step-determining block along this gradient direction, the results in Table VI are obtained. Since the second design point yields values for y1 and y2 that lie a t the intersection of constraints G I ( X ) and G 2 ( X ) , this point is taken as a solution and the search is terminated. Therefore, the experimental solution is Ind. Eng. Chern., Process Des. Dev., Vol. 14, No. 2, 1975
157
The known solution to this test problem is
= ( 4 , 3 ) ; yo* = 6 6 :
VI*
= 25;
12*
= 27
Thus, six experimental blocks of four trials each have been used to determine a solution which is within 2 . 5 7 ~of the known solution for this hypothetical process. The progress of this search is illustrated in Figure 3. Conclusions This paper has presented a novel technique for experimental optimization of multiple-response processes. Although the efficacy of this procedure with real industrial processes remains to be demonstrated. the fact that it represents only a relatively minor departure from the proven techniques of Box. e t al., should a t least invite trial and evaluation by the industrial community. There 'are a number of aspects of this procedure which bear further scrutiny. A comparison of the type of experimental design and the number of replicates a t each design point used a t a boundary point are of considerable importance in gaining a reliable estimate of the gradient-projection direction given in (18). Brooks and Mickey (1961) state that the minimum number of experimental design points can be employed in a block used to estimate the gradient direction. Whether this conclusion holds for multiple-response processes, and hence for the gradient-projection direction, remains to be examined. The placement or spacing of design points for multiple-response experimentation bears further study. Myers and Carter (1973) have treated this problem for processes with two responses, but the extension of their results to the general problem of m 1 responses remains to be examined. The effects of various levels of process error e , must also be evaluated. Finally a careful study of different strategies for terminating experimentation is needed.
+
Nomenclature ai = lower bound on the value of the ith controllable process variable x i
bj,l = estimate of the regression coefficient for the ith controllable process variable x i , for the j t h process response function
158
Ind. Eng. Chem., Process Des. Dev., Vol. 14, No. 2, 1975
cL = upper bound on the value of the ith controllable process variable x i d j = specification level on the j t h process response 7, F( ) = objective response function of unknown form for which least-squares regression estimates are obtained Gj( ) = j t h process response function of unknown form, for which least-squares regression estimates are obtained m = number of process response functions treated as constraint functions n = number of controllable process variables x t W j , k = kth coefficient in the polynomial regression equation for the j t h process response in a step-determining block x L = value for the ith controllable process variable X k = kth base point in the succession of experimental blocks, X = ( x i , i = 1, YJ= observed value of thejth process response variable p j = predicted value of the j t h process response variable arising from a least-squares regression equation S A = gradient projection direction away from base point Xk
B , = n x g matrix of estimates of regression coefficients (first partial derivatives) of the q active constraints ( G , , ( X )j, = 1, . . . , 9 ) , a t the current base point X k Greek Letters 6 = increment used in the termination criterion t J = random error inherent in thejth process response T = denotes gradient vector, or vector of first partial derivatives 1: = denotes summation over first p terms q j = expected value of the j t h process response variable Literature Cited Box, G. E P.. Biornetrika. 39,49 (1952) Box, G E. P.. Biometrics, 10,16 (1954). Box. G. E. P., Draper, N. R . , "Evolutionary Operation: A Statistical Method for Process Improvement," Wiley, New York, N . Y . , 1969. Box, G E. P., Hunter, J S., A n n . Math Stat., 28, 195 (1957). Box. G. E. P..Wilson, K.P..J . Roy. Stat. Assoc., Ser. 8, 13, 1 (1951). Brooks, S. H.. Mickey, M . R., Biometrics, 17, 48 (1961). Myers, R. L.. "Response Surface Methodology." Allyn and Bacon, Boston, M a s s , 1971 Myers, R. L., Carter, W. H . , Technometrics. 15, 301 (1973). Rosen, J. B..J. SOC.Ind. Appl. Math.. 8, 191 (1960) Rosen, J, B . , J. SOC.Ind Appl. Math., 9, 514 (1961).
Receiuedfor review April 11, 1974 Accepted November 4, 1974
