PROCESS OPTIMIZATION IN T H E PRESENCE OF ERROR THEODORE D. AHLGREN AND W I L L I A M F. STEVENS
Department of Chemical Engineering, ih’orthwestern L’nicersity, Euanston, Ill.
Optimization of a process performance criterion in the presence of error in its measurement i s often the concern of the engineer. In many such situations no complete set of defining equations i s known, and the effect of the decision variables on the criterion function must b e determined experimentally. Direct search procedures for determination of the optimum settings of the decision variables in the presence of error have been developed and proved rigorously to converge to the optimum with unity probability. One such method i s the Kesten modification of the Kiefer-Wolfowitz procedure. Application o f Kesten’s modification to a simulated chemical process has demonstrated that convergence to the optimum cannot be accomplished in a practical number of runs. When error was large relative to the slope of the response surface, practically no improvement in the criterion resulted. However, Kesten’s accelerated procedure was extremely likely to result in an improved response, after a relatively small number of runs. A further modification proposes continued reapplication of the Kesten procedure. While the mathematical rigor i s destroyed, the improvement in ability to reach the region of the optimum i s demonstrated to b e clearly superior to the performance o f the rigorous Kesten modification. Convergence to better than 95% of the optimum response was achieved with large random errors.
IME-IXDEPENDENT O P T I ~ i I z A T I o N of a process performance Tcriterion is of considerable importance to the chemical engineer in the application of chemical engineering principles to research, design, and production problems. This criterion is generally dependent on several independent or decision variables subject to a number of nonlinear constraints. In most research and production situations, the defining set of equations is totally or in part unknown. This situation requires direct measurement of the value of the criterion function for any given set of values of the decision variables. Thus, a relationship between the criterion function and the decision variables must be developed by experimentation. To complicate further these situations for which optimization is desired, the measurements required to establish the necessary relationship between the criterion function and decision variables are not error-free. While the commonly used direct search techniques (6, 8, 101, if applied to a nondeterministic system, may result in an improved set of independent variables, reaching the optimum set of values of variables is not assured. Thus, use of different techniques is appropriate. Direct search procedures which have been proved mathematically to converge to the optimum values of the decision variables with a probability of 1 have been developed. Kiefer and Wolfowitz (5) developed a technique for finding the maximum of a regression function in one dimension with assurance, providing reasonable restrictions on the error and the underlying functions are assumed. This work was extended by Blum (2) and Sacks (7) to many dimensions. Kesten (4) developed a modification which, when applied to the KieferWolfowitz technique, can provide an acceleration of the convergence process under slightly more restrictive assumptions. The Kiefer-Wolfowitz technique and all extensions usually require the number of criterion function evaluations to approach infinity to assure convergence. This requirement could make these theoretically sound procedures relatively unimportant in chemical engineering applications. In practice, it is impossible to use an infinite number of runs to determine an optimum set of decision variables. Generally, a very
290
l&EC
PROCESS DESIGN A N D DEVELOPMENT
limited number of runs is available; convergence to the region of the optimum must be rapid. This paper summarizes the application of a normalized version of Kesten’s accelerated Kiefer-Wolfowitz procedure to a multidimensional chemical engineering problem ( 7 ) . The effect of the amount of error in criterion function evaluation on the rate of convergence is investigated. The practical applicability of Kesten’s technique is emphasized. Applicability is directly related to the speed of convergence to the optimum value of the criterion and the corresponding set of values of the decision variables. Description of Kesfen Accelerated Procedure
Direct search in the presence of error involves the same two basic considerations which are present in a deterministic search. Choose a direction of independent variable change in which to search for the optimum. Select the amount of change in that direction. When error is present, the direction of search is chosen as if error were not present-that is, the experimenter reads the experimental results and chooses the direction of future search as if these results were error-free. By proper selection of the amount of change (referred to as step size) in that direction, convergence has been proved to be assured if reasonable limitations on the behavior of the error are assumed. All workers mentioned above use a stepping sequence of the general form x n . ~= T ( x d
+
rn
(1)
where T ( x n )is some transformation, the specific form of which can be chosen by the investigator, and r, lumps all randomness. All these workers require the same general restriction on the error in the measurement of the criterion functions and in the criterion function itself. The error must be unbiased. Any bias would distort the perception of the underlying criterion function.
The sum of its va.riances must be finite, so that residual fluctuations die out in the long run. The average slope of the criterion function for any pair of measurements must be boundable by a straight line of finite slope. T h e normalized vel-sion of the Kiefer-Wolfowitz procedure for the multidimensional case is given by the equation
T h a t is, the new base point, x,+~,is determined from the previous base point, x,, plus the current value of a “stepping sequence,” a,, times the sign of estimated slope at the base point, x,. Normaliz,ition (use of the sign of the estimated slope rather than its numerical value) has been shown empirically to improve the performance of the method for certain ill-behaved criterion functions (8). The stepping sequence, a,, is a sequence which, among other properties, approaches zero as the search continues. Thus, even though the search is continuing far from the optimum, step size can already have been shortened substantially, slowing the speed of convergence, Kesten’s accelerated procedure merely eliminates shortening of the step size when movement is apparently toward the optimum. This idea, applied to the Kiefer-Wolfowitz procedure, would result in shortening of the step size only when the sign of the estimated slope changes. Thus, far from the optimum, because of the convex (concave) shape of the criterion function, there would be fewer sign changes and the step size would be less likely to be shortened far from the peak sought. T o prove convergence of this acceleration technique, Kesten requires that the sequence a, have the following properties. limit a, = 0 n+ m
(3) (4)
N
limit N-+m
un+l
un2
n=l
5
a,
