Instrumental Simplex Optimization Experimental Illustrations for an Undergraduate Laboratory Course D. J. Leggett University of Houston, Houston, TX 77012 A question frequently faced by chemists involved in pure or applied research is, "Now that the system under study works, can it he made to perform more efficiently?"This situation often occurs in 1) instrumental analvsis where the machine's performance is a function of several adjustable oarameters. 2) the imorovement of a ouantitative determination wheie the sensitivity is to be maximized or side reactions and interferences minimized. 3) industrial oroductivitv where the yield of a chemical process is to he raised, and 4) numerical analysis where experimental data are fit to a mathematical model by the simultaneous adjustment of a numher of variables. Although there are many methods availahle to perform the above tasks including factorial design of experiments, the stochastic approach, the one-factor-at-a-time approach (for 1) to 3)),and many nonlinear least squares algorithms (for 4)) most of these techniques are either prone to failure, too time consuming, or too difficult to understand for the average chemistry major's background of statistics and mathematics. The Slmplex Method The original simplex concept was derived by Box and Wilson ( I ) who combined response-surface methodology and hill-climbing techniques to produce an algorithm capable of varying many factors simultaneously and arriving a t the optimum level of resoonse. At that time the techniaue was known as "evolutionary operation" (EVOP) (2) and it remained for Soendlev et al. ( 3 ) .in 1962, to develoo the method d"sequentinl appliwtitm c t t aimplrx ilri~gnsin optmuatlon and ewlurionary oprr;~ri." Nrldt.r and Mead I II pruvided a simple yrt ~r,u.vriulm o d i i ~ ~ ; r t i 111 w ~the griginal iirnpler r n r t h d , and at thi3 tirnr, t ht. methud has I ~ e musrd inn u.de variety of applications. The principles of the fixed (3)and variable (4) sized simplex and their implementation in a fourth-year undergraduate instrumental analysis course will he discussed. The simplex technique has been reviewed in detail by Deming e t al. (5-7, bibliography). The Slmplex Algorithm Consider an analytical instrument, the response of which is to he optimized. The response is believed to be dependent uoon two variable inouts. Let these inouts he restricted. arhltrarily, to values hdunded by 0.0 and'100.0. If three experiments are performed, each a t different levels of the variable inputs, then three different responses from the instrument will be obtained. Assuming that these resoonses are measurable then one of the three &I he the worst'response. Since the object of the procedure is to improve the response of the instrument, it is logical to reject the input variahles that led to the lowest response replacing them with a new pair that will provide, hopefully, an improved response. The above description is the basic aim of the simplex method and may he strictly formulated in terms of five simple rules. The following terminology will he used to describe the simplex method. The variahle inputs, upon which the response depends, are known as factors. The particular input value of that factor is known as its leuel. A simplex figure is a geometric
figure having one more vertex than the number of factors. Rule 1: A moue toward the optimum response is made after each observation of the response. This move is made once the responses have been ranked in the order best, next-to-worse, worst. For the situation involving more than two factors, the ranking is performed a t best,. . . , next-to-worst, worst. Rule 2: A moue is made such that the uertex having the worst response, W , is rejected and replaced by a new wertpx generated bv a reflection throueh the midouint of the renlalnlrr~t,\/wrfaw Fiyurv I i n d ~ < . ~the ~ t epn,grrs> i t r m the initial simuler ( : ~ / N ' I thv lin.11s~tnolrxIJKO. i ~ t ~the d tilhle details the ranking, rejection, and generation of each new simplex. Rule 3: If, on rankrng the vertices of the next simplex, the neurlv created uertex is the worst uertex, then reject the ,I' xr.fr,-r~r,r.\t wrlcr. lfthii prtredure is not adopted, the next simplex gtmt~otedwdl be identical to the pre\iou.; wnplvx ,ind ahruptl? hence thr the optim~zilt~trn [)ruwsa will terlni%II~ rank111~ wder. \Ye Are only interested in fj, .V, m d I\' for any n-dim&sional problem. This rule has been invoked for runs 10,13,14 and 16 in the table. Rule 4. Boundary constraint violations are accorded a suitably bad response. If one or more of the new vertex levels are found to he outside the boundary constraints, that particular experiment is not performed. However, the response is assigned a suitably low (i.e., had) value and included in the next round of calculations. Rule 5: I f a uertex has been retained for n 1 moues, where n is the number ofuertices, then the responsefor that uertex is re-eualuated. This procedure prevents the simplex from becoming stranded on an experimental point whose response mav have been erroneouslv" hieh. .. It should he noted that the retention of a single vertex for n 1times does not necessarily mean that the optimum has been reached. Figure 1shows that vertex H is retained five times before a vertex, K, having a
+
+
FACTOR 1 Figure 1. Progress of fixed size simplex, superimposed on a hypothetical reSO0"SB surlace.
Volume 60
Number 9
September 1983
707
higher response, is found. For a two-dimensional simplex, starting from an equilateral simplex, the optimum is reached when the simplexes close pack to f o m a hexagon. This can he seen in Figure 1 where vertex K has the highest response. However, this phenomenon may not be observable when dealing with systems of higher dimensions. These five rules are the basis of the simplex method. The Fixed Size Simplex
Three limitations are apparent when using the fixed size simplex. First, the optimum is never precisely located, except by chance. Second, a false optimum may be located. Third, the progress towards the optimum can only proceed at a fixed rate. None of these limitations are severe and in fact may, in certain situations, prove to he beneficial. T o obtain a precise location for the optimum, a smaller simplex may he started within the region of the optimum located by the larger simplex. This process may he repeated as many times as desired. On the other hand, if an imurovement in response of only greater than dOPc oi the presumed maximum response is desired, then the fixed ii7e simplex wvuld pmve t o he entirely adeq"ate. The location of a local rather than the global optimum is a problem not only restricted to this type of search procedure. Restarting the process from a different point in factor space
ESTABLISH INITIAL SIMPLEX AND EVALUATE RESPON
will, in all ~robability,establish whether the first optimum was local or not. That the fixed size simplex cannot accelerate its progress in accordance with the response surface was recognized by Nelder and Mead (4). However, the fixed size simplex provides a valuable tool for localization and trackine of a movine- oo. timum. For example, instrumentation that is prone to drift mav he continuallv monitored and held at its peak performance by employing a small fixed size simplex and interfacing the equipment to a small computer. The Variable Size Simolex Precise optimum location and simplex acceleration have been described hv Nelder and Mead ( 4 ) . with the incorporation of the operations expansion and contraction. ~ o g e t h e r , these operations provide the means by which a simplex figure may expand and accelerate towards the region of the optimum and then reduce its search region until the optimum is located with the desired precision. The movement of the simplex is governed by the same basic set of rules as for the fixed size simplex. Additional tests are used to decide which operation to perform in a given situation. These rules are presented as a flow chart in Figure 2. The possible operations in the variable size simplex are shown in Figure 3. The Relevant Calculations Assume that the responses for the vertices BNW, Figure 4. have been ranked as indicated earlier. Expressing the coordinates of these vertices as vectors we have
w = (1 I IS NEW VERTEX WORST VERTEX
1)
N = ( 5 8) B = ( 9 1)
The reflection oneration is nerformed bv reflectine the worst ve&x througk the centroib. of the remaining hypegace, 3. Thus the coordinates of 3 are needed.
Flgure 2. Flow chart summarizing rules for lhe acceptance or relenion of e vertex, for e i m fixed or variable size sim~lex.
Progress of Slrnplex (Figure 2) Showing Rejected and New Vertices and Resoonses
Move 4 5 6 7 6 9 10 11 12 13 14 15 16
Current Simplex
Rankeda
Relected
Venlces
Vertex
ABC BCD CDE DEF DFG FGH FHI HIJ
ABC4 BCO CEO EDF DGF GFH IFH IJH JHK LHK MLK MNK ONK
A B C
KN(
HKL KLM KMN KNO
E D G FS I J Hr LC M NS
New Vertex
0 E F G H I J K L M N 0
Ranked in WCef W t . nM-lo-uomf. beol. 'Re$pons* lor initial simpler are A = 37, B = 45. C = 53. AppiiCBtlOn01 Rule 3.
708
Journal of Chemical Education
4=Jl
Response for New Vertex
Figure 3.The simplex figwe illustrating the inter-relationshipsbetween each move.
65 60 76 71 91 66 77 95 72 66 85 63 77
Figwe 4. The simpla figure with calculated coordinates for each vertex. See
text for details of calculations.
By definition, P = average of the sum of the remaining vertices
Useful discussions on the various aspects of the fixed and variable sized simplex have been written by several authors. These have been included in the Biblioaraphy, in a suggested reading order.
where n = number of vertices
The Simplex Technique In an Undergraduate Laboratory Course The introduction of the simplex technique to an undergraduate class has been achieved by using a computer to simulate an analytical instrument or chemical process. This approach has been used for the following reasons. Although the simplex technique is simple to comprehend and apply (often in hindsight) it demands that the student fully understand the orincioles and practices involved. Most of our students have had no prior experience with analytical instrumental techniaues (except s~ectrophotometricmethods). These two facts can lead to rapid disenchantment with simulex if students are r e ~ u i r e dto optimize, immediately, a real Hna~ytica~ instrurnent.~nyundetected arithmetic mistake or misunderstandine- of the method may lead much wasted laboratory time. The use of a computer to simulate an analytical instrument ensures that 1) the "response" is always correct and free from experimental
Therefore, p = (7 4.5) The distance between p and W is the numerical difference between the two vectors,
The coordinates ofthe reflected vertex are obtained by adding ( p - W) to the centroid coordinates, (p - W) = (6 3.5) + p = (7 4.5) p+(p-W)=(13 8 ) = R For the fixed size simplex the calculations finish a t this point. For the variable size simplex the expansion or contraction coordinates are dependent upon the acceleration factor chosen. For the majority of operations acceleration factors of two for an expansion and 0.5 for a contraction, are satisfactory. In principle, however, there is no reason why other factors cannot be chosen. Thus expansion coordinates for E are given by and for the example in Figure 4, The contraction towards the reflection, CR, or towards the worst vertex, Cw, are defined by Cn=p+0.5(p-W) and for the example in Figure 4.
Once the coordinates of the reflection vertex are calculated, and the response evaluated, the remaining vertices (E, CR, and Cw) are examined in accordance with the flow chart, Figure 2. The Initial Simplex The establishment of the initial simplex is also an automatic procedure. The first vertex is normally selected a t factor levels that give a measurable response. Coordinates for the other vertices are calculated by applying the general technique of Spendley e t al. (3).For a two-dimensional simplex, if the first vertex coordinates are (20,20) then the other twovertices will be a t (20 a, 20 b ) and (20 b, 20 a), where
+
+
b
+
S"
= -((n
+
+
- 1)
n\/2 where n = number of factors and S, = step size for each factor. This topic is reviewed in areater detail by Yarbro and Deming (8).
~~~
~
~
eTTOI;
2) there is no waiting time between the commencement of an "experiment" at the new factor levels and obtaining the re-
sponse; 3) students are freed of concerns involving the incorrect use of the
analytical instrument and therefore may concentrate on perfecting their understanding of the simplex teehnique. This familiarization process is broken down into three parts: 1) Attempts to obtain the best response from a system with three interacting factors by any method, other than the simplex teehnique. This is, in effect, an exercise in frustration and emphasizes the need far a systematic approach to optimization. 2) The application of the fixed size simplex technique to a two factor system. 3) The application of the variable size simplex technique to a two and three factor system. When dealing with a two-dimensional simplex the coordinates of each vertex, after they have been calculated, are plotted. This allows the student to check the accuracy of the arithmetic, and also provides visual encouragement that the process is being optimized. The ability to plot the progress of the simplex is particularly valuable when dealing with the variable size simplex. It has been found that most arithmetic errors are made in the early stages of this part of the experiment. Finally, the three-dimensional simplex experiment is run and this, since it is not easy to plot, reinforces the need to check all calculations. Experience has shown that by this stage a few mistakes are made. It should he pointed out that the simplex method does not, in principle, fail if the vertex coordinates were incorrectly calculated. Either it will take a few more experiments to realign the simplex figure, or, in the most severe instance, the simplex will be distorted sufficiently that in subsequent moves one vertex may partially collapse. Instrument Simulation usiua - a Com~uter An interactive computer program (9) has been developed that will provide a numerical response to a set of factor levels. The program consists of a number of different experiments including the "exercise in frustration" mentioned earlier. Depending upon the particular experiment chosen the proeram uromots . . the student for information, such as student name, experiment type and problem number, factor levels, and the tvoe .. of simplex move contemplated, if the variable sized simplex is heing w ~ r k t don The program does not periorm an" calculati
