488
Ind. Eng. Chem. Res. 2001, 40, 488-489
CORRESPONDENCE Comments on “Global Optimization for the Parameter Estimation of Differential-Algebraic Systems” Rein Luus* Department of Chemical Engineering, University of Toronto, Toronto, ON M5S 3E5, Canada
Sir: In a recent paper Esposito and Floudas1 compared two approaches for parameter estimation of differential-algebraic systems, and by using six examples, they concluded that the integration approach is better than the collocation approach. They did not, however, consider the Luus-Jaakola optimization procedure,2 which has been shown to be well suited for parameter estimation3-5 when the region sizes are determined by the extent of the variation of the variables.6 The goal here is to provide such a comparison for two of their examples. Let us first consider their last example involving the Lotka-Volterra parameter estimation problem where the system is described by two differential equations
dz1 ) θ1z1(1 - z2) dt
(1)
dz2 ) θ2z2(z1 - 1) dt
(2)
with the initial state
z(0) ) [1.2 1.1]T
(3)
The parameter estimation problem is to select positive values for θ1 and θ2 such that the deviations from the data values for the state variables at 10 points is minimized. Therefore, the performance index to be minimized is 10
I)
[z1(tk) - zˆ 1(tk)]2 + [z2(tk) - zˆ 2(tk)]2 ∑ k)1
(4)
The 10 data points zˆ i(tk) are given in the paper.1 By taking 100 sets of initial values for θ1 and θ2 at random in the range 0.1 e θi e 9.9, i ) 1, 2, the minimum value of the performance index I ) 1.249237 × 10-3 was obtained 84 times when the upper limit of 10 was imposed on the parameters. The second best local optimum, as reported in the paper,1 was obtained 15 times, and the third best local optimum was obtained once, as shown in Table 1. These results were obtained by using 15 randomly chosen points at each iteration (R ) 15) and 10 passes, each consisting of 51 iterations. The initial value of the region was chosen as the value of the parameter, and the region was reduced by γ ) * Phone: 416-978-5200. Fax: 416-978-8605. E-mail: luus@ ecf.utoronto.ca.
Table 1. Local Solutions for the Lotka-Volterra Problem Using LJ Optimization Procedure When an Upper Bound of 10 Is Placed on the Parameters performance index
θ1
θ2
frequency
1.249237 × 10-3 1.920081 × 10-2 5.101019 × 10-1
3.24343 10.00000 8.78718
0.92090 6.49621 2.11905
84 15 1
Table 2. Local Solutions for the Lotka-Volterra Problem Using LJ Optimization Procedure When No Upper Bound Is Placed on the Parameters performance index
θ1
θ2
frequency
1.249237 × 10-3 1.249237 × 10-3 1.249237 × 10-3 1.249237 × 10-3 1.249237 × 10-3 1.249237 × 10-3 8.45894 × 10-1
3.24343 15.07830 26.9132 38.7481 50.5831 62.4178 0.30227
0.92090 4.28114 7.6414 11.0016 14.3618 17.7221 0.10000
28 35 26 4 2 3 2
0.95 after every iteration. The region at subsequent passes was chosen according to the change in the parameter value during the previous pass as outlined by Luus.3 For integration, the standard integration subroutine DVERK7 with a local error tolerance of 10-8 was used to give reliable results. The total computation time for the 100 cases was 362.2 s on a Pentium3/500 personal computer. Much more interesting results were obtained when no upper bound was put on the parameters and initial values for θ1 and θ2 were chosen at random in the range 0.1 e θi e 20. As is seen in Table 2, where 100 sets of initial values were chosen, numerous local optima exist with identical values of the minimum performance index. One of the nonglobal local minima obtained by Esposito and Floudas was obtained twice. The global minima occur at regular intervals of 11.83 for θ1 and 3.36 for θ2. Therefore, the global optimum is not unique for the given data. However, the data given by Luus in Table 6 of ref 3 for this problem provides a unique optimum at θ1 ) 26.22763, θ2 ) 26.22540, which is much more difficult to obtain as θ1 ) 1.000007, θ2 ) 0.999986 give I ) 1.5975 × 10-9, which is just marginally higher than the minimum value I ) 1.5819 × 10-10 that exists considerably further away in the parameter space. To solve the parameter estimation problem as stated in example 5 of the paper,1 10 sets of starting points for the five parameters were chosen at random in the range 0.1 e θi e 1.9, i ) 1, ..., 5. In the LJ optimization procedure, 10 passes were used, each consisting of 20
10.1021/ie000602r CCC: $20.00 © 2001 American Chemical Society Published on Web 12/01/2000
Ind. Eng. Chem. Res., Vol. 40, No. 1, 2001 489
iterations. In each case, the minimum value of the performance index, I ) 0.1069306, was obtained with R ) 40, R ) 20, and R ) 15. The optimum values of the parameters were θ1 ) 5.240716, θ2 ) 1.217644, and θi ) 0, i ) 3-5. The computation time for the 10 cases was 27.9 s with R ) 40 and 10.8 s with R ) 15 on a Pentium3/500. In each case here, only the global optimum was obtained. Therefore, for these two examples, the global optimum is obtained very easily with the LJ optimization procedure, and the computational effort is substantially less than that required by either of the methods in the paper.1 The LJ optimization procedure also avoids most of the numerous local optima that are present in these two problems. The programming is straightforward, requiring no more than 125 lines of Fortran code. The global optimum of highly nonlinear optimization problems is very difficult to establish with absolute certainty, so further research in this area should continue to provide the user a variety of methods that can be used to cross-check the results to increase the certainty of obtaining the global optimum.
Literature Cited (1) Esposito, W. R.; Floudas C. A. Global Optimization for the Parameter Estimation of Differential-Algebraic Systems. Ind. Eng. Chem. Res. 2000, 39 (5), 1291. (2) Luus, R.; Jaakola, T. H. I. Optimization by Direct Search and Systematic Reduction in the Size of Search Region. AIChE J. 1973, 19 (4), 760. (3) Luus, R. Parameter Estimation of Lotka-Volterra Problem by Direct Search Optimization. Hung. J. Ind. Chem. 1998, 26 (4), 287. (4) Luus, R. Application of Direct Search Optimization for Parameter Estimation. In Scientific Computing in Chemical Engineering II; Mackens, K., Werther, V., Eds.; Springer: New York, 1999; pp 346-353. (5) Luus, R. Iterative Dynamic Programming; Chapman & Hall/CRC: London, U.K., 2000; pp 54-58. (6) Luus, R. Determination of the Region Sizes for LJ Optimization Procedure. Hung. J. Ind. Chem. 1998, 26 (4), 281. (7) Hull, T. E.; Enright, W. D.; Jackson, K. R. User Guide to DVERKsa Subroutine for Solving Nonstiff ODE's; Report 100; Department of Computer Science, University of Toronto: Toronto, Canada, 1976.
IE000602R
