4136
Ind. Eng. Chem. Res. 1995,34, 4136-4139
Use of Piecewise Linear Continuous Optimal Control for Time-Delay Systems Rein Luus* and Xiaodong Zhang Department of Chemical Engineering, University o f Toronto, Toronto, Ontario, Canada M5S l A 4
Frank Hartig and Frerich J. Keil Chemische Reaktionstechnik, Technische Universitat Hamburg Harburg, Eissendorferstrasse 38, 0-21071 Hamburg, Germany
By using piecewise linear continuous control, rather than piecewise constant control, excellent control can be obtained with the use of a relatively small number of stages. For nonlinear systems, the use of a multipass method in iterative dynamic programming (IDP) is convenient and effective in obtaining convergence to the optimum. However, a good initial control policy, which can be readily obtained by the use of Taylor series approximation and piecewise constant control, may be necessary. Computational experience with a two-stage CSTR system shows the viability of using IDP for typical chemical engineering 'systems with time delays.
Introduction Since many engineering systems involve time delays, for feasibility studies and control it is important to determine the optimal control of such systems. Numerous approaches for the optimal control of systems exhibiting time delays have been used, and further developments in this area are continuing, especially for the case where state constraints are present, as considered recently by Kaji and Wong (1994). Many of these methods seek a piecewise constant control policy. The use of piecewise linear continuous control instead of piecewise constant control, however, yields a considerably better approximation to the optimal control policy, if the optimal control policy is a smooth continuous function. Even if the optimal control policy is discontinuous, as in the case of a fed-batch reactor, piecewise linear control policy can still give a very good operating procedure (Luus, 1993a). For very high dimensional problems, it is a practical decision to use a piecewise linear, rather than piecewise constant, control, so that as few stages as possible can be used to give an adequate approximation. Recently, Luus (199313) showed that, for the optimal control of a chemical reactor, the use of seven piecewise linear control segments gave better results than the use of piecewise constant control with 40 stages. A convenient way of obtaining piecewise linear continuous control is through the use of iterative dynamic programming (IDP) as developed by Luus (1989,1990). The advantages of IDP over other methods are the ease of programming and the greater likelihood of obtaining the global optimum of highly nonlinear systems, as was shown by Luus et al. (1992). When dealing with systems having time delays, we may encounter convergence difficulties with highly nonlinear systems, especially when piecewise linear continuous control is specified. As was shown by Dadebo and Luus (1992) using the piecewise constant control policy, obtaining convergence with time-delay systems is more difficult than with nondelay systems, and a sufficiently large number of grid points had to be
used for convergence. Using piecewise linear continuous control, we are restricted to the use of a single grid point at each time stage to preserve continuity. To illustrate the computational aspects of IDP in establishing piecewise linear continuous optimal control for typical chemical engineering systems, we consider a two-stage nonlinear CSTR system with a time delay. This system was used for optimal control studies by Oh and Luus (1976), by Jimenez-Romero and Luus (19911, and by Dadebo and Luus (1992). The system is described by the four time-delay differential equations
+
where x1 and x3 are normalized concentration variables in tanks 1 and 2, respectively, and x2 and x4 are normalized temperature variables in tanks 1 and 2, respectively. The reaction terms in tanks 1 and 2 are given by
R, = [x,(t)
+ 0.251)exp : : ; :x(
and the initial state profiles for this time-delay system are given by x,(t) = 0.15,
* Author to whom correspondence should be addressed. E-mail:
[email protected]: 416-978-8605. 0888-5885/95/2634-4136$09.00/0
+
dx,(t) - x,(t--t) - 2x4(t)- u,(t) [x4(t) 0.251 dt R, - 0.25 (4)
--
x,(t) = -0.03,
--t
I t I0
-Z I tI 0
0 1995 American Chemical Society
(7)
Ind. Eng. Chem. Res., Vol. 34, No. 11, 1995 4137 x&O) = 0.10 0'4
x&O) = 0 The problem is t o find the controls ul(t) and uz(t)in the time interval 0 It 2, t o minimize the performance index
L 1 t
(8) where the normalized final time tf is specified as 2.0. To solve this optimal control problem, we first divide the given time interval (0,tf)into P subintervals (O,tl), ( t 1 , t 2 ) , ...,(tp-l,tp), each of length L, so that
L = tdP
(9)
and then seek a piecewise linear continuous control policy, so that in the time interval (tk,tk+l)the control is given by
-"._
0.0
0.5
1.o
1.5
2.0
TIME
Figure 1. Optimal control for the nondelay system, using 40 stages of piecewise constant control.
where u ( k )is the value of u at the time t k and u(kf1) is the value of u at time t k t l . The optimal control problem then is to find u(k),K = 0, 1, ..., P , such that the performance index in eq 8 is minimized. At the last stage, instead of specifying a constant control policy as was done by Luus (1993a), we determine both u(P-1) and UP). This means that for the last stage we must determine four values for the control variables. Although the use of iterative dynamic programming (IDP) poses no conceptual difficulties for establishing piecewise linear continuous control for time-delay systems, for highly nonlinear problems, the use of a single grid point may lead to convergence difficulties if a good initial control policy is not available. To overcome any difficulties, a good initial guess may be necessary, and a multipass procedure may have to be adopted in IDP as was done by Luus (1993b)in determining the optimal control of a system described by 130 ordinary differential equations and having 130 control variables.
Table 1. Effect of the Number of Grid Points and the Number of Allowable Values for Control on the Performance Index for the Nondelay System Using 15 Passes, Each Consisting of 30 Iterations no. of allowable values for control no. of grid points N 3 11 5 0.02431 0.02429 0.02344 7 0.02329 9 0.02341 0.02326 11 0.02328 0.02325 15 0.02326 0.02325 0.02326 19 0.02325 21 0.02325 41
-dX3 _ - x1(t)- x 3 ( t )- t f , ( t ) - R, + 0.25 dt
d"4 = x 2 ( t )- 2x,(t) - u2(t)[x4(t)+ 0.251 dt +
Numerical Results The computations were done in double precision on a 486/66 personal computer using the WATCOM 9.5 FORTRAN compiler. For solving the time-delay differential equations, we used the subroutine RETARD give by Hairer et al. (1987). Several preliminary runs showed that convergence to the optimum could not be obtained with an arbitrarily chosen initial control policy. Therefore, to obtain a good initial control policy, we consider the nondelay system obtained by using a truncated Taylor series expansion for the delay terms, as used by Mutharasan and Luus (1975) for analysis of time-delay systems. If the delay t is sufficiently small, the original system can be approximated by the nondelay system of equations
(11)
(13)
tf,(t) R, - 0.25 (14) By using these nondelay differential equations with z = 0.1, we now seek a piecewise constant control policy with P = 40 stages. Although for many systems the use of a very small number of grid points with IDP yields convergence to the global optimum (Bojkov and Luus, 1993),for others, a relatively large number of grid points is required as shown by Luus (1994). For this system, as is shown in Table 1, we need at least 21 grid points t o obtain convergence to I = 0.023 25 if three allowable values of control are used and 11grid points if 11allowable values for control are used. For these runs the initial value of control was chosen t o be do) = 0, with do)= 0.5. The region contraction factors were y = 0.85 and 9 = 0.70. A maximum of 15 passes of 30 iterations each was allowed. The piecewise constant control policy for this nondelay system is given in Figure 1 and serves as a good starting control policy for the original system. When this control policy was used with the original time-delay system, a performance index of 0.023 18 was obtained. By fitting a piecewise linear control policy with 10 stages of equal length to the control policy in
4138 Ind. Eng. Chem. Res., Vol. 34, No. 11, 1995
0.0
0.5
1.o TIME
1.5
-0.4l 0.0
2.0
'
'
0.5
I
1.o TIME
,
I
1.5
,
I
l
2.0
Figure 4. Seven-stage piecewise linear continuous optimal control policy with r = 0.1; I o = 0.023 18.
Figure 2. Ten-stage piecewise linear continuous optimal control with r = 0.1: Io = 0.0317.
- 0 . 2 1
-0.041 0.0
'
I
0.5
I
1.o TIME
/
-0.4
I
1.5
Figure 3. State trajectories resulting from the application of the 10-stage piecewise linear continuous control policy: ( 0 )XI; (+I ~ 2 (A)X 3 ;
0.0
2.0
(a)X4.
Figure 1, we have a good initial control policy for the piecewise linear continuous control using P = 10 stages. By taking an initial region size of 0.05 for each control variable and using 100 randomly chosen candidates for control, we obtained convergence from the initial value of the performance index of 0.0320 to the value 0.023 17 in four passes, each consisting of 30 iterations. We used a region contraction factor y = 0.85 for each iteration and a reduction factor 7 = 0.70 after each pass. The computation time for each pass consisting of 30 iterations was 10 min on the 486166 personal computer. By using 200 randomly chosen candidates for control convergence to I = 0.023 17 was obtained in only two passes. The optimal control policy is given in Figure 2, and the corresponding state trajectories are shown in Figure 3. By using the control policy for 10 stages as an initial 'control policy for 7 stages, convergence to 0.023 18 was
;
0.5
1.o TIME
1.5
2.0
Figure 5. Seven-stage piecewise linear continuous optimal control policy with 5 = 0.4; P = 0.024 64.
obtained, yielding the control policy in Figure 4. Therefore, the use of only seven linear sections yielded the same value for the performance index as 40 stages with piecewise constant control. When this control policy for z = 0.1 was used as an initial control policy for t = 0.4 and with an initial control region size of 0.2 for each control variable, convergence t o the performance index 0.024 64 was obtained in seven passes with the use of 100 randomly chosen candidates for the control. By using 200 randomly chosen candidates for control, convergence to the same performance index was obtained in only three passes. The 7-stage optimal control policy for z = 0.4 is given in Figure 5. It is interesting to note that for both of these time delays, even with only 7 stages, the results are slighly better than those obtained by control vector iteration by Oh and Luus (19761, and provide a noticeable improvement over the piecewise constant control policy obtained by Dadebo and Luus (1992).
Ind. Eng. Chem. Res., Vol. 34,No. 11, 1995 4139
Conclusions
Literature Cited
When the optimal control policy is relatively smooth, a small number of stages give excellent results when piecewise linear continuous control is used. To obtain convergence, a good initial control policy may be necessary. Such a starting control policy is readily obtained by using the Taylor series approximation to convert a delay system with small time delay to a nondelay system and then solving this easier problem by using piecewise constant control. The use of continuation in progressively increasing the time delay can then be used for any specified time delay. Since there are no auxiliary variables to be determined, iterative dynamic programming is easy to use and the computations are readily carried out on a personal computer.
Bojkov, B.; Luus, R. Evaluation of the Parameters Used in Iterative Dynamic Programming. Can. J . Chem. Eng. 1993,71, 451-459. Dadebo, S.;Luus, R. Optimal Control of Time-Delay Systems by Dynamic Programming. Optim. Control Applic. Methods 1992, 13,29-41. Jimenez-Romero, L. F.; Luus, R. Optimal Control of Time-Delay Systems. Proceedings of the 1991 American Control Conference. Boston, MA, June 26-28, 1991; American Control Council, IEEE Service Centre, Piscataway, NJ, 1991; pp 1818-1819. Hairer, E.; Norsett, S. P.; Wanner, G. Solving Ordinary Differential Equations I, Springer Series in Computational Mathematics, Springer: Berlin, 1987; Vol. 8. Kaji, K.; Wong, K. H. Nonlinearly Constrained Time-Delayed Optimal Control Problems. J . Optim. Theory Applic. 1994,82, 295-313. Luus, R. Optimal Control by Dynamic Programming Using Accessible Grid Points and Region Contraction. Hung. J . Znd. Chem. 1989,17,523-543. Luus, R. Optimal Control by Dynamic Programming Using Systematic Reduction in Grid Size. Znt. J . Control 1990,51, 995-1013. Luus, R. Piecewise Linear Continuous Optimal Control by Iterative Dynamic Programming. Znd. Eng. Chem. Res. 1993a,32, 859-865. Luus, R. Application of Iterative Dynamic Programming to Very High-Dimensional Systems. Hung. J . Znd. Chem. 1993b,21, 243-250. Luus, R. Optimal Control of Batch Reactors by Iterative Dynamic Programming. J. Process Control 1994,4 , 218-226. Luus, R.; Dittrich, J.; Keil, F. J. Multiplicity of Solutions in the Optimization of a Bifunctional Catalyst Blend in Tubular Reactor. Can. J . Chem. Eng. 1992,70,780-785. Mutharasan, R.; Luus, R. Analysis of Time-Delay Systems by Series Approximation. AZChE J . 1975,21,567-571. Oh, S. H.; Luus, R. Optimal Feedback Control of Time-Delay Systems. AZChE J. 1976,22,140-147.
Acknowledgment Financial support from the Natural Sciences and Engineering Research Council of Canada (Grant A-3515) is gratefully acknowledged.
Nomenclature f = nonlinear function of x , x ( t - r ) , and u (n x 1) I = performance index L = length of stage m = number of control variables n = number of state variables N = number of grid points P = number of stages r = region over which the control is taken ( m x 1) Ri = reaction term t = time tf = final time u = control vector ( m x 1) x = state vector (n x 1) Greek Letters y = region contraction factor used after every iteration 7 = initial region size reduction factor used from pass to pass z = delay time
Received for review April 3, 1995 Revised manuscript received July 25, 1995 Accepted August 23, 1995@ IE950216N
Abstract published in Advance A C S Abstracts, October 15, 1995. @
