1030
Ind. Eng. Chem. Res. 2004, 43, 1030-1038
A Reliable Neural Network Model Based Optimal Control Strategy for a Batch Polymerization Reactor Jie Zhang* Centre for Process Analytics and Control Technology, School of Chemical Engineering & Advanced Materials, University of Newcastle, Newcastle upon Tyne NE1 7RU, U.K.
One of the most important issues of empirical model based batch process optimal control is that the calculated optimal control profile can degrade very significantly when applied to the actual process because of model-plant mismatches. “Optimal on the model” can be quite different from “optimal on the process”. To address this issue, this paper presents a reliable optimal control method where the optimization objective function includes an additional term to penalize wide model prediction confidence bounds at the end point of a batch. Bootstrap aggregated neural networks are used to model a batch polymerization reactor from limited batches of process operational data. The model can predict the number-average molecular weight, weight-average molecular weight, and monomer conversion at several points during a batch from the batch recipe and reactor temperature profile. A further advantage of bootstrap aggregated neural network models is that model prediction confidence bounds can be obtained. By penalization of wide model prediction confidence bounds at the end point of a batch, the calculated optimal control profile is much more reliable in the sense that, when it is applied to the actual process, the degradation in the control performance is limited. 1. Introduction Polymer production facilities face increasing pressure for production cost reduction and more stringent quality requirements. Appropriate process control technology and optimization provide leverage points for cost reduction and improvement in product uniformity by enabling processes to be operated close to economic and plant constraints. Achieving the specified quality of the desired product is a more complex issue in polymerization than in more conventional short-chain reactions. This is due to the fact that the molecular architecture of the polymer is so sensitive to reactor operating conditions. Upsets in feed conditions, mixing, and reactor temperature can alter critical molecular properties such as the molecular weight distribution, copolymer composition distribution, chain sequence distribution, and degree of branching distribution.1 Although computer monitoring and control have been applied to the polymerization industry, applications of advanced control methods to polymerization reactors has been largely limited to the control of temperature, pressure, and polymer melt flow index. The lack of progress in closed-loop control of polymer quality is due to a variety of reasons such as the highly nonlinear dynamics in polymerization reactors, the difficulty in the formulation of a meaningful objective function in terms of end-use properties of polymer products, and the lack of sensors to provide on-line measurements of polymer properties. Basically, two types of optimization problems are encountered in polymerization reactors.1 The first is the static optimization problem, which deals with the selection of the optimum time-invariant controls so that, without disturbance, the molecular properties attain some desired values. The second is the time optimal * To whom correspondence should be addressed. Tel.: +44-191-2227240. Fax: +44-191-2225292. E-mail: jie.zhang@ newcastle.ac.uk.
control problem, which refers to the determination of the optimal control trajectories, to translate a polymerization process from its initial state to a desired final one. The former problem is usually related to the steadystate operation of continuous polymer reactors, while the latter is concerned with the dynamic operation of batch and semibatch reactors and the start-up policies for continuous reactors. As pointed out by Kiparissides,1 the general solution of these optimization problems can be obtained in terms of the following four elements: (1) an accurate model of the process; (2) a selected number of control variables; (3) an objective function; (4) a suitable numerical method for solving the specified optimization problem. The core of the optimal control problem is an accurate model of the process. Earlier researches on optimal control of polymerization reactors utilize mechanistic models of polymerization processes.2-8 However, mechanistic models of polymerization processes are usually very complicated and difficult to develop. This has limited the practical applications of mechanistic model based optimal control strategies. To overcome the difficulty in developing mechanistic models, neural network models based upon process operation data can be developed. Neural networks have been shown to be able to approximate any continuous nonlinear functions9-11 and have been applied in process modeling and control.12-15 To build an accurate neural network model, ideally a large set of training data should be available. When the training data are limited, conventional network training often tends to overfit the training data and results in poor network generalization capabilities. Because of the difficulties in measuring the number- and weight-average molecular weights, operating data from a polymerization reactor are usually not abundant. The innovative nature of certain specialty polymers also leads to limited batch runs for a particular grade of product. An efficient approach to building an accurate model from a limited data set is to develop several neural net-
10.1021/ie034136s CCC: $27.50 © 2004 American Chemical Society Published on Web 01/22/2004
Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004 1031
works and then combine them.16-19 Zhang et al.18,19 used stacked neural networks for inferential polymer quality estimation and polymer quality prediction in a batch polymerization reactor. Tian et al.20 used stacked neural networks in developing hybrid neural network models for a batch polymerization process. An advantage of aggregated neural networks is that model prediction confidence bounds, which indicate prediction reliability, can be calculated from individual network predictions. Another issue in batch polymerization optimal control is that the “optimal” control profile may not give optimal performance when applied to the actual process because of model-plant mismatches; i.e., “optimal on the model” may not be “optimal on the process”. Some techniques have been developed to address this issue. Ruppen et al.21 proposed a method where a discrete probability distribution of some uncertain model parameters in a mechanistic model is assumed and used in a differential/ algebraic optimization problem. A difficulty associated with this technique is that a mechanistic model needs to be first developed and the distribution of uncertain model parameters needs to be identified. Terwiesch et al.22 proposed a technique to optimize a probabilistic measure of success that accounts for possible uncertainties or variations in model parameters. This technique also requires the probability distribution of uncertain model parameters. The technique is suitable for situations where a small number of parameters in a mechanistic model are uncertain and their probability distributions can be identified. An empirical data based model, such as a neural network model, usually contains a large number of model parameters, and it is generally very difficult to identify the probability distributions of these model parameters. To enhance the reliability of empirical model based optimal control, this paper presents a bootstrap aggregated neural network model based optimal control approach where the optimization objective function is modified by including a term to penalize wide model prediction confidence bounds at the end of a batch. The result of this is that the model predictions under the optimal control are reliable and, hence, “optimal on the model” is highly likely to be or to be close to “optimal on the process”. Bootstrap aggregated neural networks are employed to model the relationship between a trajectory of control actions (reactor temperature setpoints) and polymer quality variables at several time points in the later stage of a batch. Model prediction confidence bounds are obtained from the predictions of the individual networks and used in the optimization. Optimization is carried out by using each of these time points as a possible batch ending time, and the most appropriate batch ending time is selected based on the optimization results. Thus, the proposed method does not assume that all batch runs are of the same length and can lead to batches of unequal lengths. This paper is structured as follows. Section 2 presents a batch polymerization process. Modeling of the batch polymerization process using bootstrap aggregated neural networks is presented in section 3. A reliable optimal control strategy and its application to the batch polymerization process are detailed in section 4. Finally, section 5 concludes this paper. 2. Batch Polymerization Reactor The simulated batch polymerization reactor studied here is based on a pilot-scale polymerization reactor
Figure 1. Batch polymerization reactor.
installed at the Department of Chemical Engineering, Aristotle University of Thessaloniki, Thessaloniki, Greece. The reaction is the free-radical solution polymerization of methyl methacrylate (MMA) with a water solvent and benzoyl peroxide initiator. A schematic diagram of the reactor is shown in Figure 1. The reactor is provided with a stirrer for thorough agitation of the reacting mixture. Heating and cooling of the reacting mixture are achieved by circulating water at an appropriate temperature through the reactor jacket. The reactor temperature is controlled by a cascade control system consisting of a primary proportional-integral-derivative (PID) and two secondary PI controllers. The reactor temperature is measured through a temperature sensor (TT) and fed back to the primary controller whose output is taken as the setpoint of the two secondary controllers. The two secondary controllers manipulate the cold water (CW) and the hot water (HW) flow rates so that the mixed water is circulated through the reactor jacket at appropriate temperatures. The temperature of the circulating water at the jacket exit is measured through a temperature sensor (TT) and fed back to the two secondary controllers. A general description of the reactions during the freeradical solution polymerization of MMA initiated by benzoyl peroxide is as follows:
Initiator decomposition kd
I 98 2R0 Initiation ki
R0 + M 98 R1 Propagation kp
Rx + M 98 Rx+1 Transfer to monomer km
Rx + M 98 Px + R1 Transfer to solvent ks
Rx + S 98 Px + R1 Termination by disproportionation ktd
Rx + Ry 98 Px + Py Termination by combination ktc
Rx + Ry 98 Px+y
1032 Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004
a weighted combination of the individual neural network outputs. This can be represented by the equation n
f(X) )
Figure 2. Bootstrap aggregated neural network.
In the polymerization process, initiator I is decomposed into initiator radical R0. The initiator radical R0 reacts with monomer M, and a radical R1 of length 1 is generated. Monomer M is added onto the end of the radical Rx of length x, forming a new radical Rx+1 of length x + 1. The chains of radical Rx are transferred to monomer M and solvent S, forming dead polymers Px and radicals R1 of length 1. Termination by disproportionation generates polymers Px and Py, while termination by combination generates polymers Px+y. In the above reaction schemes, kd, ki, kp, km, ks, ktd, and ktc are reaction kinetic rate constants and their values can be found from refs 23 and 24. The dynamic equations for this polymerization process can be found in refs 23 and 24. A detailed mathematical model covering reaction kinetics and heat and mass balances has been developed for the bulk polymerization of MMA.23,24 On the basis of this model, a rigorous simulation program was developed and used as the real process to generate polymerization data under different batch operating conditions and to test the developed control strategies. 3. Modeling of the Batch Polymerization Process Using Neural Networks 3.1. Bootstrap Aggregated Neural Networks. A limitation of neural network models is that they can lack generalization when applied to unseen data. Several techniques have been developed to improve the neural network generalization capability, such as regularization,25 early stopping,26 Bayesian learning,27 training with both dynamic and static process data,28 and combination of multiple networks.16-19 Among these techniques, the combination of multiple networks is a very promising approach to improving model predictions on unseen data. The emphasis of this approach is on the generalization accuracy on future predictions (i.e., predictions on unseen data). When neural network models are built, it is quite possible that different networks perform well in different regions of the input space. By the combination of multiple neural networks, the prediction accuracy on the entire input space could be improved. A diagram of bootstrap aggregated neural networks is shown in Figure 2, where several neural network models are developed to model the same relationship. Instead of selection of a “best” single neural network model, these individual neural networks are combined together to improve the model accuracy and robustness. The overall output of the aggregated neural network is
wi fi(X) ∑ i)1
(1)
where f(X) is the aggregated neural network predictor, fi(X) is the ith neural network, wi is the aggregating weight for combining the ith neural network, n is the number of neural networks, and X is a vector of neural network inputs. Proper determination of the stacking weights is essential for good modeling performance. Because the individual neural networks are highly correlated, appropriate stacking weights could be obtained through principal component regression (PCR).18,19 Another advantage of a bootstrap aggregated neural network is that model prediction confidence bounds can be calculated from individual network predictions.29 The standard error of the ith predicted value is estimated as
σe )
{
1
n
}
[y(xi;Wb) - y(xi;‚)]2 ∑ n - 1b)1
1/2
(2)
n y(xi;Wb)/n and n is the number of where y(xi;‚) ) ∑b)1 neural networks. Assuming that the individual network prediction errors are normally distributed, the 95% prediction confidence bounds can be calculated as y(xi;‚) ( 1.96σe. A narrower confidence bound, i.e., smaller σe, indicates that the associated model prediction is more reliable. 3.2. Modeling of the Batch Polymerization Reactor. The maximum batch time for this reactor is about 180 min. It is assumed in this study that the possible batch ending time can range from 60 to 180 min. Because polymer quality variables are difficult to measure on-line, only a few samples of polymer quality measurements are collected during a batch. In this study, it is assumed that samples of the monomer conversion and the number- and weight-average molecular weights are collected at 20 min intervals after 60 min from the start of a batch. Thus, during a batch, up to seven samples of molecular weights are collected. The control variables considered here are the initial reactor temperature setpoint for the time interval [0 min, 40 min] and the reactor temperature setpoints at the following time intervals: [40 min, 60 min], [60 min, 80 min], ..., [160 min, 180 min]. These reactor temperature setpoints form a control trajectory for the reactor. A neural network model for the prediction of polymer quality variables at time tN is then of the form
Y(tN) ) f[I0,U(tN)]
(3)
where Y(tN) ) [Conv(tN), Mn(tN), Mw(tN)]T, U(tN) ) [Tsp0, Tsp1, Tsp2, ..., TspN]T, Tsp0 to TspN are the trajectories of reactor temperature setpoints in the time interval [0, tN], I0 is the initial initiator weight, and Conv(tN), Mn(tN), and Mw(tN) are the monomer conversion, numberaverage molecular weight, and weight-average molecular weight at time tN, respectively. In the above model, tN can take one of the following values: 60, 80, 100, 120, 140, 160, and 180 min. When using this model in optimal control, each of these values is considered as a possible batch ending time and the best batch ending time is selected based on the optimization results.
Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004 1033
4. Reliable Optimal Control through the Incorporation of Model Prediction Confidence Bounds The objective in optimum batch polymerization operation is to produce a maximum amount of polymer product with desired quality within a short time. To improve the reliability of the optimal control strategy, the following optimization problem with a modified objective function is proposed in this study:
min J ) [1 - Conv(tf)]2 + wtf + λTσe(tf) U, tf
(4)
s.t. 0.85 e Mn(tf)/Mnd(tf) e 1.15 2 e Pd(tf) e 3 Figure 3. Model errors of individual networks.
To “simulate” the building of neural network models in an industrial environment, 50 batches were simulated with controls generated from Monte Carlo simulation. The sampled data were corrupted with typical measurement noises. From the generated data, bootstrap resampling with replacement30 was used to generate 30 replications of the data. For each resampled data set (a replication of the original data), seven neural network models in the form of eq 3 for predicting the polymer quality at the seven discrete time points were developed. Each of the neural networks contains 10 hidden neurons, and the network weights were initialized as random numbers in the range (-0.1, +0.1). The networks were trained using the Levenberg-Marquardt optimization algorithm with regularization and crossvalidation based “early-stopping”. Because of the different magnitudes of the model input and output data, the data for neural network training have to be scaled first. In this study, Mn is scaled down by a factor of 105, Mw is scaled down by a factor of 106, I0 is scaled down by a factor of 3, and the reactor temperature setpoint is scaled through (Tsp - 338)/20. The 30 individual networks for predicting the polymer quality at a particular time tN were then combined through PCR. A further 20 batches were simulated to generate a set of unseen data to validate the developed neural network models. Note that the model represented by eq 3 can be trained using historical process operation data with unequal batch lengths. This feature is desirable because unequal batch lengths are common in practice. Figure 3 shows the scaled sum of squared errors (SSEs) of the individual networks on training and validation data sets. It can be observed that the performances of these networks on the training and validation data sets are not consistent. A network having small SSEs on the training data set may have quite large SSEs on the validation set. This indicates the nonrobust nature of a single neural network model. The minimum SSEs of individual networks on the training and validation data sets are 18.0 and 19.0, respectively. The SSEs from the aggregated network on the training and validation data sets are 9.8 and 13.8, respectively. Thus, the model accuracy is significantly improved by combining multiple imperfect models.
324 e ui e 352, 1 e i e 8 where U ) [u1, u2, ...]T is a vector of control actions (reactor temperature setpoints), u1 is the reactor temperature setpoint for the time interval [0 min, 40 min], u2-u8 are the reactor temperature setpoints for the time intervals [40 min, 60 min], ..., [160 min, 180 min], Conv is the predicted monomer conversion, tf is the batch ending time, w is a weighting factor for batch duration (w ) 0.0001 min-1 in this study), σe ) [σ1, σ2, σ3]T is a vector of standard prediction errors in Mn, Mw, and Conv, respectively, λ ) [λ1, λ2, λ3]T is a vector of weightings for σe, Mn is the predicted number-average molecular weight, Mnd is the desired value of Mn, and Pd ()Mw/Mn) is the polydispersity of the polymer. This objective function maximizes the monomer conversion and minimizes the batch time subject to the desired molecular weight distribution constraints. A higher monomer conversion represents better utilization of the raw materials, and a shorter batch duration time means more production for a given time period. Minimization of the above objective function represents a high efficiency in polymer production and maximal utilization of the row materials. The last term in the objective function, eq 4, is to penalize wide model prediction confidence bounds. Thus, neural network model prediction under the optimal control policy calculated by minimizing eq 4 will have a narrow prediction confidence bound. This means that these model predictions under the calculated optimal control policy are reliable and so is the calculated optimal control policy. Because model-plant mismatches are unavoidable, the constraints on product quality variables used in eq 4 should be soft constraints, which are allowed to be violated to certain extents. This is because, although the model predicted product quality variables satisfy their constraints, the actual ones may not because of model-plant mismatches. The hard constraints on product quality variables, which should not be violated, are considered as follows in this study:
0.8 e Mn(tf)/Mnd(tf) e 1.2
(5)
1.8 e Pd(tf) e 3.2
(6)
The differences between the soft and hard constraints represent a backoff reflecting the extent of the modelplant mismatches. If the model-plant mismatches are small, then the backoff can be small, and vice versa.
1034 Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004 Table 1. Five Case Studies case
λT
case
λT
1 2 3
[0, 0, 0] [0.05, 0.05, 0.05] [0.1, 0.1, 0.05]
4 5
[0.15, 0.15, 0.05] [0.2, 0.2, 0.05]
Table 2. Minimal Objective Function Values at Different Batch Ending Times batch ending time (min) case
60
80
100
120
140
160
180
1 2 3 4 5 single network
0.0396 0.0640 0.0830 0.1019 0.1209 0.0919
0.0212 0.0491 0.0611 0.0719 0.0812 0.0204
0.0158 0.0361 0.0458 0.0546 0.0630 0.0104
0.0249 0.0684 0.1081 0.1471 0.1859 0.0125
0.0329 0.0508 0.0662 0.0816 0.0969 0.0183
0.0266 0.0444 0.0527 0.0602 0.0677 0.0268
0.0280 0.0427 0.0544 0.0634 0.0724 0.0183
Because the neural network model only predicts polymer qualities at 20 min intervals starting from the 60th min into reaction, it is considered in this study that the batch ending time only takes one of the following seven values: 60, 80, ..., and 180 min. The optimization problem is solved by considering each of the possible batch ending times and selecting the one resulting in the smallest objective function value. Thus, this method does not require all batch runs of the same length and can lead to batch runs with unequal lengths. If the considered batch ending time is 60 min, then the calculated optimal control profile U contains two elements, u1 and u2, whereas if the considered batch ending time is 180 min, then the calculated optimal control profile U contains eight elements, u1-u8. The calculated optimal temperature setpoints are passed to the cascade temperature controller, which controls the reactor temperature by manipulating the cold water and hot water flow rates as shown in Figure 1. Because tf can only take one of the seven values, constraints on tf are implicitly enforced. At each of the seven values of tf, the following optimization problem is solved. The minimal objective function values at the 2 T min J ) [1 - Conv(tf)] + wtf + λ σe(tf) U
(7)
s.t. 0.85 e Mn(tf)/Mnd(tf) e 1.15 2 e Pd(tf) e 3 324 e ui e 352, 1 e i e 8 seven values of tf are then compared. The optimal reactor temperature control profile U and the batch ending time tf corresponding to the smallest minimal objective function values are selected as the final optimal solution to the optimization problem defined in eq 4.
Remark. The proposed approach is similar to the dual control method in dynamic optimal control systems where the parameter uncertainty in the observation equation of the linear quadratic Gaussian problem is considered in the optimal control objective function.31 Although the idea of the dual control problem is simple, the required analysis is rather involved and is still a research topic in the field of automatic control.32 If a dynamic model of a batch process and the prior probability distributions of the uncertain parameters in the observation equation are available, then the dual control method31,32 can be used. However, developing dynamic models for batch processes and obtaining the prior probability distributions of the uncertain parameters in the observation equation are generally difficult and time-consuming. The approach proposed in this paper utilizes bootstrap aggregated neural networks to improve model prediction reliability and to provide a model prediction confidence measure, which is then used in the optimization objective function. In this study, Mnd is taken as 1.9 × 105 g/mol, corresponding to a specific grade of product. An optimal reactor temperature control profile can be obtained by solving the above optimal control problem. In this study, the optimal control problem is solved using the sequential quadratic programming (SQP) method available in the MATLAB Optimization Toolbox.33 The SQP method is very effective for constrained optimization problems. The SQP method is constructed by solving a quadratic programming (QP) subproblem at each major iteration. An approximation is made of the Hessian of the Lagrangian function using a quasi-Newton updating method. This is then used to generate a QP subproblem whose solution is used to form a search direction for a line search procedure. When the optimal reactor temperature control profile is calculated using the bootstrap aggregated neural network models, the initial initiator weight I0 is fixed to its nominal value of 2.5 g. Five cases with different values of λ were studied, and they are listed in Table 1. Table 2 presents the minimal objective function values at each possible batch ending time. It can be seen from Table 2 that the optimum batch time is found to be 100 min in all cases. The optimization and simulation results for the five cases are given in Table 3. Note that the standard prediction errors given in Table 3 are scaled values. The effect of penalization of wide model prediction confidence bounds during optimization is clearly seen in Table 3. In case 1, because the model prediction confidence bound is not penalized, neural network model predictions under optimal control have wide confidence bounds. This means that these model predictions are not reliable and, thus, the calculated optimal control may not give optimal performance when applied to the actual process. This is indeed confirmed by simulation on the mechanistic model. When this “optimal” control is applied to the actual process (i.e., the simulation on the mecha-
Table 3. Optimization and Simulation Results for the Final Product Quality neural network model T
mechanistic model
case
σe
Mn (g/mol)
Conv
Pd
Mn (g/mol)
Conv
Pd
1 2 3 4 5 single network
[0.254, 0.190, 0.063] [0.117, 0.102, 0.048] [0.091, 0.090, 0.039] [0.086, 0.085, 0.033] [0.085, 0.077, 0.028]
1.85 × 105 1.77 × 105 1.83 × 105 1.89 × 105 1.96 × 105 1.62 × 105
0.92 0.89 0.87 0.87 0.86 0.98
3.00 3.00 3.00 3.00 2.99 3.0
1.39 × 105 1.60 × 105 1.71 × 105 1.77 × 105 1.85 × 105 2.21 × 105
0.89 0.87 0.87 0.86 0.86 0.79
3.71 3.24 3.10 3.04 2.97 3.32
Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004 1035
Figure 4. Optimal control profile and reactor temperature in case 1.
Figure 6. Mn in cases 1 and 5.
Figure 7. Pd in cases 1 and 5. Figure 5. Monomer conversion in cases 1 and 5.
nistic model), the resulting product satisfies neither the soft nor the hard quality constraints on Mn and Pd. The neural network model predicted Mn, Pd, and Conv under this “optimal” control are 1.85 × 105 g/mol, 3.0, and 0.92, respectively. However, the actual Mn, Pd, and Conv under this “optimal” control are 1.39 × 105 g/mol, 3.71, and 0.89, respectively. This means that the objective function value on the actual process is much higher than that on the neural network model. Hence, there is a significant degradation in performance when this “optimal” control policy is implemented on the actual process. A significantly large backoff between the soft and hard constraints would be required in this case, which can lead to inefficient production. Figure 4 shows the optimal control profile (reactor temperature setpoints) and the actual reactor temperature in case 1. It can be noticed that the control errors exist because of the limitation of the controller. Although the temperature controller could be further improved, this is not done in this study. Because the purpose of this study is to present a reliable optimal control technique, a not well-tuned temperature controller can also help to demonstrate the proposed technique. In industrial practice, performance degradation of an “optimal” controller can also be due to inappropriately tuned lower level controllers. The trajectories of Conv, Mn, and Pd under this control profile are shown in Figures 5-7, respectively. In Figures 6 and 7, the dash-dotted lines and the dotted lines represent
respectively the hard and soft constraints at the batch end. As the weightings for the standard prediction errors are increased, the model prediction confidence bounds become narrower, and the actual final product quality moves toward the constraints. In case 2, the hard constraints on Mn are satisfied whereas the hard constraints on Pd are still not satisfied. In case 3, as the weightings are further increased, both quality variables are within their respective hard constraints. As the weightings are further increased, the soft constraints are also satisfied in case 5. Figures 8-10 show respectively the box plots of the individual neural network predictions of Mn, Mw, and Conv. The symbols “+” in these figures indicate outliers. It can be seen from Figures 8-10 that the individual networks can give diverse predictions under the “optimal” control policy calculated without penalization of wide model prediction confidence bounds (case 1). Some neural networks even predict higher than 100% monomer conversion in case 1. As the weightings for the standard prediction errors are increased, predictions from the individual networks under the calculated optimal control become less diverse, indicating improved prediction reliability. The appropriate weightings can be selected based on the standard errors of the neural network predictions. From Table 3, as well as Figures 8-10, it can be seen that the reductions in standard errors of neural network model predictions become less significant from case 3 onward. Therefore, the weightings in case 3 could be used. Indeed, the actual product quality variables in
1036 Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004
Figure 8. Box plot of the individual neural network predictions of Mn.
Figure 11. Optimal control profile and reactor temperature in case 5.
the batch end. This demonstrates that the optimal control police obtained in case 5 is much more reliable than that obtained in case 1. The proposed control strategy is compared with single neural network based optimal control, where the optimal reactor temperature was obtained by solving the following optimization problem:
min J ) [1 - Conv(tf)]2 + wtf U, tf
(8)
s.t. 0.85 e Mn(tf)/Mnd(tf) e 1.15 2 e Pd(tf) e 3 324 e ui e 352, 1 e i e 8 Figure 9. Box plot of the individual neural network predictions of Mw.
The values of w and Mnd are the same as those used in the earlier case studies. The hard constraints on product quality are given by eqs 5 and 6. The single neural network used is the first individual network of the 30 individual networks developed earlier. As can be seen from Figure 3, this network is one of the best individual networks in terms of performance on the training and validation data. Once again for each of the seven values of tf, the following optimization problem is solved using the SQP algorithm. 2 min J ) [1 - Conv(tf)] + wtf U
(9)
s.t. 0.85 e Mn(tf)/Mnd(tf) e 1.15 2 e Pd(tf) e 3 324 e ui e 352, 1 e i e 8 Figure 10. Box plot of the individual neural network predictions of conversion.
case 3 satisfy their hard constraints. Figure 11 shows the optimal control profile and the actual reactor temperature in case 5. The trajectories of Conv, Mn, and Pd under this control profile are shown in Figures 5-7, respectively. It can be seen from Figures 6 and 7 that both Mn and Pd in case 1 are significantly outside their soft and hard constraints at the batch end. However, both Mn and Pd in case 5 are within their soft and hard constraints at
The optimization results under different possible batch ending times are given in Table 2. It can be seen that the optimal batch ending time in this case is also 100 min. Thus, the “optimal” control profile corresponding to the batch ending time of 100 min is selected. Under this “optimal” control profile, the neural network model predicted and the actual (simulated from the mechanistic model) product quality variable values are given in Table 3. It can be seen from Table 3 that the neural network predicted product quality variable values are quite different from the actual values. The neural
Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004 1037
Figure 12. Optimal control profile and reactor temperature in the case of single neural network based control.
Figure 13. Mn in the case of single neural network based control.
network predicted Mn is on the lower bound of the soft constraint. However, the actual Mn exceeds the upper bound of the soft constraint. Thus, a large backoff between the soft and hard constraints would be required in this case, which could lead to inefficient production. Under this “optimal” control profile, the neural network predicted monomer conversion is 0.98 whereas the actual monomer conversion is only 0.79. Figure 12 shows the “optimal” control profile and actual reactor temperature in the case of single neural network based optimal control. It can be seen that the “optimal” control profile is quite different from those in the cases of bootstrap aggregated neural network based optimal control. Figures 13 and 14 show respectively the trajectories of Mn and Pd under this “optimal” control profile. It can be seen from Figure 14 that Pd of the final product is significantly outside its hard constraints. This means that the “optimal” control profile obtained from a single neural network will lead to off-specification products and this “optimal” control profile is far from being optimal on the actual process. Comparison of this single neural network based optimal control with the proposed approach demonstrates that the proposed approach is effective in enhancing the robustness of empirical model based control. 5. Conclusions A reliable optimal control strategy based on bootstrap aggregated neural network models is proposed for batch
Figure 14. Pd in the case of single neural network based control.
polymerization reactor control. Bootstrap aggregated neural networks are used to model batch polymerization processes using a limited amount of process operational data. A model prediction confidence bound is incorporated in the optimization objective function so that a wide confidence bound at the end of a batch is penalized. By such a means, the reliability of the calculated optimal control is significantly improved. It is shown in this study that model prediction reliability has a significant impact on model based optimal control strategies. The “optimal” control strategy calculated based on unreliable predictions may not be optimal at all on the actual process. This indicates that such a control strategy is unreliable, and it would therefore be very risky to apply it to real plants. Application of the proposed modeling and optimal control strategy to a simulated batch MMA polymerization reactor demonstrates that the proposed technique is very effective. Acknowledgment The work was supported by the U.K. EPSRC through Grants GR/N13319 and GR/R10875. The author thanks Prof. C. Kiparissides of Aristotle University of Thessaloniki, Thessaloniki, Greece, for providing the polymerization simulation program. Literature Cited (1) Kiparissides, C. Polymerisation Reactor Modelling: A Review of Recent Developments and Future Directions. Chem. Eng. Sci. 1996, 51, 1637-1659. (2) Chen, S.; Jeng, W. Minimum End Time Policies for Batchwise Radical Chain Polymerisation. Chem. Eng. Sci. 1978, 33, 735-743. (3) Jang, S. S.; Yang, W. L. Dynamic Optimisation of Batch Emulsion Polymerisation of Vinyl AcetatesAn Orthogonal Polynomial Initiator Policy. Chem. Eng. Sci. 1989, 44, 515-528. (4) Ponnuswamy, S. R.; Shah, S. L.; Kiparissides, C. Computer Optimal Control of Batch Polymerisation Reactors. Ind. Eng. Chem. Res. 1987, 26, 2229-2236. (5) Ray, W. H. Polymerisation Reactor Control. IEEE Control Syst. Mag. 1986, 6, 3-8. (6) Secchi, A. R.; Lima, E. L.; Pinto, J. C. Constrained Optimal Batch Polymerisation Reactor Control. Polym. Eng. Sci. 1990, 30, 1209-1219. (7) Takamatsu, T.; Shioya, S.; Okada, Y. Molecular Weight Distribution Control in a Batch Polymerisation Reactor. Ind. Eng. Chem. Res. 1988, 27, 93-99. (8) Thomas, I. M.; Kiparissides, C. Computation of the Nearoptimal Temperature and Initiator Policies for a Aatch Polymerisation Reactor. Can. J. Chem. Eng. 1984, 62, 284-291.
1038 Ind. Eng. Chem. Res., Vol. 43, No. 4, 2004 (9) Cybenko, G. Approximation by Superposition of a Sigmoidal Function. Math. Control Signal Syst. 1989, 2, 303-314. (10) Girosi, F.; Poggio, T. Networks and the Best Approximation Property. Biol. Cybernetics 1990, 63, 169-179. (11) Park, J.; Sandberg, I. W. Universal Approximation Using Radial Basis Function Networks. Neural Comput. 1991, 3, 246257. (12) Bhat, N. V.; McAvoy, T. J. Use of Neural Nets for Dynamical Modelling and Control of Chemical Process Systems. Comput. Chem. Eng. 1990, 14, 573-583. (13) Bulsari, A. B., Ed. Computer-Aided Chemical Engineering, Volume 6: Neural Networks for Chemical Engineers; Elsevier: Amsterdam, The Netherlands, 1995. (14) MacMurray, J. C.; Himmelblau, D. M. Modeling and Control of a Packed Distillation Column Using Artificial Neural Networks. Comput. Chem. Eng. 1995, 19, 1077-1088. (15) Morris, A. J.; Montague, G. A.; Willis, M. J. Chem. Eng. Res. Des. 1994, 72, 3-19. (16) Sridhar, D. V.; Seagrave, R. C.; Bartlett, E. B. Process Modelling Using Stacked Neural Networks. AIChE J. 1996, 42, 2529-2539. (17) Wolpert, D. H. Stacked Generalization. Neural Networks 1992, 5, 241-259. (18) Zhang, J.; Morris, A. J.; Martin, E. B.; Kiparissides, C. Inferential Estimation of Polymer Quality Using Stacked Neural Networks. Comput. Chem. Eng. 1997, 21, s1025-s1030. (19) Zhang, J.; Morris, A. J.; Martin, E. B.; Kiparissides, C. Prediction of Polymer Quality in Batch Polymerisation Reactors Using Robust Neural Networks. Chem. Eng. J. 1998, 69, 135143. (20) Tian, Y.; Zhang, J.; Morris, A. J. Modeling and Optimal Control of a Batch Polymerization Reactor Using a Hybrid Stacked Recurrent Neural Network Model. Ind. Eng. Chem. Res. 2001, 40, 4525-4535. (21) Ruppen, D.; Benthack, C.; Bonvin, D. Optimization of Batch Reactor Operation under Parametric Uncertaintys Computational Aspects. J. Process Control 1995, 5, 235-240. (22) Terwiesch, P.; Ravemark, D.; Schenker, B.; Rippin, D. W. T. Semi-batch Process Optimization under Uncertainty: Theory and Experiments. Comput. Chem. Eng. 1998, 22, 201-213.
(23) Achilias, D.; Kiparissides, C. Development of a General Mathematical Framework for Modelling Diffusion-controlled Freeradical Polymerisation Reactions. Macromolecules 1992, 25, 37393750. (24) Penlidis, A.; Ponnuswamy, S. R.; Kiparissides, C.; O’Driscoll, K. F. Polymer Reaction Engineering: Modelling Considerations for Control Studies. Chem. Eng. J. 1992, 50, 95-107. (25) Bishop, C. Improving the Generalisation Properties of Radial Basis Function Neural Networks. Neural Comput. 1991, 13, 579-588. (26) Bishop, C. Neural Networks for Pattern Recognition; Oxford University Press: Oxford, U.K., 1995. (27) MacKay, D. J. C. Bayesian Interpolation. Neural Comput. 1992, 4, 415-447. (28) Zhang, J. Developing Robust Neural Network Models by Using Both Dynamic and Static Process Operating Data. Ind. Eng. Chem. Res. 2001, 40, 234-241. (29) Zhang, J. Developing Robust Non-linear Models through Bootstrap Aggregated Neural Networks. Neurocomputing 1999, 25, 93-113. (30) Efron, B. The Jackknife, the Bootstrap and Other Resampling Plans; Society for Industrial and Applied Mathematics: Philadelphia, 1982. (31) Feldbaum, A. A. Optimal Control Systems; Academic Press: New York, 1965. (32) Li, D.; Qian, F.; Fu, P. Variance Minimization Approach for a Class of Dual Control Problems. IEEE Trans. Autom. Control 2002, 47, 2010-2020. (33) Grace, A. Optimisation Toolbox User’s Guide; The MathWorks Inc.: 1995.
Received for review September 18, 2003 Revised manuscript received December 2, 2003 Accepted December 17, 2003 IE034136S
