Ind. Eng. Chem. Res. 1999, 38, 2345-2349
2345
A Neural-Network-Based Nonlinear Controller Using an Extended Kalman Filter Furong Gao,* Fuli Wang, and Mingzhong Li Department of Chemical Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong
A neural-network-based control scheme is presented to control unknown nonlinear dynamic processes. A multilayered feedforward neural network is used to model the process dynamics, and the control input is estimated by an extended Kalman filter based on the neural model. The effectiveness of the proposed scheme is verified by both simulations and experiments. 1. Introduction Because of their exceptional ability in approximating an arbitrary nonlinear function,1 neural networks have become an attractive means for modeling complex nonlinear processes. Numerous neural network models and their corresponding learning strategies, particularly multilayered feedforward neural networks with the backpropagation (BP) learning algorithm, have been proposed and applied successfully to the identification and control of nonlinear dynamic processes in the past decade.2-4 A neural-network-based control commonly involves the process dynamics inversion, which can be direct or indirect. In a direct inversion approach, a neural network is trained to model the process inverse dynamics. The trained network model is directly cascaded with the controlled process, resulting in a composed system with identity mapping between the desired and the controlled process responses.5,6 In such a configuration, the network acts directly as the controller. The other approach, which is indirect, was initially proposed by Linden and Kindermann7 and subsequently applied to the neural-network-based controller designs by Hoskins et al.8 and Behera et al.9 In this approach, a neural network is trained to model the process dynamics, and the network model is subsequently iteratively inverted on-line by a gradient-descent search scheme to generate the control input. In this paper, an alternative on-line iterative inversion scheme is proposed. When the extended Kalman filter (EKF) is applied to the network model, the control input is estimated on-line to make the process output to track a given reference signal. This idea is an extension of the work of Liguni et al.10 They applied the EKF to a multilayered neural network for the network weight estimation. Their simulation results show that the EKF-based learning algorithm converges in fewer iterations than the gradient-descent-based algorithm. In this work, the control input, rather than the weights of the network, is estimated by the EKF. The dimension of the input is normally much smaller than the number of the network weights. Furthermore, with this arrangement, the last control input can be taken as a wellguessed initial estimate for the current control estima* Author to whom correspondence should be addressed. Telephone: +852-2358-7139. Fax: +852-2358-0054. E-mail:
[email protected].
tion. Hence, the proposed method is expected to have the advantages of simpler calculation and faster convergence. 2. Neural Network Model In this section, the neural network model of a nonlinear process is described. The process input-output dynamic relationship is captured via a forward neural network model by identifying appropriate weights for the network either off-line or on-line. Consider a multi-input and multi-output (MIMO) nonlinear process described by
y(k) ) f(y(k-1),y(k-2),...,y(k-p),u(k-1),u(k-2),..., u(k-q)) (1) where y(k) ) [y1(k), y2(k), ..., yn(k)]T ∈ Rn and u(k) ) [u1(k), u2(k), ..., um(k)]T ∈ Rm are the process output and input vectors, respectively, p and q are the process orders, and f(‚) is an unknown vector function f(x) ) [f1(x), f1(x), ..., fn(x)]T. The process of eq 1 can be represented in a compact form as
y(k) ) f(X(k-1))
(2)
where X(k-1) ) [u(k-1)T, ..., u(k-q)T, y(k-1)T, ..., y(kp)T]T ∈ Rnp+qm is a vector consisting of the process inputs and outputs. Figure 1 shows a ni-input no-output feedforward neural network with one hidden layer. It is has been shown that the network can approximate any continuous nonlinear function to an arbitrary accuracy.1 The output of the hidden units can be represented as
hˆ j ) F(sˆ j) ) F(VjTI)
j ) 1, 2, ..., nh
(3)
where VjT ) [Vj0, Vj1, ..., Vjni], IT ) [1, XT], XT ) [x1, x2, ..., xni], and F(x) ) 1/(1 + e-x) are the usual sigmoid functions. The output of the output layer neurons can be represented as
yˆ i ) WiThˆ
i ) 1, 2, ..., no
(4)
where WiT ) [Wi1, Wi2, ..., Winh] and hˆ ) [hˆ 1, hˆ 2, ..., hˆ nh]T. The neural network defines a mapping G:X f Y where X ∈ Rni is an input vector and Y ∈ Rno is an output vector. The nonlinear process dynamics of eq 1 can be approximated by the network through properly deter-
10.1021/ie980727k CCC: $18.00 © 1999 American Chemical Society Published on Web 04/27/1999
2346 Ind. Eng. Chem. Res., Vol. 38, No. 6, 1999
viewed as a state vector. From eq 8, the following nonlinear system equations can be obtained for the time instant k:
Figure 1. Feedforward neural network.
mined parameters V and W. The process predictive output can thus be obtained via the network as
yˆ (k+1) ) NN(X(k),V,W) Rnp+mq
3. EKF Estimate of Control Input For an output tracking control, the controller is designed to produce a proper control input u(k), based on the information available at the time instant k, such that the process output y(k+1) is made as close as possible to the desired output yd(k+1). If the process is modeled with sufficient accuracy by a neural network, the controller can be designed based on such a network model. 3.1. Estimate Algorithm. At any time instant k, y(k), y(k-1), ..., y(k-p+1), u(k-1), ..., and u(k-p+1) are known. The neural network output can, therefore, be represented by
yˆ (k+1) ) NN(u(k))
(6)
and the controlled process output is given by
y(k+1) ) yˆ (k+1) + ζ(k) ) NN(u(k)) + ζ(k)
(7)
where ζ(k) represents the modeling error. For a given reference signal, yd(k+1), the control input is determined to satisfy the following equation:
yd(k+1) ) NN(u(k)) + ζ(k)
(8)
The EKF is a state estimation method. The method can be applied here if the control input vector u(k) of eq 8 is
(9)
yd ) NN(ut) + ζt
(10)
In the above equations, the time index k is omitted for simplicity, and the subscript t denotes the number of the iterations during the kth control period. Following the convention in the literatures, ζt is assumed to be a white noise vector with a covariance matrix Rt. The application of the EKF to eqs 9 and 10 gives the following iterative estimation algorithm:
(5)
is the network input and yˆ (k+1) ∈ where X(k) ∈ Rn is the network output representing the process output prediction. The training of a neural network may be classified into batch learning and pattern learning. In batch learning, the weights of the neural network are adjusted after a complete sweep of the entire training data, while in pattern learning the weights are updated at every time step in an on-line fashion. Batch learning has a better mathematical validity in the sense that the gradient-descent method can be exactly implemented. Pattern learning, usually derived as batch learning approximations, can be used to modify the network weights on-line so that the model can track a timevarying process dynamics. In this paper, batch learning is used for off-line training of the network to obtain the initial weights, and pattern learning is used for on-line updating of the weights. For simplicity, the standard BP algorithm4 is employed to train the weights both offline and on-line.
ut+1 ) ut
uˆ t ) uˆ t-1 + Kt(yd-NN(uˆ t-1))
(11)
Kt ) Pt-1At-1T(At-1Pt-1At-1T + Rt)-1
(12)
Pt ) (I - KtAt-1)Pt-1
(13)
where At-1 ) (∂NN(u)/∂u)u)uˆ t-1. After some straightforward manipulations, At-1 is expressed by
h At-1 ) WTΛt-1V
(14)
where
W ) [W1, ..., Wn] Λt-1 ) diag{F(s1,t-1)(1 - F(s1,t-1)), ..., F(snh,t-1)(1 - F(snh,t-1))} V h ) [V h 1, ..., V h m]
and
V h i ) [V1,i, ..., Vnh,i], i ) 1, ..., m
Starting from the initial estimate u0 ) u(k - 1), which is the control input at the last control instant, the above estimate procedure comprising eqs 11-13 can be performed iteratively until the termination condition |yd - NN(uˆ t)| e δ is satisfied, where δ is a prespecified error-tolerance level. To prevent excessive iterations, a maximum iteration number Tmax may be set. If t > Tmax and the termination condition is still not satisfied, the estimate procedure ends with the latest obtained estimate uTmax as the control input, i.e.,
u(k) ) uˆ t, if |yd - NN(uˆ t)| e δ u(k) ) uˆ Tmax, otherwise To summarize, the control algorithm is executed in the following steps: (1) Set t ) 1, uˆ 0 ) u(k-1), and P0 ) λI (λ > 0). (2) Calculate At-1 using eq 14. (3) Compute uˆ t from eqs 11 and 12 and update Pt using eq 13. (4) Apply uˆ t to the network and compute the tracking error |yd - NN(uˆ t)|; if |yd - NN(uˆ t)| e δ or t g Tmax, then go to step 6. (5) Set t ) t + 1; go to step 2. (6) Set u(k) ) uˆ t and output u(k) to the process. Remark 1: Following the convention in the literature, the covariance matrix Rt in eq 12 is usually chosen as a constant for the EKF. In our case, Rt represents the covariance of the model error; it is thus estimated by the following recursion:11
Ind. Eng. Chem. Res., Vol. 38, No. 6, 1999 2347
Figure 2. Neural-network-based controller with a feedback compensation controller.
1 Rt ) Rt-1 + [(yd - NN(uˆ t-1))(yd - NN(uˆ t-1))T t Rt-1] (15) with an initial value R0 ) σI (σ > 0). Remark 2: It is found through simulations that with a properly selected error-tolerance level, δ, and a reasonable iteration number, Tmax, the algorithm converges. When Tmax is set to be very small, the algorithm may not converge within Tmax in certain control periods; i.e., the condition |yd - NN(uˆ Tmax)| e δ is not satisfied. In this case, acceptable control can also be achieved with uˆ Tmax as the controller output. This will be demonstrated through simulations in the next section. Remark 3: The control variable is bounded in practice. When the estimated ut exceeds the bound, the bound value is output to the process for simplicity. For the case where a higher computational requirement is allowable and higher control accuracy is required, it may be useful to incorporate control inequality constraints into the optimization via, for instance, a moving horizonbased estimation approach.12 3.2. Feedback Compensating Controller. Neural networks, theoretically, can approximate any nonlinear function to a prespecified accuracy. However, the modeling error unavoidably exists in reality and so do disturbances. To compensate for the modeling error and to reject the disturbances, a feedback compensation is introduced to the control system. The neural-networkbased controlled system together with a feedback compensation controller is graphically illustrated in Figure 2. The overall control input u consists of the feedforward control uff and the feedback control ufb. uff is determined by the iterative estimation algorithm described above, while ufb is determined by the feedback controller. The feedback controller can be designed to be a conventional PID controller or a sophisticated intelligent controller such as a fuzzy controller or an expert controller. This is not the focus of this paper. 4. Simulation and Experiment This section illustrates the validity of the proposed control strategy through its simulation application to the control of a MIMO nonlinear process and its experimental application to the temperature control of a water tank containing a strong nonlinear unit. 4.1. Simulation Result. Consider a two-input-twooutput nonlinear dynamic process described by
Figure 3. Tracking performance of the proposed algorithm: (a) setpoint yd1 and process output y1; (b) setpoint yd2 and process output y2.
[
]
[ ] y1(k)
[ ]
y1(k+1) 1 + y2(k)2 u (k) ) + 1 y1(k) y2(k) y2(k+1) u2(k)
(16)
1 + y2(k)2
A feedforward neural network with four inputs of u1(k), u2(k), y1(k), and y2(k), 15 hidden units, and two outputs yˆ 1(k+1) and yˆ 2(k+1) is chosen to learn the dynamics of eq 16. One hundred pairs of input-output training data are first generated with random input of uniform distribution from -1 to 1 and subsequently are used for the training of the network in an off-line fashion. With the trained network, the proposed algorithm with an error-tolerance level δ ) 0.05 and the maximum iteration number Tmax ) 4 is applied to control the process. Both of the process outputs are controlled to track the same sinusoidal reference signal, sin[(π/60)k]. The simulation result, as shown in Figure 3, indicates that the proposed algorithm has a satisfactory tracking ability for such a strong nonlinear MIMO process. To study the effect of Tmax on the control performance, the above simulation is repeated with different values of Tmax. Figure 4 shows the control results over the first 50 samples with Tmax ) 1-3. When Tmax ) 1, the
2348 Ind. Eng. Chem. Res., Vol. 38, No. 6, 1999
Figure 6. Nonlinear saturation link.
Figure 4. Effect of Tmax on the control performance: (a) process output y1 with Tmax ) 1-3; (b) process output y2 with Tmax ) 1-3.
Figure 7. Setpoint tracking performance comparison of EKF and PID: (a) water temperature (°C); (b) control signal (V).
Figure 5. Water tank temperature control system.
condition |yd - NN(uˆ Tmax)| e δ is violated frequently; when Tmax increases to 3, this case only happens occasionally; and when Tmax ) 4, the iteration converges within Tmax for every control period. As a result, the control performance is continuously improved as Tmax increases. It can be seen from Figure 4 that the acceptable control performance can still be obtained even when Tmax ) 1. 4.2. Experiment. A water tank system is a common component in the chemical process industry. A simplified diagram of the water tank temperature control system is given in Figure 5. The system consists of six major components: a 10-L stirred-tank, a thermal sensor and transducer, an I/O interface, a microcomputer, a 2000-W heater, and a thyristor-based actuator. The control objective is to regulate the water temperature by providing a proper dc signal to the thyristor that subsequently controls the heater. The control signal is constrained between 0 and 5 V dc. The sampling period is 25 s, and the system has a pure time delay of one sample. To test the ability of the proposed algorithm to control nonlinear processes, a nonlinear saturation link, with the functional relationship shown in Figure 6, is intentionally placed between the controller and the actuator.
A conventional PID controller is first tuned to control the water temperature at a setpoint of 40 °C with an acceptable result. However, when the setpoint is changed to 50 °C, the control performance deteriorates, and when it is changed to 60 °C, the closed-loop responses become oscillatory, as shown in the dashed line of Figure 7a. It can be seen from Figure 7b that the control signal sweeps a large portion of the saturation link of Figure 6 during the transient periods. As the link makes the process nonlinear, this results in an inconsistent PID control, as shown in Figure 7a. The same experiment is repeated with the proposed EKF-based nonlinear control. The overall process dynamics is modeled by a neural network with three inputs y(k), y(k-1), and u(k) and six hidden neurons and one output, where y(k) represents the water temperature and u(k) the dc control signal. The network is trained via batch learning with the open-loop inputoutput responses. The proposed control algorithm combined with a conventional PI feedback compensation controller is applied to the process. The error-tolerance level and the maximum iteration number are set as δ ) 0.05 and Tmax ) 4, respectively. The PI parameters are roughly tuned, because they do not critically affect the control performance. The experimental control result for setpoint tracking is shown as the solid line in Figure 7a. The process output can track the setpoints quite well, and a clear superiority of the proposed algorithm over the PID controller can be seen. To test the ability of the proposed
Ind. Eng. Chem. Res., Vol. 38, No. 6, 1999 2349
Acknowledgment This work was supported in part by HKUST under Grant RI95/96.EG03. Literature Cited
Figure 8. Disturbance rejection responses with the proposed algorithm.
algorithm to reject disturbances, a disturbance is produced by pouring 3 L of cold water into the tank at the 100th sample, and the corresponding response is shown in Figure 8. The experimental results show that both good setpoint tracking and disturbance rejection can be obtained with the proposed control strategy for the nonlinear process.
5. Conclusion A neural-network-based control algorithm using the extended Kalman filter has been proposed in this paper with successful applications to nonlinear processes. The algorithm has relatively simple calculations and fast convergence. Furthermore, the key parameter (the gain matrix Kt) governing the control performance is automatically determined by the algorithm itself. All of these features make the proposed algorithm attractive to industrial process control applications.
(1) Blum, E. K.; Li, L. K. Approximation Theory and Feedforward Networks. IEEE Trans. Neural Networks 1991, 4, 511. (2) Ahmed, M. S.; Tasadduq, I. A. Neural-net Controller for Nonlinear Plants: Design Approach Through Linearisation. IEE Proc.sD: Control Theory Appl. 1994, 141, 315. (3) Jin, L.; Nikiforuk, P. N.; Gupta, M. M. Direct Adaptive Output Tracking Control Using Multilayered Neural Networks. IEE Proc.sD: Control Theory Appl. 1993, 140, 393. (4) Narendra, K. S.; Parthasarathy, K. Identification and Control of Dynamical Systems Using Neural Networks. IEEE Trans. Neural Networks 1990, 1, 4. (5) Psaltis, D. A Multilayered Neural Network Controller. IEEE Control Syst. Mag. 1988, 17. (6) Miller, W. T.; Sutton, R. S.; Werbox, P. J. Neural Networks for Control; MIT Press: Cambridge, MA, 1990. (7) Linden, A.; Kindermann, J. Inversion of Multilayer Nets. Int. Joint Conf. Neural Networks 1989, 425. (8) Hoskins, D. A.; Hwang, J. N.; Vagners, J. Iterative Inversion of Neural Networks and Its Application to Adaptive Control. IEEE Trans Neural Networks 1992, 3, 292. (9) Behera, L.; Gopal, M.; Chaudhury, S. Inversion of RBF Networks and Applications to Adaptive Control of Nonlinear Systems. IEE Proc.sD: Control Theory Appl. 1995, 142, 617. (10) Liguni, Y.; Sakai, H.; Tokumaru, H. A Real-time Learning Algorithm for A Multilayered Neural Network Based on The Extended Kalman Filter. IEEE Trans. Signal Process. 1992, 40, 959. (11) Ljung, L.; Soderstrom, T. Theory and Practice of Recursive Identification; MIT Press: Cambridge, MA, 1983. (12) Douglas, G. R.; Lee, J. H.; James, B. R. A Moving Horizonbased Approach for Least-squares Estimation. AIChE J. 1996, 42, 2209.
Received for review November 18, 1998 Revised manuscript received March 1, 1999 Accepted March 2, 1999 IE980727K
