Recurrent Backpropagation Neural Network Adaptive Control of

3756

Ind. Eng. Chem. Res. 1997, 36, 3756-3761

Recurrent Backpropagation Neural Network Adaptive Control of Penicillin Acylase Fermentation by Arthrobacter viscosus Mei-J. Syu* and J.-B. Chang Department of Chemical Engineering, National Cheng Kung University, Tainan, Taiwan, 70101, Republic of China

A recurrent backpropagation neural network (RBPN) was proposed for the on-line adaptive pH control of penicillin acylase fermentation with Arthrobacter viscosus. It was observed that both enzyme activity and cell growth are rather sensitive to changes in pH. Hence, the control of pH during batch fermentation is a very important issue. RBPN was chosen as the controller model for its superior ability in long-term identification. The transfer function x/(1 + |x|) proposed previously was used with this RBPN controller. The output node of this network controller was the predicted flow rate for the next control time interval. Initial pump rate and base/acid concentrations were both important factors affecting the control performance. To enhance the effective on-line learning of this network, a moving-window type of training data was supplied to train the network. In conclusion, the pH was well controlled and a maximum optical density of 6.7 was achieved as well. Therefore, a test of the RBPN controller from the pH control of this fermentation was successfully performed. Introduction Batch fermentations were performed with Arthrobacter viscosus (ATCC 15294). A. viscosus can secrete extracellular penicillin acylase (E.C. 3.5.1.11), which is a key enzyme in β-lactam antibiotics (Hiroshi et al., 1989). The kinetics of the enzymatic reaction has been investigated, with most of the work published being on the catalyzed formation of 6-APA from penicillin substrate (Robinson et al., 1960). Penicillin acylase is mainly used to catalyze the deacylation reaction during the antibiotic-producing processes. It can catalyze the following reactions: penicillin acylase

cephalosporin G 98 7-ADCA penicillin acylase

cephalosporin C 98 7-ACA penicillin acylase

penicillin 98 6-APA 6-APA, 7-ACA, and 7-ADCA from the deacylation of penicillin, cephalosporin C, and cephalosporin G, respectively, are all very important compounds for the synthesis of their respective β-lactam series derivatives of antibiotics. The β-lactam family of antibiotics is the major products in the penicillin series and cephalosporin series. 7-ADCA and 7-ACA can further synthesize the cephalosporin series of antibiotics, while 6-APA can lead to the formation of different penicillins. The cephalosporins are far more important than the penicillins. As a result, 7-ADCA and 7-ACA are two major intermediates for the production of cephalosporin series antibiotics. Penicillin acylase enzyme is very important to these processes. The growth of more cells will give a higher enzyme level. In addition, cell growth and enzyme activity are both sensitive to the change of pH during batch fermentation. Therefore, to control the pH of the penicillin acylase fermentation so that the cells can secrete the enzyme with a higher activity level under such a controlled environment is the focus of this research. * To whom correspondence should be addressed. E-mail: [email protected]. S0888-5885(96)00609-4 CCC: $14.00

A batch fermentation system is usually nonlinear and dynamic in nature. A dynamic model should be used to describe the fermentation in order to precisely control the process. A neural network with adaptive learning ability on its connecting weights can identify the correlation of the desired system variables from different nonlinear systems. Hence, it was used in this work as the on-line controller for the study of the penicillin acylase fermentation system. The on-line control of the fermentation system can be performed adaptively to adjust the control parameters according to the dynamic nonlinearity of the process. Therefore, the neural network can act as the adaptive controller for the control of the process by dynamically updating its weights from learnings (Narendra and Parthasarathy, 1990). There have been reports on the neural network control of different systems. The major components of a control system are a precise process model and a good controller with respect to certain specified objectives. When neural networks are applied to control systems, they can be used purely as the process models based on their robustic mapping abilities. The network process model can also be designed in an inverse mode. If some of the systems have already had their existing models, then neural networks are only used as the controllers. Some investigations tried to merge the process model and the controller into only one neural network or two neural networks with one being the process model and the other being the controller. Shimizu (1996) gave a brief review on bioprocess system engineering with focus on the optimization and control of bioprocesses. Neuro-fuzzy control for the gene products was also introduced. In 1994, Yang and Linkens used neural networks to control a CSTR. Khalid et al. (1995) also applied neural networks to the tuning of the reaction temperature. Boskovic and Narendra (1995) proposed a neural network controller for the fed-batch fermentation process. In their work, the control results from the network controller were compared with the other four types of linear and nonlinear controllers and better performance was concluded from the network controller. Quin et al. (1994) used a supervised network as the controller to the fermentation process which was carried by Bacillus © 1997 American Chemical Society

Ind. Eng. Chem. Res., Vol. 36, No. 9, 1997 3757

thuringiensis. In 1995, Ku and Lee proposed a diagonal recurrent neural network to control the dynamic system. They came to the conclusion that a recurrent network is more superior to a feedforward network in dynamic mapping; specifically, the recurrent network can be satisfactorily applied to the real-time processes. The neural network technique was also applied to the image analysis for the plant somatic embryo culture (Uozumi et al., 1993). In this work, neural networks were provided with a dynamic learning and prediction process that moved along the time sequence batchwise. In other words, a scheme of a two-dimensional moving window (number of input nodes by the number of training data) was proposed for reading in new data while forgetting part of the old data. The training data provided in such a mode are from the consideration of saving computation time when carry out the on-line control system and are precisely identifying the system characteristics closest to the next control point. Therefore, the control action can be efficiently predicted from the network controller and sent to the system in time. When the control time is not yet due, the neural network acts as a process model for reading in a certain amount of data and then training to obtain better mapping of the system behavior; the connecting weights of the network can be adjusted along the fermentation this way. During the control phase, the network stopped learning and switched to become a controller; the control action can thus be predicted from the controller. By switching between the learning phase and the predicting phase, the control system can proceed on-line by one single neural network. The recurrent backpropagation neural network (RBPN), a recurrent mode of the backpropagation network (Caudill and Butler, 1992) with recurrent nodes added from each next neighboring layer at a previous time, is a dynamic network. Therefore, it has the inherent ability to precisely identify the dynamic time series of a longer term than a BPN. A RBPN cannot only identify the processes in the long term but can also store the temporary memory of the recurrent nodes from previous times in each learning. Therefore, its structure can be regarded as a dynamic mode in nature. The information flows of a BPN and an RBPN are different. In a BPN, it is a unidirectional forward flow from the input layer through the network structure to the output layer, while in a RBPN, the flow is no more unidirectional and there is a recurrent flow to each layer from each next-neighboring layer. Meanwhile, during the backward error learning phase, a BPN only collects errors at the output layer and learns by sending error learning signals backward from the output layer through the structure to the input layer, while a RBPN not only learns backward through the network structure but also learns backward through time. Recurrent Backpropagation Neural Network Based on the architecture of a BPN, a recurrent mode of the network can be formed. Batch learning instead of pattern learning for BPN was applied. Besides backpropagation learning through structure, RBPN also learns through time. Based on a BPN, a corresponding RBPN structure is shown in Figure 1, which was the recurrent form applied in this study. The input layer at current time contains current input nodes and previous hidden nodes recurrent from the hidden layer. Similarly, the hidden layer at current time contains

Figure 1. Recurrent backpropagation neural network.

current hidden nodes and previous output nodes recurrent from the output layer. Forward Computation through Network and through Time (Input Layer f Output Layer, t ) 1 f N). At initial time, the values of h(0)j and y(0)k have to be set first. h(0)j and y(0)k represent the initial values of the hidden nodes and output nodes, respectively. The computation proceeded forward from t ) 1 to t ) N. At each time step, summation and transformation from the input layer through the structure to the output layer proceeded forward. For t ) 1, 2, ..., N I

net(t)j )

∑ i)1

J

∑ ωjj (t - 1)h(t - 1)j

ωji(t)x(t)i +

jr)1

r

r

h(t)j ) f(net(t)j) J

net(t)k )

(1) (2)

K

ωkj(t)h(t)j + ∑ ωkk (t - 1)y(t - 1)k ∑ j)1 k )1 r

r

(3)

r

y(t)k ) f(net(t)k)

(4)

where net(t)j is the weighted sum of the current input value and the previous value of the hidden nodes and is also the net value of the current hidden node before the transformation by a transfer function. ωji(t) is the weight connecting the jth hidden node of time t, h(t)j, and the ith input node at current time t, and x(t)i ωjjr(t - 1) is the weight connecting the jth hidden node of time t, h(t)j, and the jrth recurrent node from the hidden layer at time t - 1, h(t - 1)jr, which also becomes the node added to the input layer. ωkj(t) is the weight connecting the kth output node of time t, y(t)k, and the jth hidden node of time t, h(t)j. net(t)k is the weighted sum of the current value of the hidden nodes and the previous output value and is also the net value of the current output node before the transformarion by a transfer function. ωkkr(t - 1) is the weight connecting the kth output node of time t, y(t)k, and the krth recurrent node from the output layer at time t - 1, y(t - 1)kr, which also becomes the node added to the hidden layer. Backward Learning of Error through Network and through Time (Output Layer f Input Layer, t ) N f 1). The error learning signal was transmitted from the output layer or the very end edge of layers through the hidden layer to the input layer. It was also transferred backward through time, from the very end of the time series to the very beginning, i.e., t ) N to t ) 1.

3758 Ind. Eng. Chem. Res., Vol. 36, No. 9, 1997

For t ) N, the error learning signals, δ(N)k and δ(N)j, from the output layer and hidden layer, respectively, are calculated as follows:

δ(N)k ) (Y(N)k - y(N)k)

d f(net(N)k) dnet(N)k

N

∑ ωkjδ(N)k] k)1

δ(N)j ) [

d

f(net(N)j) dnet(N)j

(5)

(6)

in which Y(N)k is the desired kth output node at time N and y(N)k is the kth computed output node at time N. f(‚) is the transfer function, and it is equal to x/(1 + |x|) (Syu and Tsao, 1994). For t ) N - 1, N - 2, ..., 1,

δ(t)k ) [(Y(t)k - y(t)k) + K

∑ ωkk δ(t + 1)k ] k )1 r

r

r

K

d dnet(t)k

J

ωjj δ(t + 1)j ] ∑ ωkjδ(t)k + j∑ k)1 )1

δ(t)j ) [

r

r

r

f(net(t)k) (7)

d dnet(t)j

f(net(t)j) (8)

Therefore, the weights of both the current time and the recurrent ones from the previous time can be updated as follows:

∆ω(t)ji ) ηδ(t)jx(t)i + R∆ω(t - 1)ji

(9)

∆ω(t)kj ) ηδ(t)kh(t)j + R∆ω(t - 1)kj

(10)

∆ω(t)jjr ) ηδ(t)jh(t - 1)jr + R∆ω(t - 1)jjr

(11)

∆ω(t)kkr ) ηδ(t)ky(t - 1)kr + R∆ω(t - 1)kkr (12) where η is the learning coefficient with respect to the gradient term and R is the coefficient of the momentum term. If a scaling factor b was used for tuning the steepness of the transfer function, for example, bx/(1 + |x|), then the update of b can also be derived. The learning coefficient governing the update of b was set as β. With the weights of a RBPN update as described above, a RBPN was imbedded into the control system as an on-line process identification and adaptive controller. During the learning phase for identification, the weights were updated according to the RBPN algorithm. Whenever the time was up for control, the control phase was turned on, and the weights stopped learning at this moment. Only after the control action was sent to the system and new data were collected again was the RBPN turned to the learning phase again. Such cycles were repeated periodically by a constant time interval until the end of the fermentation. Experimental Section On-Line System. A 1-L fermentor (Brunswick Co., New Brunswick, NJ) with on-line control of pH as shown in Figure 2 was set up. An ADDA card was imbedded in a PC 486. A pH sensor was inserted into the fermentor. The analog signals from the pH probe were delivered to the PC through the AD portion of the interface card. The neural network control program was in the PC to compute the acquired data. The

Figure 2. On-line control system of the fermentation.

computed digital control actions would be sent to the control peristaltic pumps through the DA portion of the interface. Each of the two output channels of the DA portion was connected to a pump. The pumps were used to add basic NaOH solution and acidic HCl solution, respectively, according to the control commands. Different concentrations of the buffer solutions were set in different batches of fermentations. An output voltage in the range 0-5 V was chosen for sending the analog signal from the network controller to activate the pumps. Fermentation. Batch fermentations were performed with Arthrobacter viscosus (ATCC15294). Extracellular penicillin acylase (E.C. 3.5.1.11) was produced from this fermentation. The cells were maintained by slant cultures with fresh medium replaced regularly. The cells were precultured twice before inoculating the fermentor. The cell density was measured by a spectrophotometer (UV/Vis, UV 160A, Shimadzu, Tokyo, Japan) at a wavelength of 550 nm. The linear range for the optical density (OD) measurement of the cells vs dry cell weight as well as diluted cell density required calibration. Therefore, before each measurement of the fermentation broth for cell density, the broth solution had to be properly diluted within the range of 0.435. Results and Discussion The result from a batch fermentation with no pH control was carried out first. The pH was initially kept at 7.0, which was suggested to be the optimal pH for the growth of A. viscosus. After 200 min of operation, the pH quickly dropped due to a corresponding fast growth of cells. The growth of cells did not favor such an acidic environment, and therefore, the activity for growth started to become lower. At a time of around 400 min, the cells stopped growing due to the unfavorable acidic pH, and therefore, the pH no longer decreased. A final optical density of 5.0 was reached at the final pH of 4.5. It was observed from this fermentation that without pH control, the growth of cells would thus be limited. Since the secretion of enzyme was linearly correlated to the growth of cells in this system, the enzyme activity would be limited as well. In addition, the activity of the cell’s metabolism was strongly governed by the pH; as a result, the pH should be well controlled during such a batch fermentation. Therefore, the fermentation system with pH control was chosen as the testing system for the implemetation of neural network control. The transfer function of bx/(1 + |x|) was used with the RBPN. This function was proposed in our previous work and achieved good performance. Also, the RBPN was given a batch mode

Ind. Eng. Chem. Res., Vol. 36, No. 9, 1997 3759

of learning which could cost more computation time than a pattern mode of learning. This is why the longterm prediction instead of the short-term one can be performed by an RBPN. As a result, bx/(1 + |x|) was chosen because it saved a lot of computation time for such a mode of learning compared to the other functions, especially a function in exponential form. As the data for the batch mode of learning increased, the effect from this function became more significant. This is why this type of function instead of exponential sigmoid functions was applied in this work. The RBP type of network was used in this work, and pH control was chosen as the test system. The RBPN was chosen for its dynamic structure and ability for long-term identification. The factors that may affect the performance of the control system will be discussed as follows. Data from several batches of the penicillin acylase fermentation were provided to the RBPN for offline learning. The RBPN controller was able to learn the knowledge about this fermentation from the provided data before it was imbedded as the on-line controller. The learning coefficient η and the momentum term coefficient R through the off-line learning were determined to be 0.2 and 0.6, respectively. These parameters related to learning would be set before the on-line control. The input node of the RBPN was set to ∆pH(t). When the neural network acted as the process model, ∆pH(t) ) pH(t) - pH(t - 1). As the network was set to be the controller, ∆pH(t) ) pH(t)setpoint - pH(t). pH(t) denotes the current measured pH. Initially, when determining the input nodes of the network controller, a time delay term was included. However, it was found that the time delay term was not necessary for the RBP type of network. The conclusion is somehow different from what was expected. Since the fermentation is a dynamic system of slow reaction, a time delay often exists. However, the conclusion above can be realized from the dynamic nature of the RBPN structure. The RBPN has already had, in its structure, the recurrent nodes representing the time delay term from the previous time interval. This explains why a result appeared that the input layer did not need to include the time delay element required for a BPN. At the beginning, the network controller was initiated from a state of “no knowledge” in its memory. The predictor/controller then started on-line learning along with the fermentation. To effectively control the computation of this network controller, a moving window type of learning mode was used. Hence, the size of the training data had to be limited. The window size corresponded to the size of the training data string. With a data string of a smaller size, the learning as well as the prediction would not be effective. However, with an oversized data string, the computation would be reduntant and not efficient as well. At the beginning of the fermentation, on-line data were collected and learned, until the collected data size reached the window size; the moving window mode of the training data set was initiated by adding freshly acquired data and, at the same time, erasing the oldest data. The learning from then on proceeded by a fixed size of training data. A window size of 15 for the training data provided to the network controller was determined experimentally. The sampling time during a control process might also affect the control results. If the sampling time interval was rather large, the control action would not be generated frequently enough and would not be sent to

Figure 3. Control result from the network controller with intended error.

control the process in time. On the contrary, if the sampling time was rather small, then, the control action would be too frequent and might not be necessary. Meanwhile, the control might not be effective under such a circumstance as the fluctuation would occur. A sampling time of 5.0 min was found to be appropriate experimentally. In Figure 3, the initial pH started intendedly from 5.2 instead of 7.0. An error control action was also given to the control system. It was observed that a quite stable control result could be obtained from the network controller by such an intended error strategy. However, the controlled pH showed a certain amount of offset from the set point. The controlled pH was around 6.0 instead of the set point of 7.0. Hence, it can be concluded that such an operating condition was stable but not good enough. Nevertheless, there was one thing we can be sure of from the above experimental results of this network control system: when an uncertainty was introduced into the system, the network control can recognize the error in a very short time and, therefore, will not lead to the wrong control mechanism. Initial Pump Rate. The initial pump flow rates of 0 and 0.5 mL/min were compared. Figures 4 and 5 indicate that the initial rate affected the control path and, therefore, the control performance of the system. In Figure 4, an initial rate of 0.5 mL/min for adding basic solution to the pump was used. Initially the pH went up to around 7.0 and the control until this point seemed good. However, from then, the controlled pH shifted down to around 6.2 and had an offset from the set point afterwards. On the other hand, in Figure 5, with a zero initial pump flow rate, the pH can be quite stably maintained at 7.0, which demonstrated a successful control result. An initial pump flow rate of 0.2 mL/min was also tried after the comparison of the two cases above for initial pump flow rates and did not achieve any better control result. With an initial rate of 0.2 mL/min, the control was unstable and the pH increased gradually during the fermentation, which indicated a poor control performance. Base/Acid Concentration. The base/acid concentrations used for the pH control were also tested. The concentrations of 1.0/1.0 and 2.0/2.0 N, respectively, were set for comparison. The results are shown in Figures 5 and 6. For the base/acid concentration of 2.0/

3760 Ind. Eng. Chem. Res., Vol. 36, No. 9, 1997

Figure 4. Control result from RBPN controller with initial pump flow rate of 0.5 mL/min.

Figure 5. Control result from RBPN controller with zero initial pump flow rate and base/acid concentration of 1.2/1.2 and 1.0/1.0 N, respectively.

2.0 N, although the control seemed to stay around pH 7.0, fluctuation could be significant. Hence, it was not an effective control. From Figure 6, a more stable control was obtained and the offset was rather small. It can be realized that if any uncertainty occurred in the control action, when the predicted action was delivered to the system, with higher concentrations of base or acid, larger amplification of the error might be caused. As a result, more fluctuation can be observed for the case of higher base/acid concentration. Therefore, a concentration of 1.0 N was used for this control system. In conclusion, the initial rate for the control pump should be set to zero. The base/acid concentration should not be too high; 1.0 N can carry out good control of pH for this fermentation. In addition, the other operating parameters as well as network parameters should also be properly chosen. The RBP controller with a single input of ∆pH(t), on-line training data of moving window size 15, and sampling time of 5.0 min can together obtain the best control of this fermentation system. Figure 5 has elucidated the fact. The control was well performed repeatedly as shown in Figure 7. Therefore, we can be sure from the experiments that

Figure 6. Control result from RBPN controller with base/acid concentration of 2.0/2.0 N.

Figure 7. Another fermentation control result performed from zero initial pump flow rate, 1.0/1.0 N base/acid concentration, and the RBPN controller with input node of ∆pH(t).

stable control as well as precise control results can be achieved by the network controller with the aforementioned operating conditions. Apparently, the repeated performance can also be ensured. Conclusions The penicillin acylase fermentation with on-line pH control was used to test the performance of a recurrent backpropagation neural network process model with adaptive control. With the RBPN controller, the time delay term being considered for a dynamic process is no longer necessary. During the learning phase, the network was used as the model and the input node of the network is the difference of pH values measured at two consecutive times; while during the control phase, the input node was defined as the difference of the pH at current time from the set point of 7.0. At the beginning of a control, the initial rate for the control pump should be set to zero. The concentrations of base and acid solutions for control were both 1.0 N. With such a combination, the RBPN control of this fermentation system achieved a more stable control of pH at 7.0

Ind. Eng. Chem. Res., Vol. 36, No. 9, 1997 3761

and higher optical density of 6.7 by the end of the fermentation. Even with an intended error strategy, the RBPN controller can still respond well. The control performance of this fermentation was repeatedly ensured by the same summarized operating conditions. With the success of the RBPN adaptive control tested by the pH control of penicillin acylase fermentation, the RBPN control can also be tested on other control systems of similar bioprocesses. Acknowledgment This work was supported by Grant NSC 85-2214E006-001. Nomenclature h(0)j ) initial values of hidden nodes y(0)k ) initial values of output nodes ωji(t) ) weight connecting the jth hidden node of time t, h(t)j, and the ith input node at current time t, x(t)i ωjjr(t - 1) ) weight connecting the jth hidden node of time t, h(t)j, and the jth recurrent node from hidden layer at time t - 1, h(t - 1)j ωkj(t) ) weight connecting the kth output node of time t, y(t)k, and the jth hidden node of time t, h(t)j ωkkr(t - 1) ) weight connecting the kth output node of time t, y(t)k, and the krth reccurent node from output layer at time t - 1, y(t - 1)kr net(t)j ) weighted sum of current input value and previous value of hidden nodes net(t)k ) weighted sum of current value of hidden nodes and previous output value δ(N)k ) error learning signals from the output layer at time N δ(N)j ) error learning signals from the hidden layer at time N Y(N)k ) desired kth output node at time N y(N)k ) kth computed output node at time N

∆ω(t)ji ) update amount of ω(t)ji ∆ω(t)jjr ) update amount of ω(t)jjr ∆ω(t)kj ) update amount of ω(t)kj ∆ω(t)kkr ) update amount of ω(t)kkr η ) learning coefficient R ) momentum term coefficient β ) learning coefficient for b

Literature Cited Boskovic, J. D.; Narendra, K. S. Automatica 1995, 31 (6), 817. Caudill, M.; Butler, C. Understanding Neural Networks: Computer Explorations; MIT: Cambridge, MA, 1992; Vol. II. Hiroshi, O.; Yumiko, K.; Toshio, K. Appl. Environ. Microb. 1989, 55 (6), 1351. Khalid, M.; Omatu, S.; Yusof, R. IEEE Trans. Neural Networks 1995, 6 (3), 572. Ku, C. C.; Lee, K. Y. IEEE Trans. Neural Networks 1995, 6 (1), 144. Narendra, K. S.; Parthasarathy, K. IEEE Trans. Neural Networks 1990, 1 (1), 4. Quin, Z.; et al. Biotechnol. Bioeng. 1994, 43, 483. Robinson, G. N.; Batchelor, F. R.; Butterworth, F.; Wood, J. C.; Cole, M.; Eustage, G. C.; Hart, M. V.; Richards, M.; Chan, E. B. Nature 1960, 187, 236. Shimizu, K. Comput. Chem. Eng. 1996, 20 (6/7), 915. Syu, M.; Tsao, G. Proc. 1994 IEEE Int. Conf. Neural Networks 1994, 5, 3265. Uozumi, N.; Shi, Z. P.; Shiba, S.; Cayuela, C.; Iijima, S.; Kobayashi, T. J. Ferm. Bioeng. 1993, 76 (6), 505. Yang, Y. Y.; Linkens, D. A. IEE Proc. Control Theory Appl. 1994, 141 (5), 341.

Received for review October 1, 1996 Revised manuscript received May 19, 1997 Accepted May 20, 1997X IE9606092

X Abstract published in Advance ACS Abstracts, August 1, 1997.