88
Biofechnol. frog. 1995, 11, 88-92
Biomass Estimation in Plant Cell Cultures: A Neural Network Approach Joan AlMol, Carles Campmajb, Carles Casas, and Mane1 Poch' Unitat d'Ehginyeria Quimica, Universitat Autbnoma de Barcelona, 08193 Bellaterra (Barcelona), Spain
The special characteristics of plant cell cultures make it difficult to use conventional analytical techniques for on-line biomass monitoring. Meanwhile, promising results have been obtained using mathematical models and recursive estimation algorithms. However, in this case, additional experimental effort is necessary to obtain a reasonable description of the process. Recently, techniques using more empirical approaches have been proposed to describe complex processes, minimizing the experimental work needed for their application. In this paper, we report on the use of artificial neural networks to monitor biomass evolution in plant cell cultures. The results obtained with a threelayered network are presented. Method requirements and capabilities are compared with the method based on the extended Kalman filter used in previous work.
Introduction The use of large-scale cultures of plant cells has been suggested as an alternate source of high-value natural products, such as foods, oils, and medicinal compounds. One of the key problemg in obtaining successful fermentations is monitoring of the time course of the main system variables. However, the special behavior of in vitro plant cell cultures, such as slow growth rate, formation of aggregates, or adhesion to bioreactor walls (Taticek et al., 1991), makes the use of conventional techniques of on-line monitoring difficult (Kwok et al., 1992; Zhong et al., 1993). Considerable effort has been invested in the development of specific probes to measure these cultures, but although some results have been obtained, due to the special characteristics of plant cell cultures it is necessary to rely upon estimation procedures based on related variables using software sensors (Albiol et al., 1993; Montague et al., 1992). In the past, different authors (San and Stephanopoulos, 1984; Stephanopoulos and San, 1984; Valero et al., 1990;Albiol et al., 1993)have used methodology based on a mathematical model and the application of the extended Kalman filter (EKF) to solve this problem. In this case it is necessary to obtain a descriptive model of the process, which relates the predicted and measured variables and, therefore, requires a deeper understanding of the process. A more recent approach based on the use of neural network methodology has been proposed for the prediction of fermentation variables (Thibault et al., 1990; Link0 and Zhu, 1992; Lassey et al., 1994), as has been applied successfully for different subjects ranging from deconvolution of spectra (McAvoy, 1992) and flow injection analysis modeling (Campmaj6 et al., 1992; Hartnett et al., 1993) to cardiac disease detection from echocardiographic images (Krzysztof et al., 1990). Artificial neural networks (A") are mathematical algorithms (Lippmann, 1987; Bressloff and Weir, 1991; Zupan et al., 1991) derived from artificial intelligence techniques that try to model, in a simplified way, our present understanding of the human brain. They are a group of interconnected neurons forming a net where every neuron receives input signals, through its dendrites, from every neuron connected to it. The transmission efficiency depends on the synapsis strength that
joins two dendrites. The sum of all the input signals to the neuron body excites it and, up to a threshold level, makes the neuron produce an output signal through its axon. If the input signal is too high, only a maximum output signal level is produced. The distinctive advantage of A" is that it does not require any prior knowledge about the structure of the relationships that exist between input and output signals. Although successful results have been obtained in the on-line prediction of fermentation variables using simulated data, many fewer results are available with real data as bacterial (Syu and Tsao, 1993) or fungal growth (Willis et al., 19921, especially in complex systems like plant cell cultures. But, as has recently been pointed out (Collins, 19931, process biotechnologists need to see more published examples of neural network applications before they feel confident with it. In this paper, we present results obtained in the evaluation of the use of neural networks for biomass estimation in plant cell cultures in order to test its efficiency and to compare it with a previously used, recursive estimation method.
Materials and Methods Cell Line and Culture Methods. Duucus curotu cells were obtained from a root explant in solid medium made with Murashigue Skoog basal salt mixture supplemented with 0.1 g L-' myo-inositol, 0.5 mg L-l nicotinic acid, 0.5 mg L-' pyridoxine hydrochloride, 0.5 mg L-l thiamine hydrochloride, 2 mg L l glycine, 1 pM (2,4dic1orophenoxy)aceticacid, 1pM kinetine, 30 g L-l(87.7 mM) sucrose, and 10 g L-l agar. Suspension cultures were obtained placing some calli in t h i s medium, without agar, in a reciprocal shaker at 100 oscillations min-l and subcultured every 2 weeks for more than a year. All cultures were incubated in the dark at 30 "C. Cultivation on bioreactors was done using either a l-L fermentor (Biolab Braun-Melsungen) or a 4-L fermentor with a low-shear stirring system (Celligen, New Brunswick Scientific). Operational conditions were 30 "C,100 rpm, and 60% dissolved oxygen without using the pH controller, unless otherwise indicated. The culture medium used was the same as before with different initial sugar concentrations for each experiment. Known aliquots of cells were inoculated into bioreactors under
8756-7938/95/3011-0088$09,00/00 1995 American Chemical Society and American Institute of Chemical Engineers
Biotechnol. Prog., 1995, Vol. 11, No. 1
89
sucRosE(1.I)
a
GLUWSE(1-1) FRUCTOSE(I.1)
GLUCOSE(MI
Figure 1. Three-layered neural network structure, as used in this work.
aseptic conditions. Usually the inoculum medium carried along some glucose and fructose not consumed in the previous culture, modifying the total carbohydrate concentration in the medium. Carbohydrate Analysis. Sucrose, glucose, and fructose levels were analyzed using an HPLC system (Hewletb Packard 1090) coupled to a refractive index detector (Hewlett-Packard 1037A) with a Bio-Rad HPX-87N column. The running conditions were as follows: eluent, 15 mM Na2S04; column temperature, 80 "C; flow rate, 0.5 mL min-'. Biomass Determination. For the cell mass determination, 20-mL culture samples from the bioreactors were filtered using a preweighed glass fiber filter (Nucleopore D49). Dry cell weight was measured after drying the filter at 80 "C until a constant weight was obtained. The cell mass concentration was calculated over the filtered medium volume instead of the total volume, as explained previously (Albiol et al., 1993). Neural Network Architecture. In standard architecture, neurons are grouped into layers, as is the feedforward structure (Figure 1)in which all M neurons in the same layer receive the same K inputs from the preceding layer, giving an M x K matrix of weights associated to their connections. Different layers can be considered within the net: an input layer that has one input neuron for each signal fed to the net; an output layer with one output neuron for each output signal the net must predict; and some hidden layers, usually one (Hornik, 1982; Fu and Poch, 1993) with a variable number of neurons, depending upon the complexity of the process to be modeled. Every neuron in a hidden or output layer gives an output signal obtained from a weighted sum of the input signals, to which a sigmoidal function is applied (in order to obtain a signal ranging between 0 and 1): K
input$ = z(weight$Joutput$')) J=l
for each I neuron in an L layer, and o u t p u c = F(input$) =
1 1 + exp(-input$)
TrainingAlgorithm. The signals fed to the network are propagated forward, through the neurons, to give the output of the net. In order to train the ANN to predict correct outputs (those obtained from experimentation), a back-error propagation algorithm-based on the multilayered feed-forward net-was used. Using the quadratic error measured from the output of the net (between simulated and experimental data) as an objective func-
5 0
100
200
300
400
500
600
Number o f iterations Figure 2. Quadratic error evolution versus the number of iterations for different neuron numbers in the hidden layer.
tion, weights associated with the different connections are changed, using a gradient descent method as the oDtimization algorithm. in a backward direction through tAe layers. Th;! change in the weights is calculated i s A%'
= &(outputf-')
where AWfJ =
(wJw - Tid)
where q is called the learning rate. The algorithm can be summarized as follows: First, a n input data set is fed to the network, and the output set obtained is compared with the desired set to obtain a n error. Second, all the weights in the different layers are corrected, in a backward direction, to the input layer. This iterative process is repeated for the whole set of inputs several times, until a convergence criterion is achieved. After this learning phase, the net is tested with new data sets in order to validate its performance. Software. The algorithm and data processing routines were programmed in Quick Basic. All programs were run on a n Intel 486DW33 MHz PC computer with 4MB of RAM.
Results ANN Configuration. The ANN used was a threelayered (input, hidden, and output layers) feed-forward network with bias. There were eight input neurons for time (t),biomass cYt-l), and sugars (sucrose (St-', St-2), glucose (Gt-', Gt-2), and fructose (Ft-', Ft-2))at two preceding times and four output neurons (biomass (Xt), sucrose (St),glucose (Gt),and fructose (Ft)).The hidden layer has three neurons, determined as the optimal configuration that gives lower error on training with minimal computing time. Figure 2 shows quadratic error (biomass data only) evolution versus the number of iterations for different numbers of neurons in the hidden layer of the network. The biomass used as the input signal came from the output of the network at the preceding time, when there was no experimental data at that time. Furthermore, sugar data were provided at two preceding times in order to give information to the network about their changes during the interval. Training Procedure. In order to implement the training procedure and to test the ability of the neural
Biotechnol. Prog., 1995, Vol. 11, No. 1
90 14,
12
I 10
0 10
-
2
8
/
c
v u
r
.-0
6
a,
3 x 6
4
2
0
0 100
100
v h _J
1
-E" E
-
v Sucrose
Sucrose
A
Fructose
80
60
Ln
4-
e
-0 X 1 0
40
e
0
20 0
...... ...... .-
..a
0 0
5
10
15
J
20
**
3
25
Time (days)
Time (days)
Figure 3. Kinetics of Daucus carotu cell growth and sugar concentrations for experiment BL1603. Symbols represent experimental data: 0, biomass; V, sucrose; 0, glucose; A, fructose. Lines represent the network predictions: dotted line, output for learning using data from experiment BL1603; continuous line, learning using data from experiments BL1603 and CG1204.
networks to predict the behavior of the cultures, Merent fermentation experiments performed in two types of fermentors with changes in culture conditions were used. Initially, the network was trained with data from only one fermentation experiment, with a learning rate of 4.0 with 101 iterations. This experiment (BL1603) was performed in the Biolab reador with an initial biomass concentration of 0.75 g/L (dry weight basis) and sugar concentrations of 69.4 mM sucrose, 20.0 mM glucose, and 19.5 mM fructose. In this experiment, cell biomass increased slowly during the fist 15 days, without a clear lag phase, and afterward it entered the normal exponential growth phase. As is illustrated in Figure 3, where the results of simulated and experimental data are presented, the network provides a poor description of system behavior. This is specially true for biomass, where the network did not adapt to the changing behavior of the system, predicting a faster increase in the early moments and a longer delay in the start of the exponential phase (a mean absolute error over the whole biomass data set of 1.59 g/L). Once this preliminary test was made, we examined the capability of the trained network to describe a new experiment (BL907)conducted in the same bioreactor but with rather different operational conditions of inocula (0.58 g/L) and initial sugar concentrations of 61.4 mM
Figure 4. Kinetics of Daueus carota cell growth and sugar concentrations for experiment BL907. Symbols represent experimental data: 0, biomass; V, sucrose; U, glucose; A, fructose. Lines represent the network predictions: dotted line, output for learning using data from experiment BL1603;continuous line, learning using data from experiments BL1603 and CG1204.
sucrose, 26.5 mM glucose, and 26.8 mM fructose. Then there was a long lag phase, as no increase in biomass was detected until the 29th day. As can be seen in Figure 4, the deviation between experimental and calculated data is again significant (a mean absolute error of 1.06 particularly for biomass during the lag phase period. From these experiments, it seemed very clear that, for the learning step, the use of data from only one experiment was not sufficient to obtain an accurate description of the biomass growth pattern in these kinds of processes. In order to improve the results obtained from the network, we used two experiments in the training process that represented different culture behaviors. The first experiment (BL1603) was the same as that used in the previous training process, and the second one (CG1204) was performed in a Celligen bioreactor. In this experiment, a higher inoculum (0.96 g/L) and initial sucrose concentration (83.8mM) were used, which caused the lag phase to be reduced to a 6-day period. The different experimental conditions also decreased the exponential growth rate to a value of 0.15 day-', maintaining the yield relative to carbohydrates consumed. As illustrated in Figures 3 and 5, the additional information given in the learning process made the network able to predict biomass and sugar data with reasonable accuracy, taking into account the different
a),
Biotechnol. Prog., 1995, Vol. 11, No. 1
91
10, 10
0
5
v
4 - 6 L
0, ._ 01
3
/
t . 4
n
2
0 100
100 > O
i
'r0 Sucrose A Fructose 0 Glucose
6
40 20
LA
o\ 0
-? E
V Sucrose
80
60
u)
W +
?
72 x L
40
0
e 0
20
0
5
A
10
15
20
25
30
35
Time (days)
0
I
I
I
10
20
30
40
50
Time (days)
Figure 5. Kinetics of Daucus curotu cell growth and sugar concentrations for experiment CG1204. Symbols represent experimental data: 0, biomass; V, sucrose; 0,glucose; A, fructose. The continuous line is the network prediction for learning using data from experiments BL1603 and CG1204.
Figure 6. Kinetics of Daucus carotu cell growth and sugar concentrations for experiment CG907. Symbols represent ex-
growth rates shown by Daucus carota cells in each experiment. The mean absolute error for the BL1603 experiment decreased by 0.37 g/L, and a value of 0.35 g/L was obtained for the CG1204 simulation. Again, a learning rate of 4.0 with 101 iterations of the learning data set was used. Validation Step. With these results, the training step was finished and the A" was validated using data from new experiments (CG907 and BL907). In the first case (Figure 61, cell growth was performed in the Celligen reactor with an inoculum of 0.58 g/L. With the established conditions, cells grew exponentially after a 29-day lag phase and biomass reached the maximum value of 4.8 g/L after 49 days. As illustrated in Figure 6, changes in biomass and sugar behavior were correctly identified by the network output, even though biomass data in the lag phase are overestimated slightly (with a mean absolute error of 0.94 gL). These results were considered satisfactory since they corresponded to the validation step when only sugar values were provided to the network. In order to compare the results of the training procedure, evaluation was also carried out on the experiment (BL907) assayed in the former learning step (with one fermentation). As can be seen in Figure 4, this ANN gave a better description of the experimental data for sugar consumption and biomass (the mean absolute error decreased by 0.78 g/L) in both the lag and exponential phases of growth.
Performance Evaluation. In a previous study, the authors reported the use of a digital filter to monitor the biomass evolution in plant cell cultures, applying a deterministic model based on sugar consumption data (Albiol et al., 1993). As is shown in Figure 7, which represents the flow diagrams of two procedures, in the EKF approach biomass identification was based on the use of a deterministic mathematical model that describes substrates, biomass, and products. To build the model, it was necessary to form some hypotheses about their relationships (variation of specific growth rates, preferential consumption, maintenance rates, etc.) and to carry out some experiments to evaluate the kinetic parameters that appear in the differential equations. Once the model was built, supplementary experiments were performed in order to obtain some specific parameters related to the identification algorithms, such as system and measurement covariance matrices. Thus, with this approach it was necessary to conduct several experiments until a successful procedure for biomass estimation was obtained. Furthermore, the method furnished information about the evolution of culture parameters and related process malfunctions. The results presented in this paper show how with the use of just two experiments, the neural networks approach allows us to accurately describe biomass and sugar evolution, overcoming the main disadvantage of conventional modeling. It reduces the amount of infor-
perimental data: 0, biomass; V, sucrose;0,glucose; A, fructose. The continuous line represents the network prediction for learning using data from experiments BL1603 and CG1204.
Biofechnol. Rug,, 1995, Vol. 11, No. 1
92 EXTENDED KALMAN FILTER APPROACH (AI biol et al. 1993)
NEURAL NETWORKAPPROACH (This work)
NETWORK CONFIGURATION
NEW EXPERME
Ll+ VALIDATION STEP
MEASUREMENTNOISE COVARlANCE MATFHX
1 I
PARAMETER EVOLUTION
-
- -
.
ESTlMATtON 1 _
.
Figure 7. Schematic diagram of the two methodologies used for biomass identification based on the extended Kalman filter and neural network approaches.
mation necessary for obtaining a reasonable description of biomass evolution and can be applied to the prediction of different experimental conditions, if they are considered in the training step. However, neural networks do not permit us to obtain information about process malfunctions, as can be done with the previously used modeling tools. They are of interest when they are focused on complex bioprocesses, such as plant cell cultures, where there is great difficulty in accurately obtaining biomass data.
Acknowledgment This work received financial support from the CIRITGeneralitat de Catalunya and the Ma. F. Roviralta Foundation. J.A. was supported by a n FPI grant from the Ministerio de Educacih y Ciencia (Spain). Literature Cited Albiol, J.; Robust6, J.; Casas, C.; Poch, M. Biomass Estimation in Plant Cell Cultures Using Extended Kalman Filter. Bwtechnol. Prog. 1993,9,174-178. Bressloff, P. C.; Weir, D. J. Neural Networks. GEC J.Res. 1991, 8,151-169. Campmaj6, C.; Poch, M.; Robust.6, J.; Valero, F.; Lafuente, J. Evaluation of Artificial Neural Networks for Modelling Enzymatic Glucose Determination by Flow Injection Analysis. Analusis 1992,20,127-133. Collins, M. Empiricism Strikes Back: Neural Networks in Biotechnology. BiolTechnoEogy 1993,11,163-166. Fu, C. S.; Poch, M. Application of a Multi-layered Neural Network to System Identification. Znt. J. Syst. Sci. 1993,8, 1601-1609. Glassey, J.;Montague, G. A.; Ward, A. C.; Kara, B. V. Enhanced Supervision of Recombinant E. coli Fermentations via Artificial Neural Networks. Process Biochem. 1994,29,387-398. Hartnett, H.; Diamond, D.; Barker, P. G. Neural Network Based Recognition of Flow Injection Patterns. Analyst 1993,118, 347-354. Hornik, IC; Stinchcombe, M.; White, H. Multilayer Feedforward Networks are Universal Approximators. Neural Networks 1982,2,359-366. Krzysztof, J. C.; Chen, K.; Langenderfer, R. A. Use of Neural Networks in Detecting Cardiac Diseases from Echocardiographic Images. ZEEE Eng. Med. Biol. 1990,September, 5860.
Kwok, K. H.; Tsoulpha, P.; Doran, P. M. Limitations Associated with Conductivity Measurement for Monitoring Growth in Plant Tissue Culture. Plant Cell Tissue Organ Cult. 1992, 29,93-99. Linko, P.; Zhu, Y.-H. Neural Network Modelling for Real-time Variable Estimation and Prediction in the Control of Glucoamylase Fermentation. Process Biochem. 1992,27,275283. Lippmann, R. P. An Introduction to Computing with Neural Nets. ZEEE ASSP 1987,April, 4-22. McAvoy, T. J.; Te Su, H.; Wang, N. S.; He, M.; Horvath, J.; Semerjian, H. A Comparison of Neural Networks and Partial Least Squares for Deconvoluting Fluorescence Spectra. Bwtechnol. Bioeng. 1992,40,53-62. Montague, G. A.; Morris, A. J.; Tham, M. T. Enhancing Bioprocess Operability with Generic Software Sensors. J. Biotechnol. 1992,25,183-201. San, K.; Stephanopoulos, G. Studies on On-Line Bioreactor Identification. 11.Numerical and Experimental Results. Biotechnol. Bioeng. 1984,26,1189-1197. Stephanopoulos, G.; San, K. Studies on On-Line Bioreactor Identification. I. Theory. Biotechnol. Bioeng. 1984,26,11761188. Syu,M.; Tsao, G. T. Neural Network Modelling of Batch Cell Growth Pattern. Biotechnol. Bioeng. 1993,42,376-380. Taticek, R. A.;Moo-Young, M.; Legge, R. L. The Scale-up of Plant Cell Culture: Engineering Considerations. Plant Cell Tissue Organ Cult. 1991,24,139-158. Thibault, V.; Van Breusegem, A; Cheruy, A. On-line Prediction of Fermentation Variables Using Neural Networks. Biotechnol. Bioeng. 1990,36,1041-1048. Valero, F.; Lafuente, J.; Poch, M.; Soh, C. Biomass estimation using on-line glucose monitoring by flow injection analysis. Appl. Biochem. Biotechnol. 1990,24125,591-602. Willis, M. J.; Montague, G. A.; Di Massimo, C.; Tham, M. T.; Morris, A. J. Artificial Neural Networks in Process Estimation and Control. Automatica 1992,28,1181-1187. Zhong, J. J.; Fujiyama, K.; Seki, T.; Yoshida, T. On-Line Monitoring of Cell Concentration of Perilla frutescens in a Bioreador. Bwtechnol. Bioeng. 1993,42,542-546. Zupan, J.; Gasteiger, J. Neural Networke: A New Method for Resolving Chemical Problems or Just a Passing Phase? J . Anal. Chim. Acta 1991,248,1-30. Accepted July 22, 1994.@ @
Abstract published in Advance ACS Abstracts, October 15,
1994.
