Online Monitoring and Quality Prediction of Multiphase Batch

Article pubs.acs.org/IECR

Online Monitoring and Quality Prediction of Multiphase Batch Processes with Uneven Length Problem Zhiqiang Ge* and Zhihuan Song State Key Laboratory of Industrial Control Technology, Institute of Industrial Process Control, Department of Control Science and Engineering, Zhejiang University, Hangzhou 310027, Zhejiang, China ABSTRACT: In a practical multiphase batch process, the durations of batches are probably different, and at the same time, the length of each phase may also vary from batch to batch. This paper proposes an effective method to handle the uneven length problem in multiphase batch processes, which is also useful when the number of training batches is limited. To address the data nonlinearity, two nonlinear modeling techniques are introduced. For online monitoring and quality prediction, a key step is how to locate the current data sample to the specific phase, on which basis an appropriate model can be employed. In this paper, a phase discrimination model for data localization in different phases that is based on the support vector data description model is developed. For performance evaluation of the proposed method, detailed illustrations of a typical multiphase batch process are provided.

1. INTRODUCTION Batch processes play an important role in modern industries, such as the chemical, pharmaceutical, plastic, and food sectors. Because of the complexity of batch processes, it is always difficult to develop first-principles models. With more and more data being generated in the process, various data-based modeling methods have gained more attention in recent years. Particularly, since the pioneering work of Nomikos and MacGregor,1,2 a large number of research works have been published on multivariate-statistics-based methods for monitoring and quality prediction of batch processes.3−12 However, most existing methods have assumed that the reference/testing batches have the same duration. In practice, because of process disturbance, different initial settings, changes of operation conditions, etc., the durations or lengths of the batches are often different from each other. For those batch processes that have multiple phases, the duration of each phase may also vary from batch to batch, which makes the uneven length problem more complicated in batch processes. In this case, both the whole duration and phase durations of the process vary from batch to batch. To handle the uneven length problem in batch processes, various methods have been developed in the past years. For example, the simplest method is to cut the batches to the minimum length as long as there are enough batches for modeling.13 Another simple method is to treat the absent part of short-duration batches as missing data, which can be estimated by various approaches.1 Unfortunately, these two methods are suitable only when the uneven length problem is not very serious and the main data characteristics have been captured by their common trajectories. A more reasonable solution is to utilize a proper indicator variable to track the batch progress, which acts monotonically through the time dimension.14 With the same starting and ending values for each batch, process modeling, monitoring, or quality prediction can be carried out on the basis of the progress of this indicator variable. However, prior knowledge is needed to select such an indicator variable, and data interpolation is also necessary for calculation of corresponding measurements at regular sampling © 2013 American Chemical Society

intervals of the indicator variable. If the interpolations are not performed well, the real process data characteristic may be distorted. Moreover, such an indicator variable may not exist in every batch process. To date, several warping methods have also been introduced for handling the uneven length problem in batch processes, such as dynamic time warping (DTW) and correlation optimization warping (COW).15,16 Both of these methods synchronize the uneven batches by appropriately correcting the samples in different trajectories. However, these two methods are complicated and may also distort the relationships between process variables. Furthermore, with an inappropriate warping result, the modeling performance may become even worse. Other methods for handling the uneven length problem include the local batch time approach, similarity-based methods, etc.17−20 In multiphase batches, different phases have their own variable correlations and data characteristics. However, within each phase, both the data characteristics and correlations between process variables are similar. Therefore, without losing systematic variations around normal trajectories, we can unfold the three-way batch data set variable-wise in each phase. As a result, the uneven batch duration problem can be spontaneously solved, which means no trajectory synchronization or missed data estimation is needed. However, different from the typical batchwise unfolding method, trajectory normalization among different batches has been ignored. Therefore, a serious nonlinearity of the batch process data that can be efficiently removed by trajectory normalization may exist in each phase. For online monitoring or quality prediction, another problem is to locate the current data sample to its correct phase in the batch process and then to employ the corresponding model for utilization. Another advantage of this method is that it is Received: Revised: Accepted: Published: 800

September 27, 2013 November 26, 2013 December 21, 2013 December 22, 2013 dx.doi.org/10.1021/ie403210t | Ind. Eng. Chem. Res. 2014, 53, 800−811

Industrial & Engineering Chemistry Research

Article

where a is the center of the hypersphere, C gives the trade-off between the volume of the hypersphere and the number of errors, and ξi is the slack variable of data sample i. To simplify the optimization problem, the dual form can be solved:22

particularly useful when the number of modeling batches is limited in the process.21 Similar to the uneven length problem, the limited batch problem is also very common in practice. While it is easy for certain batch processes with short batch duration and low cost to run trials, some other batch processes may not be able to collect sufficient batches for modeling because of long batch duration or too great an expense to run trials. In those cases, the statistical model needs to be developed on the basis of only a limited number of batch data. In the present paper, the support vector data description (SVDD) method is introduced for nonlinear modeling in each phase of the batch process. While SVDD has been widely used for one-class classification in pattern recognition and machine learning areas,22 the effectiveness of this method for process monitoring has also been explored in recent years.23−26 On the basis of the SVDD method, the operation region of each phase in the batch process is modeled and enveloped by a hypersphere in the high-dimensional feature space. The distance between the data sample and the center of the hypersphere is used as the statistic for process monitoring. For phase localization of the data sample along the batch duration, a nonlinear discriminant model that is also based on the SVDD model is formulated. To predict the final quality of the batch process, a nonlinear regression approach called relevance vector machine (RVM) is employed for modeling of the phase relationship between the process and quality variables. The main benefits of the RVM model compared with other nonlinear modeling approaches have been well demonstrated in refs 27 and 28. Generally speaking, RVM and SVM have better generalization performance and can work better under limited training data samples than other nonlinear modeling methods. However, SVM does not allow for the use of an arbitrary kernel function and requires a determination of the trade-off and insensitivity parameters, which may generally entail a cross-validation procedure, thus enlarging the computation burden. Besides, the output of SVM is only a point estimation of the predicted variable, and the uncertainty of the prediction cannot be captured. The remainder of this paper is structured as follows. In section 2, preliminaries about the two nonlinear modeling methods (SVDD and RVM) are introduced. A detailed description of the methodology for handling the uneven length and limited batch problems, including process monitoring and quality prediction schemes for multiphase batch processes, is given in section 3. Section 4 shows a detailed illustration example of the penicillin production process. Finally, conclusions are made.

n αi

i=1

i=1 j=1

s.t. 0 ≤ αi ≤ C ,

∑ αi = 1 (2)

i=1

where K(xi,xj) = ⟨Φ(xi),Φ(xj)⟩ is a kernel function that represents the nonlinear inner product in the feature space and αi is a Lagrange multiplier. 2.2. Relevance Vector Machine. Compared with the support vector machine method, RVM is a new nonlinear modeling algorithm that was originally proposed by Tipping.29 With the introduction of a prior distribution for each parameter weight, the RVM algorithm can be developed under a similar framework as SVM through Bayesian inference. Given the training data set as {xi, yi}i=1,2,···n, the nonlinear relationship can be represented as y = f (x , w) + e

(3)

where e is the random error, which is assumed to be independent and Gaussian-distributed with zero mean and variance σ2 [i.e., e ∼ N(0,σ2)], and w is the weighted parameter, which is also assumed to be Gaussian and is given by n

p(w|α) =

∏ N(wi|0, αi−1) i=0

=

1 (2π )(n + 1)/2

n

⎛ αw2 ⎞ i i ⎟ 2 ⎠ ⎝

∏ αi1/2 exp⎜− i=0

(4)

where α is the hyperparameter. The optimal values of α can be obtained by carrying out the following optimization: max L(α , σ 2) = max log{p(y|X, α , σ 2)}

∫

= max log{ p(y|X, w, σ 2)p(w|α) dw} (5)

As a result, many elements of w become zero, and the optimal w value has only a few nonzero elements. The data samples corresponding to those nonzero elements of w are termed the “relevance vector” in the RVM method. Similar to the SVM algorithm, the nonlinear function f(x) can be expressed as a linearly weighted sum of a set of basis functions:29 RV

f (x , w) =

T ∑ wK i (x , x i) + w0 = w ψ (x) i=1

(6)

where w = [w0,w1,w2,···,wRV]T is the weighted parameter vector of the basis functions w, ψ(x) = [1,K(x,x1),K(x,x2),···, K(x,xRV)]T, and RV is the number of relevance vectors. Because of the probabilistic assumption of the model parameters, RVM can provide detailed uncertainty information for the output variable, which is given as a Gaussian distribution p(y|x) = N( f(x,w),σ2).

n

min R2 + C ∑ ξi i=1

s.t. || Φ(x i) − a ||2 ≤ R2 + ξi , ξi ≥ 0, i = 1, 2, ···, n

n

n

2. PRELIMINARIES 2.1. Support Vector Data Description. On the basis of a nonlinear transformation function Φ:x→F, SVDD first maps the data from the original space to the feature space and then constructs a hypersphere in the feature space. The main idea of the SVDD method is to minimize the volume of the hypersphere. Precisely, the following optimization problem is solved:22

R ,a,ξ

n

min ∑ αiK (x i , x j)− ∑ ∑ αiαjK (x i , x j)

(1) 801

dx.doi.org/10.1021/ie403210t | Ind. Eng. Chem. Res. 2014, 53, 800−811


Article

Figure 1. Illustration of the uneven length problem in multiphase batch processes.

Figure 2. Uneven-length data unfolding in multiphase batch processes.

3. METHODOLOGY In this section, the detailed methodology is presented, including a brief introduction of the uneven length problem in multiphase batch processes, the data unfolding method, phase-based nonlinear modeling approaches, and online monitoring/quality prediction algorithms. 3.1. Uneven Length Problem in Multiphase Batch Processes. In multiphase batch process, the durations of batches may be different in various phases. In this situation, phase division becomes very difficult. In the present paper, however, we assumed that the phase division has already been carried out for each batch. As has been mentioned, with our method only several batches are needed for modeling (e.g., five batches). Therefore, through trajectory analyses of process

variables, indictor variables, or even prior knowledge of the process, each of the batch data can be precisely divided into several phases. An illustration of the uneven length problem in multiphase batch processes is given in Figure 1, in which it can be seen that both the phase duration and the length of the whole batch differ from batch to batch. 3.2. Data Unfolding. We assume that for each batch there is a three-way data set Xi(1 × J × Ki) with i = 1,2,···,I, where I is the number of batches, J is the number of process variables, and Ki is the duration of the specific batch i. Through variable direction, each batch can be unfolded into a two-dimensional data matrix Xi(Ki × J), and if it is supposed that the whole batch process is divided into S phases, Xi(Ki × J) can be further partitioned as follows: 802



Article

1 2T 2 sT s ST S T X i(K i × J ) = [X1T i (K i × J )X i (K i × J )···X i (K i × J )···X i (K i × J )]

ignored the trajectory normalization step. Therefore, a significant nonlinearity may exist in each phase of the batch process. In this subsection, nonlinear models for process monitoring and quality prediction are constructed. 3.3.1. Process Monitoring Model. On the basis of the data structure given in eq 8, we intend to build a SVDD model for each phase of the batch process. The detailed formulation of phase-based SVDD models is given as follows. First, the modeling data set of each phase is extracted from the whole data set and is given by

(7)

where s = 1,2,···,S. Here we have assumed that the phase switch points are known beforehand. Generally, there are three types of phase division methods: the expert-knowledge-based method, the process-analysis-based method, and the data-based method. Compared with the other two types of phase division methods, the data-based method is more flexible and thus can be used in various batch processes. More information about the phase division problem can be found in Yao and Gao.30 Combining all I batches together, the two-dimensional modeling data set can be represented as follows:

⎡ X s (K s × J ) ⎤ ⎢ 1 1 ⎥ I ⎢ X 2s (K 2s × J )⎥ s s X (∑ K i × J ) = ⎢ ⎥ ⋮ ⎢ ⎥ i=1 ⎢ s s ⎥ ⎣ XI (KI × J ) ⎦

⎡ X (K × J ) ⎤ ⎥ ⎢ 1 1 ⎢ X 2(K 2 × J )⎥ X(∑ K i × J ) = ⎢ ⎥ ⋮ ⎥ ⎢ i=1 ⎥ ⎢ ⎣ XI (KI × J ) ⎦ I

where s = 1,2,···,S, i = 1,2,···,I, and is the number of data samples in the ith batch that belong to the sth phase. After removal of the mean value and normalization of the data variance, the SVDD model can be developed upon each of the phase data sets. Following the optimization problem given in section 2.1, the center as and radius Rs of each hypersphere for the phase SVDD model are calculated as follows:

⎧[X1T(K 1 × J )X2T(K 2 × J )···X sT(K s × J )···X S T(K S × J )]T ⎫ 1 1 1 1 1 1 ⎪ 1 1 ⎪ 1 2T 2 sT s ST S T⎪ ⎪ ⎪[X 1T ⎪ 2 (K 2 × J )X 2 (K 2 × J )···X 2 (K 2 × J )···X 2 (K 2 × J )] ⎬ =⎨ ⎪ ⎪ ⋮ ⎪ 1T 1 ⎪ ⎪[X (K × J )X2T(K 2 × J )···XsT(K s × J )···X S T(K S × J )]T ⎪ ⎩ I I ⎭ I I I I I I

(8)

While the process monitoring incorporates only the process data set, the quality data set is needed for prediction modeling. However, we only have I quality data, which correspond to the I batches. Therefore, it is infeasible to carry out the regression modeling with unbalanced input (∑iI= 1Ki) and output (I) data. To solve this problem, the quality variable yi(1 × Jy) with i = 1,2,···,I, where Jy is the number of quality variables, is copied Ki times for each batch. To avoid the numerical problem and introduce a reasonable diversity among different data samples, the noise injection method can be used. As a result, the extended quality data set can be represented as follows: ⎡ Y (K × J ) ⎤ y ⎥ ⎢ 1 1 ⎢ Y (K × J )⎥ I 2 2 y ⎥ Y(∑ K i × Jy ) = ⎢ + N (0, σ 2) ⎢ ⎥ ⋮ i=1 ⎢ ⎥ ⎢ YI (KI × J ) ⎥ y ⎦ ⎣

(11)

Ksi

I

∑ K is

as =

∑ j=1 i=1

αjs Φ(x sj)

(12)

I

∑ K is

1 − 2 ∑ij==11

Rs =

I

αjsK (x 0s ,

x sj)

∑ K is

+

I

∑ K is

∑ij==11 ∑li==11

αjsαlsK (x sj , x ls)

(13)

where s = 1,2,···,S, K(xj,xl) = ⟨Φ(xj),Φ(xl)⟩ is the kernel function for each pair of data samples, αsj is the Lagrange multiplier of each data sample in the sth phase, and xs0 is one of the support vectors determined by the SVDD model. Actually, a support vector is a normal data sample that lies in the surface of the hypersphere in the SVDD model. Therefore, the distance between the support vector and the center of the hypersphere is exactly the radius of the hypersphere, which serves as the control limit of the monitoring model. Therefore, the data sample should be considered as a normal one if its distance to the center of the hypersphere does not exceed the radius. Otherwise, the data sample will violate the modeling region of the SVDD method. Hence, a statistic based on the distance between the data sample and the center of the SVDD hypersphere can be developed for batch process monitoring; this statistic, denoted as Dist, is given by22

(9)

where Yi(Ki × Jy) is the quality matrix that is copied Ki times from the quality variable yi and N(0,σ2) is a Gaussian distribution data set with appropriate dimensions. Similarly, the noise-injected quality data set Y(∑i I= 1Ki × Jy) can also be divided into different phases: ⎧[Y1T(K 1 × J )Y 2T(K 2 × J )···Y S T(K S × J )] ⎫ 1 1 1 y 1 y y ⎪ ⎪ 1 1 ⎪ ⎪ ST S 1T 1 2T 2 I ⎪[Y2 (K 2 × Jy )Y 2 (K 2 × Jy )···Y 2 (K 2 × Jy )]⎪ ⎨ ⎬ Y(∑ K i × Jy ) = ⎪ ⎪ ⋮ i=1 ⎪ ⎪ 1T 1 2T 2 S T S ⎪[Y I (KI × Jy )Y I (KI × Jy )···Y I (KI × Jy )] ⎪ ⎩ ⎭

Dist = || Φ(x) − as || I

∑ K is

=

1 − 2 ∑ij==11

I

αjsK (x,

∑ K is

x j) +

I

∑ K is

∑ij==11 ∑li==11


(14)

The confidence limit of the monitoring statistic is the radius of the hypersphere. The confidence level of the monitoring statistic can be determined by tuning the parameter C in the SVDD model, which controls the trade-off between the volume of the hypersphere and the classification error of the model. 3.3.2. Quality Prediction Model. Given the process data set and the quality data set, we intend to develop an RVM regression for quality prediction in each phase of the batch process. Similarly, in the first step the input data Xs and output

(10)

On the basis of each pair of input and output data sets in eqs 8 and 10, a regression model can be developed in each phase of the batch process. An illustration of the data unfolding and phase separation procedure is provided in Figure 2. 3.3. Phase-Based Nonlinear Modeling Approaches. As mentioned above, the variable-wise data unfolding method has 803



Article

data Ys of each phase RVM model are extracted from the whole data set and are given by

where ψs(xs) = [K1(xs,xs1),K2(xs,xs2),···,K ∑I

i=1

⎡ Y s (K s × J ) ⎤ y ⎥ ⎢ 1 1 ⎢ Y s (K s × J )⎥ I 2 2 y ⎥ Y s(∑ K is × Jy ) = ⎢ ⎢ ⎥ ⋮ i=1 ⎢ ⎥ ⎢ Y sI (KIs × J ) ⎥ y ⎦ ⎣

f (x s , ws̅ ) =

∏ j=0

(15)

N (ws , j|0, αs , j−1) I

=

∑ K is

1 I

∏ j=0 i=1

s

(2π )(∑i=1 K i + 1)/2

αs , j1/2

⎛ α w 2⎞ s,j s,j ⎟ exp⎜⎜ − 2 ⎟⎠ ⎝ (16)

(21)

I

∑ K is

= s

RVs

Dist new(s) = || Φ(x new) − as ||

The optimal value of αs can be obtained by optimizing the following likelihood function: 2

s

1 − 2 ∑ij==11 αjsK (x new , x sj) +

I

I

∑ K is

∑ K is

i=1

i=1

∑ j = 1 ∑l = 1

2

max L(αs , σe ) = max log{p(Y |X , αs , σs , e )}

where s = 1,2,···,S. When the volume of each hypersphere is taken into consideration, the relative distance between the new data sample xnew and the center of the sth hypersphere (RDistnew) can be defined as

∫

(17)

As a result, the optimal values of αs and σs,e can be determined, from which the posterior distribution of the weighted parameter ws can be calculated as follows:29 2

RDist new(s) =

p(Y s|ws , σs , e 2)p(ws|αs)

(23)

RDist new(h) = min{RDist new(s)}s = 1,2, ···, S ph(x new) = h

(24)

where h is the determined phase index of the new data sample. 3.4.2. Online Monitoring. In most normal cases, the data sample xnew will stay inside the SVDD hypersphere region that is constructed in phase h and be outside all of the other SVDD hyperspheres:

(18)

the mean and variance values of which are given by (19)

and Σs = [σs , e

Dist new(s) || Φ(x new) − as || = Rs Rs

where Rs is the radius of the sth SVDD hypersphere. To determine the phase affiliation of the new data sample, the following discriminant criterion can be used:

p(Y s|αs , σs , e 2) 1 = |Σs|−1/2 I (∑i = 1 K is + 1)/2 (2π ) ⎡ 1 ⎤ × exp⎢ − (ws − μs )T Σs−1(ws − μs )⎥ ⎣ 2 ⎦

ws̅ = μs = σs , e−2 Σsψ T(x s)Ys


(22)

= max log{ p(Y s|Xs , ws , σs , e 2)p(ws|αs) dws}

p(ws|Y s , αs , σs , e 2) =

)] is

where s = 1,2,···,S and RVs is the number of relevance vectors in the sth phase RVM model. 3.4. Online Monitoring and Quality Prediction. After the monitoring and quality prediction models have been constructed in each phase of the batch process, online algorithms can be implemented for the new batch in specific time intervals. An important issue is how to localize the phase affiliation of each data sample. Since different monitoring and prediction models have been developed in different phases, this localization issue should be fixed carefully. Here a discriminant model is utilized for online phase localization of the data sample, which is based on the relative distances of the data sample to different SVDD hyperspheres. When the phase affiliation of the data sample has been determined, the corresponding models can be employed for online monitoring and quality prediction. 3.4.1. Discriminant Model for Online Phase Localization. For an online data sample xnew ∈ RJ in the new batch, the distance between this data sample and the center of the sth SVDD hypersphere (Distnew) can be calculated as follows:

I

p(ws|αs) =

s I ∑i = 1 K is

∑ ws̅ ,jK (x s, x sj) + ws̅ ,0 = w̅ Ts ψsRV (x s) j=1

Then the data scaling procedure is carried out for both the input and output data sets. For simplicity, we present the RVM model for only one quality variable in each phase of the batch process, which means Jy = 1 in eq 15. Therefore, the training data set samples for each phase RVM model can be represented as {xsj ,ysj } with j = 1,2,···∑i I= 1Ksi . On the basis of the principle of the RVM method, we first assume a Gaussian-type prior distribution for the weighted parameter, given by ∑ K is

s

the set of kernel functions corresponding to the sth training data sample As = diag(αs,0,αs,1,···,αs , ∑I K is ). As a result, many i=1 elements of w̅ s are zero. Only those data samples whose corresponding weighted parameters are nonzero are determined as relevance vectors in the RVM model. Finally, each phase-based RVM model can be formulated as follows:

⎡ X s (K s × J ) ⎤ ⎢ 1 1 ⎥ I s s ⎢ ⎥ ( ) X K × J Xs(∑ K is × J ) = ⎢ 2 2 ⎥ ⋮ ⎢ ⎥ i=1 ⎢ s s ⎥ ⎣ XI (KI × J ) ⎦

i=1

(x , x

K is

RDist new(h) ≤ 1 ψsT(x s)ψs(x s)

−2

−1

+ A s]

RDist new(s)s = 1,2, ···, S , s ≠ h > 1

(20) 804

(25)



Article

Figure 3. Penicillin fermentation process.

The monitoring statistic for the new data sample can be calculated using the hth SVDD model and is given by

Table 1. Phase Division Results for the Nine Training Batches

Dist new(h) = || Φ(x new) − ah || I

∑ K ih

=

1 − 2 ∑ j=1 i=1

I

αjhK (x new ,

x hj )

+

no.

phase #1 (h)

phase #2 (h)

1 2 3 4 5 6 7 8 9

0−44 0−45 0−43 0−45 0−46 0−44 0−43 0−44 0−42

45−400 46−405 44−395 46−410 47−390 45−420 44−385 45−415 43−380

I

∑ K ih

∑ K ih

i=1

i=1

∑ j = 1 ∑l = 1

αjhαlhK (x hj ,

x lh)

(26)

When the value of the monitoring statistic exceeds the control limit Distnew(h) > Rh, this new data sample is judged to be abnormal, and thus, a fault is detected in the hth phase of the batch process. To differentiate the fault from the normal phase change of the process, the monitoring decision can be made as follows

Table 2. Variables Selected in the Penicillin Production Process

⎫ Dist new(h) > R h ⎪ ⎬ → fault Dist new(h + 1) > R h + 1⎪ ⎭ ⎫ ⎪ ⎬ → phase change Dist new(h + 1) ≤ R h + 1⎪ ⎭ Dist new(h) > R h

(27)

3.4.3. Online Quality Prediction. Similarly, for quality prediction of a new batch xnew at a specific time interval, the corresponding RVM model can be determined by the phase discriminant result. Therefore, the prediction result of xnew can be calculated as follows: ψhRV (x new) = [1, K (x new , x1h), K (x new , x h2), ···, K (x new , x hRVh)]T

no.

variable

1 2 3 4 5 6 7 8 9

aeration rate (L/h) agitator power (W) glucose feed temperature (K) dissolved oxygen concentration (% saturation) culture volume (L) carbon dioxide concentration (mmol/L) pH temperature (K) cooling water flow rate (L/h)

Performance evaluation of the quality prediction in each phase of the batch process can be done using the root-meansquare error (RMSE) criterion, which is defined as follows:

ynew ̂ = w̅ hTψhRV (x new) (28)

K

where w̅ h is the weighted parameter, RVh is the number of relevance vectors in the corresponding RVM model, and ψRV h (xnew) is the kernal function set of the RVM model.

RMSE(s) = 805

∑ j =s 1 || yj − ŷ sj ||2 Ks

(29)



Article

Figure 4. Phase localization results for the normal batch.

Figure 5. Process monitoring results for the normal batch using (a) phase-based SVDD and (b) phase-based PCA. 806



Article

Figure 6. Process monitoring results for the step fault using (a) phase-based SVDD and (b) phase-based PCA.

where s = 1,2,···,S, ŷsj (j = 1,2,···,Ks) are the predicted values obtained in the sth phase of the batch process, and yj are the corresponding measured values. Similarly, the overall batch RMSE index can be calculated as

glucose concentration reaches a certain threshold value, which was chosen as 0.3 g L−1 in the present study. After that, the penicillin starts to be produced until the end of the batch process. In this process, cell growth must continue at a certain minimum rate to maintain high penicillin productivity. It is for this reason that glucose is fed continuously into the system during fermentation instead of being added all at once at the beginning. A detailed description has been given by Birol et al.35 The batch data set was generated using a simulator (PenSim version 2.0) developed by the monitoring and control group at the Illinois Institute of Technology (http://www.chee.iit.edu/ ∼cinar). The flowsheet of the penicillin cultivation process is illustrated in Figure 3. Here the whole duration of each batch in the process was selected to be around 400 h (with variations for different batches). In this case, the duration of the first phase was about 45 h, and then the system changed to the fed-batch production phase. The sampling interval was chosen as 1 h in the present study. Nine different batches were generated for modeling training by changing the initial conditions of the substrate concentration and the biomass concentration in the

K

RMSE =

∑ j =new1 || yj − yĵ ||2 K new

(30)

where Knew is the time duration of the new batch.

4. ILLUSTRATIONS AND RESULTS In this section, the feasibility and efficiency of the proposed method are evaluated through the well-known penicillin benchmark process, which has been widely used for algorithm evaluation and testing in the past years.31−34 4.1. Process and Data Sets. The penicillin fermentation process is a typical multiphase batch process that can be partitioned into two operation phases: the preculture phase and the and fed-batch production phase. Most of the necessary cell mass is generated during the initial preculture phase. The system is switched to the fed-batch production phase when the 807



Article

Figure 7. Process monitoring results for the ramp fault using (a) phase-based SVDD and (b) phase-based PCA.

models, respectively. For quality prediction purposes, the kernel parameters of both RVM models were selected as 2. The numbers of relevance vectors of these two models were determined as 22 and 149. For comparison, two traditional linear models were also constructed, namely, PCA for process monitoring and PLS for quality prediction. On the basis of the cross-validation criterion, the numbers of principal components were selected as 4 and 5 in the two phase-based PCA models. Similarly, the latent variables in the two phase-based PLS models were determined to be 4 and 5, respectively. First, the monitoring performance of the two phase-based SVDD models was evaluated. On the basis of the discriminant model, the phase affiliation of each data sample in the testing batch could be located. The detailed phase localization results are given in Figure 4, in which the blue line represents the relative distance of each data sample to the first SVDD model, the black line represents the relative distance of each data sample to the second SVDD model, and the red line represents the cutoff value. It can be seen that the first 44 samples of the batch data were located in the first phase, while the remaining 356 data samples were determined to be in the second phase of

volume. The durations of the two phases in these nine batches are different from each other, as shown in Table 1. For the process monitoring and quality prediction study, a total of nine variables were selected, and these are listed in Table 2. The penicillin concentration was used as the quality variable in this process. To test the performance of the monitoring and quality prediction models, several additional normal batches were also generated. Moreover, two abnormal batches were also generated for fault detection performance evaluation: (1) a step change of the aeration rate starting at 200 h and (2) a ramp change of the agitator power starting at 200 h. 4.2. Results and Discussion. On the basis of the modeling procedures given in section 3, both the SVDD monitoring model and the RVM quality prediction model can be developed for each of the two phases in this process. After data unfolding, the modeling data sets are X1(396 × 9) and y(396 × 1) in the first phase and X2(3204 × 9) and y(3204 × 1) in the second phase. The C parameters of the two SVDD models were selected as 0.0505 and 0.0062, and the confidence levels of both monitoring models were set at 95%. As a result, 21 and 161 support vectors were determined in the first and second SVDD 808



Article

Figure 8. Quality prediction results for the first testing batch using (a) phase-based RVM and (b) phase-based PLS.

the process. The monitoring results of both phase-based SVDD and PCA models are shown in Figure 5. While the monitoring results are pretty good for the SVDD model (Figure 5a), lots of false alarms were generated by the PCA model (Figure 5b), probably because PCA is only a linear modeling method. Since we unfolded the three-way data set through the variable direction and ignored the batch normalization step, there was probably some nonlinearity of the process data in the modeling data set. Figure 6 shows the monitoring results for the two models for the step fault starting from 200 h until the end of the batch. Comparing these two results, we can find that the fault detection sensitivity of the SVDD model is higher than that of the PCA model, as the false alarm rate is much lower in Figure 6a than in Figure 6b. Similarly, detailed monitoring results for the ramp-type fault are given in Figure 7 for the (a) SVDD and (b) PCA methods. Again, both the effectiveness and the superiority of the phase-based SVDD model can be confirmed. Second, the quality prediction performance of the phasebased RVM model was evaluated using two testing batches. In order to decide which phase-based model (RVM or PLS)

Table 3. Phase and Overall RMSEs for the Two Testing Batches batch first second

method

phase #1 (h)

phase #2 (h)

overall

RVM PLS RVM PLS

0.0157 0.0205 0.0117 0.0121

0.0145 0.0162 0.0100 0.0111

0.0147 0.0167 0.0102 0.0112

should be used for quality prediction, the discriminant model should also be used in this stage. The prediction results for the first testing batch using the RVM and PLS models are demonstrated in Figure 8a,b, respectively. In each panel, the actual measured value of the penicillin concentration is plotted as a red line for comparison. While all of the PLS-predicted values during the whole batch time are smaller than the actual value, the results provided by RVM are much better. Precisely, the RMSEs of both prediction models during the whole batch time and in the two operation phases are given in Table 3. It can be seen that the RMSEs of the RVM model are smaller than those of the PLS model. Similarly, the quality prediction 809



Article

Figure 9. Quality prediction results for the second testing batch using (a) phase-based RVM and (b) phase-based PLS.

significant nonlinearities among the process variables. Therefore, nonlinear modeling techniques have been introduced for both process monitoring and quality prediction in each phase of the batch process. For online implementation, an SVDD-based phase localization strategy has been developed for online discrimination of data samples to different phases. On the basis of tests using a typical multiphase penicillin production batch process, the performances of both online monitoring and quality prediction of the proposed method are improved compared with the existing approaches. However, there are still several important issues that need further investigation. For example, the current model does not incorporate the time-dependent information of the data, which may affect the prediction accuracy. For online implementation of the model, the phase affiliation of the new data sample is determined by a hard assignment strategy. In practice, however, the new data sample may not simply be assigned to a single SVDD model. Also, a faulty sample may be misclassified into a wrong phase, which will affect the monitoring and quality prediction performance of the developed method. All of these issues could be of interest for batch process modeling and monitoring.

results for the second testing batch are shown in Figure 9 for the (a) RVM and (b) PLS models. Again, the RVM-predicted results are much better than those based on the PLS model, as can also be seen from the RMSEs shown in Table 3. Compared with the traditional PLS model, the RVM model is effective for nonlinear modeling, and thus, the nonlinearity between the process variables and the final quality variable can be captured well. As mentioned before, the nonlinear modeling method is particularly useful when the process data have been unfolded without the batch normalization step. Therefore, under the current modeling framework, nonlinear methods are strongly recommended for both process monitoring and quality prediction purposes.

5. CONCLUDING REMARKS This paper is mainly focused on the uneven length problem of multiphase batch processes. Different from existing methods, the method developed herein is both simple and effective and is also particularly useful when the number of modeling batches is limited. In the proposed method, the three-way batch data set is unfolded through the variable direction, which may result in 810



■

Article

(19) Faggian, A.; Facco, P.; Doplicher, F.; Bezzo, F.; Barolo, M. Multivariate statistical real-time monitoring of an industrial fed-batch process for the production of specialty chemicals. Chem. Eng. Res. Des. 2009, 87, 325−334. (20) Zhao, C. H.; Mo, S. Y.; Gao, F. R.; Lu, N. Y.; Yao, Y. Statistical analysis and online monitoring for handling multiphase batch processes with varying durations. J. Process Control 2011, 21, 817−829. (21) Lu, N. Y.; Yang, Y.; Wang, F. L.; Gao, F. R. A stage-based monitoring method for batch processes with limited referenced data. In 7th International Symposium on Dynamics and Control of Process Systems (Dycops-7); 2004. (22) Tax, D. M. J.; Duin, R. P. W. Support vector domain description. Mach. Learn. 2004, 54, 45−66. (23) Ge, Z. Q.; Xie, L.; Kruger, U.; Lamont, L.; Song, Z. H.; Wang, S. Q. Sensor fault identification and isolation for multivariate nonGaussian processes. J. Process Control 2009, 19, 1707−1715. (24) Ge, Z. Q.; Gao, F. R.; Song, Z. H. Batch process monitoring based on support vector data description method. J. Process Control 2011, 21, 949−959. (25) Liu, X. Q.; Li, K.; McAfee, M.; Irwin, G. W. Improved nonlinear PCA for process monitoring using support vector data description. J. Process Control 2011, 21, 1306−1317. (26) Ge, Z. Q.; Song, Z. H. Bagging support vector data description model for batch process monitoring. J. Process Control 2013, 23, 1090− 1096. (27) Ge, Z. Q.; Song, Z. H. Nonlinear soft sensor development based on relevance vector machine. Ind. Eng. Chem. Res. 2010, 49, 8685− 8693. (28) Ge, Z. Q.; Song, Z. H.; Gao, F. R. Nonlinear quality prediction for multiphase batch processes. AIChE J. 2012, 58, 1778−1787. (29) Tipping, M. E. Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 2001, 1, 211−244. (30) Yao, Y.; Gao, F. R. A survey on multistage/multiphase statistical modeling methods for batch processes. Annu. Rev. Control 2009, 33, 172−183. (31) Tian, X. M.; Zhang, X. L.; Deng, X. G.; Chen, S. Multiway kernel independent component analysis based on feature samples for batch process monitoring. Neurocomputing 2009, 72, 1584−1596. (32) Alvarez, C. R.; Brandolin, A.; Sanchez, M. C. Batch process monitoring in the original measurement’s space. J. Process Control 2010, 20, 716−725. (33) Facco, P.; Bezzo, F.; Barolo, M. Nearest-neighbor method for the automatic maintenance of multivariate statistical soft sensors in batch processing. Ind. Eng. Chem. Res. 2010, 49, 2336−2347. (34) Liu, Y.; Gao, Z. L.; Li, P.; Wang, H. Q. Just-in-time kernel learning with adaptive parameter selection for soft sensor modeling of batch processes. Ind. Eng. Chem. Res. 2012, 51, 4313−4327. (35) Birol, G.; Undey, C.; Cinar, A. A modular simulation package for fed-batch fermentation: Penicillin production. Comput. Chem. Eng. 2002, 26, 1553−1561.

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Tel: +86-571-87951442. Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS This work was supported in part by the National Natural Science Foundation of China (NSFC) (61370029), the National 973 Project (2012CB720500), and the Fundamental Research Funds for the Central Universities (2013QNA5016).

■

REFERENCES

(1) Nomikos, P.; MacGregor, J. F. Monitoring batch processes using multiway principal component analysis. AIChE J. 1994, 44, 1361− 1375. (2) Nomikos, P.; MacGregor, J. F. Multi-way partial least squares in monitoring batch processes. Chemom. Intell. Lab. Syst. 1995, 30, 97− 108. (3) Ge, Z. Q.; Song, Z. H.; Gao, F. R. Review of Recent Research on Data-Based Process Monitoring. Ind. Eng. Chem. Res. 2013, 52, 3543− 3562. (4) Lu, N. Y.; Gao, F. R. Stage-based process analysis and quality prediction for batch processes. Ind. Eng. Chem. Res. 2005, 44, 3547− 3555. (5) Camacho, J.; Pico, J.; Ferrer, A. Multi-phase analysis framework for handling batch process data. J. Chemom. 2008, 22, 632−643. (6) Yao, Y.; Gao, F. R. Phase and transition based batch process modeling and online monitoring. J. Process Control 2009, 19, 816−826. (7) Reiss, R.; Wojsznis, W.; Wojewodka, R. Partial least squares confidence interval calculation for industrial end-of-batch quality prediction. Chemom. Intell. Lab. Syst. 2010, 100, 75−82. (8) Boonkhao, B.; Li, R. F.; Wang, X. Z.; Tweedie, R. J.; Primrose, K. Making use of process tomography data for multivariate statistical process control. AIChE J. 2011, 57, 2360−2368. (9) Wang, D. Robust data-driven modeling approach for real-time final product quality prediction in batch process operation. IEEE Trans. Ind. Inf. 2011, 7, 371−377. (10) Ge, Z. Q.; Zhao, L. P.; Yao, Y.; Song, Z. H.; Gao, F. R. Utilizing transition information in online quality prediction of multiphase batch processes. J. Process Control 2012, 22, 599−611. (11) Chiu, C. C.; Yao, Y. Multiway elastic net (MEN) for final product quality prediction and quality-related analysis of batch processes. Chemom. Intell. Lab. Syst. 2013, 125, 153−165. (12) Ge, Z. Q.; Song, Z. H.; Gao, F. R. Incorporating setting information for maintenance-free quality modeling of batch processes. AIChE J. 2013, 59, 772−779. (13) Kourti, T. Multivariate dynamic data modeling for analysis and statistical process control of batch processes, start-ups and grade transitions. J. Chemom. 2003, 17, 93−109. (14) Rothwell, S. G.; Martin, E. B.; Morris, A. J. Comparison of methods for dealing with uneven length batches. In Computer Applications in Biotechnology 1998 (CAB7): Proceedings of the Seventh IFAC International Conference; Pergamon Press: London, 1998. (15) Kassidas, A.; MacGregor, J. F.; Taylor, P. A. Synchronization of batch trajectories using dynamic time warping. AIChE J. 1998, 44, 864−875. (16) Fransson, M.; Folestad, S. Real-time alignment of batch process data using COW for on-line process monitoring. Chemom. Intell. Lab. Syst. 2006, 84, 56−61. (17) Kaistha, N.; Moore, C. F. Extraction of event times in batch profiles for time synchronization and quality prediction. Ind. Eng. Chem. Res. 2001, 40, 252−260. (18) Lu, N. Y.; Gao, F. R.; Yang, Y.; Wang, F. L. PCA-based modeling and on-line monitoring strategy for uneven-length batch processes. Ind. Eng. Chem. Res. 2004, 43, 3343−3352. 811


Online Monitoring and Quality Prediction of Multiphase Batch

Recommend Documents