Hierarchical Support Vector Data Description for Batch Process

Article pubs.acs.org/IECR

Hierarchical Support Vector Data Description for Batch Process Monitoring Zhaomin Lv and Xuefeng Yan* Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of Education, East China University of Science and Technology, Shanghai 200237, P. R. China S Supporting Information *

ABSTRACT: Batch process monitoring remains a challenging task due to the inherent timevarying dynamics. The occurrence of faults results in the variables having three types of variation behavior, that is, univariate variation without interaction effect, correlated variable variation with interaction effect, or all variable variation. The three behavior situations may change at any time under dynamic scenarios. Considering that an effective batch process monitoring method should clearly identify the complex variation behavior of the variables, a hierarchical support vector data description (HSVDD), which integrates univariate monitoring, subspace monitoring, and wholespace monitoring, is proposed to accurately identify the three variation behaviors. First, threedimensional data are unfolded to batch-wise data with normalization, and the obtained twodimensional data are separated according to the time axis to obtain time-slice modeling data. Then, the hierarchical structure, which contains the univariate variable monitoring structure layer, the subspace monitoring structure layer, and the whole space monitoring structure layer, is designed for each time slice. Meanwhile, support vector data description (SVDD) is adopted as the modeling method in each layer. Lastly, the online hierarchical monitoring model that has the same time point as the online data is selected, and the monitoring results of all models in the hierarchical structure are fused using two methods, that is, obtaining the average weight and the max value. A fault diagnosis method based on a hierarchical monitoring structure is designed. A simple numerical example and fed-batch penicillin fermentation process are employed to demonstrate the effectiveness of the HSVDD.

1. INTRODUCTION Batch process monitoring is essential to guarantee process safety and product quality. Traditionally, multiway principal component analysis (MPCA), multiway partial least-squares (MPLS), multiway independent component analysis (MICA), and support vector data description (SVDD) have been applied to batch process monitoring as the most popular batch process monitoring methods.1−5 Most batch processes have inherent time-varying dynamics, which may increase the monitoring difficulties. Adaptive methodology, which includes a recursive strategy and moving window technique, has been integrated in various monitoring methods to trace the inherent time-varying dynamics.6−11 These adaptive methods use local sample data to develop a local whole-space monitoring structure model to solve the inherent time-varying dynamics. Considering that the inherent time-varying dynamics of the batch process are extremely complex, the occurrence of faults will cause the process variables to have three types of variation behavior, i.e., univariate variation without interaction effect, correlated variable variations with interaction effect, or all variable variation. The three behaviors may switch among each other at any time under dynamic scenarios. If the fault only causes univariate variation without interaction effect, the fault information may be merged easily and may not be detected in the whole-space monitoring structure. If the fault causes correlated variable variation with interaction effect, the local behavior of the process is disregarded © 2016 American Chemical Society

in the whole-space monitoring structure. This disadvantage may also lead to fault information being merged and deterioration of the monitoring performance in the whole space monitoring structure. Thus, the whole-space monitoring structure, which is integrated with the adaptive monitoring strategy, may not explain the complex behavior characteristics between the different variables in a dynamic batch process. In recent years, a large number of researchers found that the process variables have complex behavior characteristics. Several methods based on the subspace monitoring structure, which can focus on process variables with similar behavior characteristics, have been proposed.12−22 Two types of subspace division methods exist. One operates according to prior process knowledge and the other operates automatically according to the data-driven method. Multiple variable subspace methods can help in learning process complexity. A linear subspace method based on PCA is proposed to approximate the nonlinear characteristics of the process.14 Four subspaces are constructed by using PCA to utilize all the information.16 Mutual information and K-means are used to calculate the subspace segmentation rule.20 These three multiple subspace monitoring methods may Received: Revised: Accepted: Published: 9205

March 7, 2016 August 8, 2016 August 8, 2016 August 8, 2016 DOI: 10.1021/acs.iecr.6b00901 Ind. Eng. Chem. Res. 2016, 55, 9205−9214

Article

Industrial & Engineering Chemistry Research

based on a hierarchical monitoring structure is proposed for fault analysis. The remainder of this paper is organized as follows: Section 2 introduces a brief presentation of the SVDD and the multiple subspace division method. Section 3 introduces the hierarchical design and the integration method of the monitoring result. Section 4 validates the feasibility and effectiveness of the hierarchical monitoring structure by simulating a simple numerical model and the fed-batch penicillin fermentation process. Section 5 presents the conclusions.

lose some correlation information among variables in different subspaces. Subspace division of the hidden variables of the original process variables is proposed.21,22 However, the subspace monitoring methods mainly consider the correlated variable variation with interaction effect under non-normal conditions. Both the univariate variation without interaction effect and the all variable variation are ignored. If the fault only causes univariate variation without interaction effect, the fault information may still be merged easily in the subspace monitoring structure. If the fault causes all variable variation, some correlation information will not be used in the subspace monitoring structure, which may have disadvantages in detecting faults. Meanwhile, the adaptive strategy is not integrated with the subspace monitoring methods in most subspace monitoring methods, that is, the inherent time-varying dynamics of the batch process are not considered. In the dynamic batch process, the correlation of variables changes with time. When a fault exists in the process, the fault information in the variables may be found in three behavior situations: in one variable, in the local space, or in the whole space, and the three behavior situations may switch among each other. If the fault information exists in one variable, a univariate variable monitoring structure may be better than a subspace monitoring structure or whole-subspace monitoring structure. If the fault information exists in the local space, the subspace monitoring structure may be better than the univariate variable monitoring structure and the whole-space monitoring structure. The traditional univariate variable monitoring structure may be unable to detect the fault because correlation information is not used in this type of univariate variable monitoring structure, and such a variation may not be detected in any univariate variable. If the fault information exists in the whole space, the whole-space monitoring structure may be better than the univariate variable monitoring structure and the subspace monitoring structure. Only in the whole-space monitoring structure can all the correlation information on the variables be used. Up to now, most batch process monitoring methods, which are integrated with an adaptive strategy, have one monitoring structure. Thus, the dynamic problem in batch process monitoring has not been completely solved. To solve this problem, this study proposes a hierarchical monitoring structure that integrates the univariate variable monitoring structure, the subspace monitoring structure, and the whole-space monitoring structure. The hierarchical monitoring structure can detect the fault more effectively than any single univariate monitoring structure, whether the fault information exists in one variable or in more conditions. First, the threedimensional data are unfolded to batch-wise with normalization and separated according to the time axis to obtain the time-slice modeling data, which is adopted as the adaptive monitoring strategy in this paper. Then, the hierarchical structure, which contains the univariate variable monitoring structure layer, the subspace monitoring structure layer, and the whole-space monitoring structure layer, is designed for each time slice of the modeling data. Meanwhile, the support vector data description (SVDD) is adopted as the modeling method in each layer. This type of adaptive monitoring strategy, which is integrated with the hierarchical structure and can capture the three types of variation behavior, may have better monitoring performance in dynamic batch processes. Lastly, the online monitoring model is selected according to time, and the monitoring result of each model is fused by two methods, that is, weight averaging and the max value. A fault diagnosis method

2. PRELIMINARIES In this part, the SVDD modeling method and the subspace division method used in this paper are briefly introduced. 2.1. SVDD Modeling Method. The SVDD modeling method is introduced briefly.4 The aim of the SVDD is to find a hypersphere with the least radius. This hypersphere can be described by an optimization equation n

min R2 + C ∑ ξi

s. t. = || Φ(yi) − a || ≤ R2 + ξi

i=1

(1)

where a represents the center of the hypersphere, R2 represents the squared radius from the center a to the boundary, C is the trade-off between R2 and wrong clustering samples, ξi is the relaxation vector, and Φ is a kernel function. The optimization equation can be expressed in another form n

n

min ∑ αiKk(yi , yj) − αi

n

∑ ∑ αiαjKk(yi, yj)

i=1

i=1 j=1 n

s.t. 0 ≤ αi ≤ C ,

∑ αi = 1

(2)

i=1

where K1(vi,vj) = ⟨Φ(vi), Φ(vj)⟩. A group of αi can be obtained by calculating the above equation and vi, whose αi > 0 is defined as SV. For any one v ∈ SV, the squared radius R2 is calculated as follows: n

n

R2 = Kk(y, y) − 2 ∑ αiKk(yi , y) + i=1

n

∑ ∑ αiαjKk(yi, yj) i=1 j=1

(3)

The squared distance from the online sample x to the center a is calculated as follows: n

n

D2 = Kk(x , x) − 2 ∑ αiKk(yi , x) + i=1

n

∑ ∑ αiαjKk(yi, yj) i=1 j=1

(4)

The index is defined as DR = D2/R2, and the control limit is DR,lim = 1. 2.2. Subspace Division Method. The multiple variable subspace division method based on mutual information and Kmeans is adopted to obtain the subspace layer in this paper and is introduced briefly.20 X(K × J) is a two-dimensional matrix, where K represents the sample number and J represents the variable number. The entropy of one variable can be calculated as follows: K

H(y) = − ∑ p(y k ) log p(y k ) k=1

9206

(5) DOI: 10.1021/acs.iecr.6b00901 Ind. Eng. Chem. Res. 2016, 55, 9205−9214

Article

Industrial & Engineering Chemistry Research where yk is the kth sample of variable y(K × 1), and p(yk) represents the probability density function of yk. The joint entropy of two variables can be calculated as follows: K

division method for each subspace layer should be determined by the monitoring aims. Only one subspace layer is adopted in this article, and the subspace layer is obtained according to the subspace division method mentioned in the preliminaries. Figure 1 illustrates the

K

∑ p(y1k , y2k ) log p(y1k , y2k )

H(y1, y2) = − ∑

1

2

1

2

k1= 1 k 2 = 1 k k where p(y11, y22) represents the joint probability density k1 k2 of y1 and y2 . The mutual information on two variables,

(6)

function i.e., I(y1,

y2), can be calculated as follows: I(y1, y2) = H(y1) + H(y2) − H(y1, y2) K

=

K

∑ ∑ p(y1k , y2k ) log 1

2

k1= 1 k 2 = 1

p(y1k1 , y2k 2 ) p(y1k1 )p(y2k 2 )

Figure 1. Flow diagram of the hierarchical monitoring structure.

structure of hierarchical monitoring in this paper. SVDD is adopted as the modeling method. The first layer is the univariate variable layer. All the variables are monitored separately, and each variable is built based on one SVDD monitoring model. The second layer is the subspace layer. The variables in the same subspace are monitored together, and each subspace is built based on one SVDD monitoring model. The third layer is the whole-space layer. All the variables are monitored together, and only one SVDD monitoring model is built. 3.2. Monitoring Result Fusion. In online monitoring, the online sample requires calculation in all the hierarchical monitoring models. Then, all the monitoring results are integrated. In this paper, two calculation methods of integration are used to calculate the final monitoring result, and a fault diagnosis method based on the hierarchical structure is proposed. The final monitoring result is obtained in two steps. First, the monitoring result of each layer is obtained, and second, the monitoring results of all the layers are integrated. In this paper, a weighting average is adopted as the first step to obtain the monitoring result of the subspace layer and univariate variable layer. The calculation is as follows:

(7)

The correlation of two variables, i.e., IR(y1, y2), can be calculated as follows: IR(y1, y2) =

I(y1, y2) min(H(y1), H(y2))

(8)

A relevant matrix RD can be calculated as follows: R D(j1 , j2 ) = IR(yj , yj ) 1

j2 = 1, 2, 3..., J

2

j1 = 1, 2, 3..., J , (9)

where yj1(K × 1) represents the j1th variable, and yj2(K × 1) represents j2th variable. On the basis of RD, the K-means clustering algorithm is adopted to cluster similar relevant feature vectors to obtain the multiple subspace segmentation rule.23 In this study, the variables are divided into two subspaces.

3. HIERARCHICAL SUPPORT VECTOR DATA DESCRIPTION The hierarchical monitoring structure contains three steps. First, the hierarchical structure design of the process is applied to each time-slice data set. Then, the SVDD models are built. Both steps are performed offline. During online monitoring, the monitoring result of each model is integrated. 3.1. Hierarchical Structure Design. Batch process data are a three-dimensional data set X(I × J × K). All batch data are assumed to have the same length in this paper. Before modeling, all the data are transformed to batch-wise unfolding. Then, all the data are normalized and transformed to variable-wise unfolding XV(IK × J) with K time intervals see (Supporting Information). The hierarchical SVDD is applied to each time interval. When an online sample exists, the corresponding time interval model is adopted as the current monitoring model. The hierarchical structure contains three types of layers: the univariate variable layer, the subspace layer, and the whole-space layer. Both the univariate variable layer and whole-space layer are easy to obtain, whereas the subspace layer is difficult to obtain. There are two steps to obtain the subspace layers. First, the number of subspace layers is determined. In most cases, this parameter is determined by the monitoring aims and the number of variables. If there is a large number of monitoring variables, the number of subspace layers may be greater. If there is a small number of monitoring variables, the number of subspace layer may be less. Meanwhile, according to the different monitoring aims, there may be other subspaces layers. Second, the subspace

N

DR i =

∑s = 1 (DR is)2 N

∑s = 1 DR is

(10)

DRji

where DRi is the monitoring result of the ith layer, is the monitoring result of the jth model of the ith layer and is also used as the weighting coefficient, and N is the number of the model. In the second step, the weighting average and maximum are adopted to obtain the final monitoring result. They can be calculated as follows: 3

DR =

∑i = 1 (DR i)2 3

∑i = 1 DR i

DR = max(DR i)

(11)

i = 1, 2, 3

(12)

In addition to the above two value calculation methods, a fault diagnosis method is proposed to support the hierarchical monitoring structure. Each model in the hierarchical structure is conducted as a voting unit. In the univariate variable monitoring layer, if one model detects the fault, then the variable receives one vote. vj1 represents the jth variable in the univariate variable monitoring layer. The voting result is calculated as follows: v1j = 1

if

DR ij > 1

j = 1 ··· J

(13)

In the subspace monitoring layer, if one model detects the fault, then the variables in the same subspace all receive one vote. 9207

DOI: 10.1021/acs.iecr.6b00901 Ind. Eng. Chem. Res. 2016, 55, 9205−9214

Article

Industrial & Engineering Chemistry Research vj2 represents the jth variable in the subspace monitoring layer, and Vs represents the sth subspace variable set. The voting result is calculated as follows: v2j = 1

if DR 2s > 1

v2j ∈ Vs

j = 1 ··· J

4. APPLICATION EXAMPLE A simple numerical example and fed-batch penicillin fermentation process are simulated to demonstrate the effectiveness and feasibility of HSVDD. A whole-space SVDD (WSVDD), a subspace SVDD (SSVDD), and a univariate variable SVDD (USVDD) are compared with HSVDD. HSVDD(A) is used to represent the integrated monitoring results of each layer using a weighting average. HSVDD(M) is used to represent the integrated monitoring results of each layer by obtaining the maximum. The false alarm rate, missed alarm rate, error rate, and first instance of fault detection are adopted to compare the monitoring methods.20 The voting results of the designed fault diagnosis method are shown in the figure. 4.1. Case Study of a Numerical Process. A simple numerical process is designed as follows:

s = 1 ··· S (14)

In the whole-space monitoring layer, if the model detects the fault, then all the variables receive one vote. The voting result is calculated as follows: v3j = 1

if

DR3 > 1

j = 1 ··· J

(15)

In this way, the number of votes of any one variable can be calculated as follows:

v j = v1j + v2j + v3j

(16)

j

v is the jth variable. The result may have four conditions: 0, 1, 2, 3. Different voting number values have different colors. More voting means the variable may contain fault information. Then, the voting results of all the variables are displayed in one colorful figure, which can help in analyzing the fault. Figure 2 presents a flow diagram of batch process monitoring based on HSVDD. The step-by-step procedure is presented as follows:

⎤ ⎡1 1 1 1 1 1 1 1 ⎥ ⎢ ⎥ ⎢ 1 1 1 1 1 1 1 ⎥ ⎢ 1 1 1 1 1 1 ⎥ ⎢ 1 1 1 1 1 ⎥ ⎢ 1 1 1 1 ⎥ ⎢ ⎥ ⎢ 1 1 1 ⎥ ⎢ 1 1 ⎥ ⎢ ⎥ ⎢ 1 ⎥ P=⎢ 1 ⎥ ⎢ ⎥ ⎢ 1 1 ⎥ ⎢ 1 1 1 ⎥ ⎢ ⎥ ⎢ 1 1 1 1 ⎥ ⎢ 1 1 1 1 1 ⎥ ⎢ ⎥ ⎢ 1 1 1 1 1 1 ⎥ ⎢ 1 1 1 1 1 1 1 ⎥ ⎢ ⎣ 1 1 1 1 1 1 1 1⎦

(17)

t1 = 16·randi(10, 800, 1) t 2 = 15·randi(10, 800, 1) t3 = 14·randi(10, 800, 1) t4 = 13·randi(10, 800, 1) Figure 2. Flow diagram of batch process monitoring based on HSVDD.

t5 = 12·randi(10, 800, 1) t6 = 11·randi(10, 800, 1) t 7 = 10·randi(10, 800, 1)

Offline modeling 1. The 3D array is unfolded through batch-wise unfolding. 2. The data set is normalized along the batch axis and transformed to variable-wise. 3. The data are divided according to the time axis. 4. The SVDD models are built based on the hierarchical monitoring structure. Online monitoring a. The monitoring model is selected based on the time of the online data. b. The online data are normalized according to the data of the monitoring model. c. The monitoring results of all the models are obtained. d. The voting decision is used to show the final monitoring result.

t8 = 9·randi(10, 800, 1) t 9 = 8·randi(10, 800, 1) t10 = 7·randi(10, 800, 1) t11 = 6·randi(10, 800, 1) t12 = 5·randi(10, 800, 1) t13 = 4·randi(10, 800, 1) t14 = 3·randi(10, 800, 1) t15 = 2·randi(10, 800, 1) t16 = 1·randi(10, 800, 1) 9208

(18) DOI: 10.1021/acs.iecr.6b00901 Ind. Eng. Chem. Res. 2016, 55, 9205−9214

Article

Industrial & Engineering Chemistry Research T = [t1 t12

t2 t13

t3 t14

t4 t15

t5

t6

t7

t8

t9

t10

t16]

(19)

noise = 0.5·randi(10, 800, 16)

Y ( i , j) =

point. The step extents are 3.0, 2.5, 2.5, 2.0, 2.0, 1.5, and 1.0. The third case is a univariate variable fault. The 16th variable has an up step fault from the 201st sample point to the 800th sample point. Its numerical value is 3.0. The fourth case is a combination of the three above faults. The period from the 201st sample point to the 400th sample point is a whole-space fault. The period from the 401st sample point to the 600th sample point is a subspace fault. The period from the 601st sample point to the 800th sample point is a univariate variable fault. Table 3 shows a comparison of the false alarm rate when using WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the numerical process. For all four cases, the false alarm rates of WSVDD, SSVDD, USVDD, and HSVDD(W) are similar. The false alarm rate of HSVDD(M) is slightly larger than the other methods. A comparison of the missed alarm rate when using WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the numerical process is shown in Table 3. For the first case, which is a whole-space fault, WSVDD, which is a whole space monitoring method, can detect the fault better than SSVDD and USVDD. For the second case, which is a subspace fault, SSVDD, which is a subspace monitoring method, can detect the fault better than WSVDD and USVDD. For the third case, which is a univariate variable fault, USVDD, which is a univariate variable monitoring method, can detect the fault better than WSVDD and SSVDD. For all cases, the missed alarm rate of HSVDD(M) is better than that of the other methods. A comparison of the error rate when using WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the numerical process is shown in Table 3. The comparison of the error rates indicates the missed alarm rate, and the result of HSVDD(M) is still the best, which demonstrates that HSVDD(M) can distinguish the normal data and the fault data more effectively than other methods in the numerical process. The comparison of the first-time fault detection using WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the numerical process is shown in Table 3. All four HSVDD(M) results scored the highest. The monitoring charts of WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) for the fourth case are shown in Figure 3. Figure 3a shows that WSVDD can detect the fault from approximately the 200th sample time to approximately the

t11

(20)

Ti ·P(: , j) + noise(i , j)

i = 1: 800

j = 1: 16

(21)

where t1 to t16 are uniformly distributed, and they all have 800 sample points. Noise is included in the process, each element of which is uniformly distributed. In the offline modeling, the hierarchical structure of the numerical process is designed. The results are shown in Table 1. Table 1. Hierarchical Structure of the Numerical Process structure whole space layer subspace layer univariate variable layer

variable no. (1−16) (1−8)(9−16) (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14) (15)(16)

The variables in the same bracket are built using one monitoring model. In the whole-space layer, all the variables are monitored together. In the subspace layer, the subspace is divided according to prior knowledge of the numerical process. Equation 17 shows that the 1st variable to the 8th variable should be gathered in one subspace, and the 9th variable to the 16th variable should be gathered in one subspace. In the univariate variable layer, all the variables are monitored individually. Four cases are designed for monitoring. The corresponding process parameters of the faults are shown in Table 2. The first case is a whole-space fault. All the variables have minimally different up step faults from the 201st sample point to the 800th sample point. The step extents are 1.5, 1.5, 2.5, 2.5, 2.5, 3.0, 3.0, 3.0, 3.0, 2.5, 2.0, 2.0, 1.5, 1.5, 1.0, and 0.5 from the 1st variable to the 16th variable, respectively. The second case is a local fault. The 10th variable to the 16th variable have minimally different up step faults from the 201st sample point to the 800th sample Table 2. Test Cases of the Numerical Process fault 1

fault 2

fault 3

fault duration

201−800

201−800

201−800

201−400

401−600

601−800

variable no.

step value

step value

step value

step value

step value

step value

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

1.5 1.5 2.5 2.5 2.5 3.0 3.0 3.0 3.0 2.5 2.0 2.0 1.5 1.5 1.0 0.5

0 0 0 0 0 0 0 0 0 3.0 2.5 2.5 2.0 2.0 1.5 1.0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3.0

1.5 1.5 2.5 2.5 2.5 3.0 3.0 3.0 3.0 2.5 2.0 2.0 1.5 1.5 1.0 0.5

0 0 0 0 0 0 0 0 0 3.0 2.5 2.5 2.0 2.0 1.5 1.0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3.0

9209

fault 4


Article

Industrial & Engineering Chemistry Research Table 3. Monitoring Results of the WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the Numerical Process false alarm rate

missed alarm rate

error rate

first time to detect fault

fault no.

WSVDD

SSVDD

USVDD

HSVDD(W)

HSVDD(M)

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

1.0 0.5 1.5 0.5 82.5 97.0 98.1 92.3 62.2 73.0 74.1 69.5 220 314 270 201

0.5 0 0.5 0 85.0 74.3 98.6 85.1 64.0 55.8 74.2 64.0 201 205 225 201

0 0 0 0 97.6 91.8 77.7 88.0 73.3 69.0 58.3 66.1 277 212 202 211

0.5 0.5 0.5 0.5 86.6 85.0 85.1 85.3 65.2 64.0 64.1 64.2 220 212 202 201

1.5 0.5 2.0 0.5 78.3 73.3 75.8 74.7 59.2 55.2 57.5 56.2 201 205 202 201

Figure 3. Monitoring charts for case 4 in the numerical process: WSVDD (a), SSVDD (b), USVDD (c), HSVDD(W) (d), and HSVDD(M) (e).

400th sample time better than it can for other periods because WSVDD is a whole-space monitoring method, and the fault from approximately the 200th sample time to approximately the 400th sample time is a whole-space fault. Figure 3b shows that SSVDD can detect the fault better from the 400th time point to the 600th sample time because SSVDD is a whole-subspace monitoring method, and the fault from approximately the 400th sample time to approximately the 600th sample time is a subspace fault. Figure 3c shows that USVDD can detect the fault better from approximately the 600th sample time to the end because USVDD is a univariate variable monitoring method, and the fault from approximately the 600th sample time to the end is a univariate variable fault. Figure 3d shows the monitoring result of HSVDD(W), which has the strongest fault-detection ability. HSVDD(W) has better detection ability than WSVDD from approximately the 400th time point to the end. HSVDD(W) has better detection ability than SSVDD from approximately the 200th time point to approximately the 400th time point and from approximately the 600th time point to the end. HSVDD(W) has

better detection ability than USVDD from approximately the 200th time point to approximately the 600th time point. Figure 3e shows the monitoring result of HSVDD(M), which contains all the fault detection abilities of the four other methods. Thus, its detection ability is the best throughout the entire fault period. The fault diagnosis of HSVDD for the fourth fault in the numerical process is shown in Figure 4. From approximately the 200th time point to the 400th time point, most time points in the variables from the 1th variable to the 8th variable can be detected in the three layers, and only a few time points can be detected in two layers or three layers from the 9th variable to the 16th variable. Thus, the first subspace has more fault information. The first subspace can detect more fault samples than the second subspace. The fault information is merged in the second subspace. However, the second subspace still contains the fault information, which can be determined based on the design of the fault. The fault information is divided into two subspaces. Although the first subspace can detect the fault, it performs more poorly than the whole-space monitoring method, which can be 9210


Article


Table 5. Test Cases of the Fed-Batch Penicillin Fermentation Process

ascertained from the missed alarm rate result of the first fault shown in Table 3. This phenomenon demonstrates that if the fault information is divided into different subspaces, then the fault cannot be detected in some subspaces, and the monitoring performance deteriorates. From approximately the 200th time point to the 400th time point, the second subspace has more fault information. From approximately the 600th time point to the end, the 16th variable has more fault information. 4.2. Case Study of the Fed-Batch Penicillin Fermentation Batch Process. The fed-batch penicillin fermentation industrial process24 is a well-known benchmark process and is used in this section to demonstrate the effectiveness of the HSVDD. All the variables were used in this study. The sampling time was 0.5 h, and the overall duration of each batch was 400 h. Approximately 100 normal batches were simulated as the historical database. For the offline modeling, the hierarchical structure of the fedbatch penicillin fermentation process is designed; the results are shown in Table 4. The variables in the same bracket are built Table 4. Hierarchical Structure of the Fed-Batch Penicillin Fermentation Process structure

variable name

variable no.

type

size (%)

fault duration (h)

1 2 3 4 5 6

aeration rate agitator power substrate feed rate aeration rate agitator power substrate feed rate

1 2 3 1 2 3

step step step drift drift drift

−1.5 −1.5 −2.0 −0.1 −0.1 −0.001

100−400 100−400 100−400 100−400 100−400 100−400

larger than the other methods. The missed alarm rates of WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the fed-batch penicillin fermentation process are compared in Table 6. In cases 1, 2, 4, and 5, the results of SSVDD are better than those of WSVDD and USVDD. In cases 3 and 6, the results of WSVDD are better than those of SSVDD and USVDD. For all cases, the results of HSVDD(M) are the best. The error rates of WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the fed-batch penicillin fermentation process are compared in Table 6. The results of the error rate show the same phenomenon as the missed alarm rate, and the result of HSVDD(M) is still the best, which demonstrates that HSVDD(M) can distinguish the normal data and the fault data better than the other methods in the fed-batch penicillin fermentation process. The first time a fault was detected by WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the fed-batch penicillin fermentation process is compared in Table 6. All six results of HSVDD(M) are the best. In case 3, a 2.5% decrease of the substrate feed rate occurs at the 100th hour and continues to the end of the process. The monitoring charts that use WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) for this abnormal case are shown in Figure 5. Figure 5a shows that the statistics of WSVDD do not exceed the threshold value at approximately 300 h and near the beginning of the 400th hour. Figure 5b shows that the statistics of SSVDD do not exceed the threshold value at approximately the 200th hour, the 300th hour, and the beginning of the 400th hour. Figure 5c shows that the statistics of USVDD do not exceed the threshold value at approximately the 200th hour and the beginning of the 400th hour. However, the statistics of USVDD show fewer undetected points than the statistics of WSVDD and SSVDD at the beginning of the 400th hour. Figure 5d shows that the statistics of HSVDD(W) lead to more detected time points than the three other methods near the 200th hour and around the 300th hour. In comparison, the statistics of HSVDD(W) show more undetected points than the statistics of USVDD at the beginning of the 400th hour. Figure 5e shows the advantages of HSVDD(M), which has a stronger detection ability than the other methods. The superior performance of HSVDD(M) demonstrates that the hierarchical monitoring structure can monitor multiple variable processes effectively. The fault diagnosis of HSVDD for all six cases in the fed-batch penicillin fermentation process is shown in Figures 6 to 11. The fault diagnosis of HSVDD for case 1 is shown in Figure 6. Variable 1 has more fault information. This result is similar to the design of case 1. The fault diagnosis of HSVDD for case 2 is shown in Figure 7. Variable 2 has more fault information. This result is similar to the design of case 2. The fault diagnosis of HSVDD for case 3 is shown in Figure 8. Variables 7, 9, 13, and 16 have more fault information. The phenomenon is different from the above two cases. The result is not similar to the design of case

Figure 4. Fault diagnosis of HSVDD for case 4 in the numerical process.

whole-space layer subspace layer univariate variable layer

fault no.

variable number (1−16) (1,2,4,6,12)(3,5,7,8,9,10,11,13,14,15,16) (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11)(12)(13)(14) (15)(16)

based on one monitoring model. In the whole-space layer, all 16 variables are monitored together. In the subspace layer, the subspace is divided according to the multiple subspace division method that is based on MI and K-means. The results show that the 1st, 2nd, 4th, 6th, and 12th variables are gathered in one subspace, and the other variables are gathered in another subspace. In the univariate variable layer, all the variables are monitored individually. Six batch monitoring cases are simulated to verify the performance of the HSVDD. The corresponding parameters simulated for each fault are shown in Table 5, where each case is run 20 times and the average value is used for comparison. The false alarm rates of WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the fed-batch penicillin fermentation process are compared in Table 6. For all six cases, the false alarm rates of WSVDD, SSVDD, USVDD, and HSVDD(W) are similar. The results of HSVDD(M) are slightly 9211


Article

Industrial & Engineering Chemistry Research Table 6. Monitoring Result of WSVDD, SSVDD, USVDD, HSVDD(W), and HSVDD(M) in the Fed-Batch Penicillin Fermentation Process false alarm rate

missed alarm rate

error rate

first time to detect fault

Fault no.

WSVDD

SSVDD

USVDD

HSVDD(W)

HSVDD(M)

1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6

9.1 6.8 5.1 9.4 5.4 4.6 78.1 71.6 55.6 56.6 95.3 41.1 61.0 55.5 43.1 44.8 72.9 32.0 146 124 169 172 271 170

6.6 5.4 6.2 8.0 4.8 5.3 38.2 33.0 64.5 38.9 85.7 43.6 30.4 26.1 50.0 31.2 65.5 34.0 102 101 146 136 163 148

6.7 4.8 5.4 7.2 4.1 2.9 43.0 35.8 63.8 42.3 87.2 45.3 34.0 28.1 49.2 33.6 66.5 34.8 108 106 148 143 162 166

6.9 5.6 6.0 9.0 4.3 4.1 41.5 34.6 59.9 40.4 88.0 42.8 32.9 27.4 46.5 32.6 67.2 33.2 105 105 143 148 166 158

13.2 10.5 8.7 13.1 8.5 8.1 29.2 24.0 52.0 35.1 80.7 39.3 25.2 20.7 41.2 29.7 62.7 31.5 102 101 131 127 126 131

Figure 5. Monitoring charts for case 3 in the fed-batch penicillin fermentation process: WSVDD (a), SSVDD (b), USVDD (c), HSVDD(W) (d), and HSVDD(M) (e).

case 5. The fault diagnosis of HSVDD for case 6 is shown in Figure 11. The detection result is similar to that of case 3; both cases have faults in variable 3. Using the design of the fault diagnosis of the process monitoring based on hierarchical structure allows for easy detection of the cause of the fault and the fault type in some situations. Thus, the proposed fault diagnosis monitoring based on hierarchical structure is useful and effective.

3, possibly because of the correlation of the variable and the control loop. Thus, this type of fault may cause a large number of variables to lose control. The fault diagnosis of HSVDD for case 4 is shown in Figure 9. Variable 1 has more fault information, and it may be a time-varying fault, which can be ascertained from its change of color. This result is similar to the design of case 4. The fault diagnosis of HSVDD for case 5 is shown in Figure 10. This fault may be too small to be detected. It can be detected at the end of the process only, and the result is similar to the design of 9212


Article


Figure 6. Fault diagnosis of HSVDD for case 1 in the fed-batch penicillin fermentation process.






5. CONCLUSIONS This paper presented a hierarchical SVDD called HSVDD. The variables were gathered in a hierarchical structure that consists of three layers, namely, the whole-space layer, the subspace layer, and the univariate variable layer. SVDD was used to build each monitoring model in each space in all layers. Two methods were used to integrate the monitoring result of each layer. A voting

decision based on the hierarchical structure was designed for fault diagnosis of the monitoring result. HSVDD was tested for a simple numerical process and a fed-batch penicillin fermentation process. The results show that the HSVDD performed better than the other methods. The designed fault diagnosis result based on the voting decision of the hierarchical structure was 9213


Article


(14) Ge, Z.; Zhang, M.; Song, Z. Nonlinear process monitoring based on linear subspace and Bayesian inference. J. Process Control 2010, 20 (5), 676−688. (15) Dunia, R.; Joe Qin, S. Subspace approach to multidimensional fault identification and reconstruction. AIChE J. 1998, 44 (8), 1813− 1831. (16) Tong, C.; Song, Y.; Yan, X. Distributed statistical process monitoring based on four-subspace construction and Bayesian inference. Ind. Eng. Chem. Res. 2013, 52 (29), 9897−9907. (17) Jiang, Q.; Yan, X. Monitoring multi-mode plant-wide processes by using mutual information-based multi-block PCA, joint probability, and Bayesian inference. Chemom. Intell. Lab. Syst. 2014, 136, 121−137. (18) Wang, B.; Jiang, Q.; Yan, X. Fault detection and identification using a Kullback-Leibler divergence based multi-block principal component analysis and bayesian inference. Korean J. Chem. Eng. 2014, 31 (6), 930−943. (19) Huang, J.; Yan, X. Double-step block division plant-wide fault detection and diagnosis based on variable distributions and relevant features. J. Chemom. 2015, 29 (11), 587−605. (20) Lv, Z.; Yan, X.; Jiang, Q. Batch process monitoring based on justin-time learning and multiple-subspace principal component analysis. Chemom. Intell. Lab. Syst. 2014, 137, 128−139. (21) Wang, B.; Yan, X.; Jiang, Q.; et al. Generalized Dice’s coefficientbased multi-block principal component analysis with Bayesian inference for plant-wide process monitoring. J. Chemom. 2015, 29 (3), 165−178. (22) Zhaomin, L.; Qingchao, J.; Xuefeng, Y. Batch process monitoring based on multisubspace multiway principal component analysis and time-series Bayesian inference. Ind. Eng. Chem. Res. 2014, 53 (15), 6457−6466. (23) Hartigan, J. A.; Wong, M. A. Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics) 1979, 28 (1), 100−108. (24) Birol, G.; Ü ndey, C.; Cinar, A. A modular simulation package for fed-batch fermentation: penicillin production. Comput. Chem. Eng. 2002, 26 (11), 1553−1565.

convenient for analyzing the monitoring result. Therefore, HSVDD was effective for process monitoring. The hierarchical monitoring structure used in this paper was a three-layer system, and only one subspace layer was used. Future studies can focus on determining the number of subspace layers effectively. Moreover, as the number of layers increases, the false alarm rate may increase, which is a difficult and important phenomenon to study.

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available free of charge on the ACS Publications website at DOI: 10.1021/acs.iecr.6b00901. Batchwise unfolding; batchwise unfolding transformed to variable-wise unfolding (PDF)

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Tel.: 0086-21-64251036. Address: P.O. BOX 293, MeiLong Road No. 130, Shanghai 200237, P. R. China. Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS The authors gratefully acknowledge the support of the following foundations: 973 Project of China (2013CB733605), National Natural Science Foundation of China (21176073), and the Fundamental Research Funds for the Central Universities.

■

REFERENCES

(1) Nomikos, P.; MacGregor, J. F. Monitoring of batch processes using multi-way principal component analysis. AIChE J. 1994, 40, 1361. (2) Nomikos, P.; MacGregor, J. F. Multiway partial least squares in monitoring batch processes. Chemom. Intell. Lab. Syst. 1995, 30, 97. (3) Yoo, C. K.; Lee, D. S.; Vanrolleghem, P. A. Application of multiway ICA for on-line process monitoring of a sequencing batch reactor. Water Res. 2004, 38 (7), 1715−1732. (4) Ge, Z.; Gao, F.; Song, Z. Batch process monitoring based on support vector data description method. J. Process Control 2011, 21 (6), 949−959. (5) Ge, Z.; Song, Z. Bagging support vector data description model for batch process monitoring. J. Process Control 2013, 23 (8), 1090−1096. (6) Lee, J. M.; Yoo, C. K.; Lee, I. B. Enhanced process monitoring of fed-batch penicillin cultivation using time-varying and multivariate statistical analysis. J. Biotechnol. 2004, 110 (2), 119−136. (7) Rännar, S.; MacGregor, J. F.; Wold, S. Adaptive batch monitoring using hierarchical PCA. Chemom. Intell. Lab. Syst. 1998, 41 (1), 73−81. (8) Lee, D. S.; Park, J. M.; Vanrolleghem, P. A. Adaptive multiscale principal component analysis for on-line monitoring of a sequencing batch reactor. J. Biotechnol. 2005, 116 (2), 195−210. (9) Zhao, C.; Wang, F.; Jia, M. Dissimilarity analysis based batch process monitoring using moving windows. AIChE J. 2007, 53 (5), 1267−1277. (10) Yao, Y.; Gao, F. A survey on multistage/multiphase statistical modeling methods for batch processes. Annual Reviews in Control 2009, 33 (2), 172−183. (11) Ge, Z.; Song, Z.; Gao, F. Review of recent research on data-based process monitoring. Ind. Eng. Chem. Res. 2013, 52 (10), 3543−3562. (12) Ge, Z.; Song, Z. Distributed PCA model for plant-wide process monitoring. Ind. Eng. Chem. Res. 2013, 52 (5), 1947−1957. (13) Zhang, Y.; Zhou, H.; Qin, S. J.; et al. Decentralized fault diagnosis of large-scale processes using multiblock kernel partial least squares. Industrial Informatics, IEEE Transactions on 2010, 6 (1), 3−10. 9214


Hierarchical Support Vector Data Description for Batch Process

Recommend Documents