Double-Weighted Independent Component Analysis for Non

Article pubs.acs.org/IECR

Double-Weighted Independent Component Analysis for NonGaussian Chemical Process Monitoring Qingchao Jiang, Xuefeng Yan,* and Chudong Tong Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of Education, East China University of Science and Technology, Shanghai 200237, P. R. China S Supporting Information *

ABSTRACT: Considering that the deviation between normal and abnormal status captured by each independent component (IC) is different from each other, a statistical analysis-driven approach by integrating kernel density estimation (KDE) with weighted independent component analysis (KDE-WICA) is developed. In KDE-WICA, KDE is used to estimate the probability and evaluate the importance of each IC, and subsequently set different weighting values on the ICs to highlight the deviation information for process monitoring. To overcome drastic fluctuations in the monitoring result, given that the previous status is not considered in determining the current status, a statistical weighting strategy is proposed to comprehensively evaluate the status of the process within a moving window (KDE-DWICA) and further improve the monitoring performance. KDE-DWICA is exemplified using a numerical study and the Tennessee−Eastman benchmark process. The monitoring results indicate that the performance of KDE-DWICA is superior to those of PCA, ICA, and other state-of-the-art variant-based methods.

1. INTRODUCTION Real-time process monitoring has an important role in chemical plant safety and product quality. With the development of advanced computing technologies, real-time process monitoring has been widely used in various chemical industries; thus, it has gained much attention in academic research.1−3 To maintain the safety and stability of chemical processes, early detection of process upset and equipment malfunctions, as well as accurate identification of the factors causing such events, is very important.4,5 In the absence of prior process knowledge or rich operating experience, multivariate statistical methods have been demonstrated to be effective, and are widely used in various chemical processes.1,6−8 Generally, multivariate statistical process monitoring (MSPM) utilizes historical data within normal operating regions to extract low-dimensional representations of normal status and to achieve process monitoring according to the variations in the generated representations. MSPM consists of the following steps:1,9 first, a statistical model of the normal status is established based on a normal historical data set collected from the industrial process; second, the statistical model is applied to project the current process data status; third, the current status is determined as either normal or abnormal, according to the difference between the projected results of the normal and the current status; fourth, if the current status is abnormal, the fault source in the industrial process is identified. In MSPM, employing a suitable multivariate statistical method for the industrial process is important, primarily because different industrial processes have different data features.5,10−13 Principal component analysis (PCA) and partial least-squares (PLS) are two of the most widely used methods for said purpose.8,14−17 However, PCA/PLS-based techniques assume that the monitored variables have a Gaussian distribution, thereby failing to provide high-order representations for non-Gaussian data.6,9,18,19 To cover this deficiency, © 2013 American Chemical Society

independent component analysis (ICA) has been proposed and applied for non-Gaussian process monitoring.9,10,18,20−25 ICA decomposes the observed data into linear combinations of independent components (ICs), which reveal more useful information than principal components (PCs)9,18 because they contain higher-order statistics. ICA has been extensively studied, and many successful applications have been reported in biomedical signal processing and chemical process monitoring, among others.12,19,26,27 Li and Wang used ICA to remove the dependencies of variables and reduce data dimension from dynamic signals.22 Kano et al. proposed an ICA-based MSPM method and demonstrated the superiority of the proposed procedure over conventional PCA-based methods.20,28 Lee et al. extracted independent components using ICA and separated them into dominant ICs, excluding the ICs for process monitoring.18 In addition, ICA has been extended to dynamic ICA (DICA), kernel ICA (KICA), and multiway ICA (MICA), which are used to monitor dynamic, nonlinear, and batch processes, respectively; several other improvements have also been discussed.11,21,23,29 Ge and Song proposed a monitoring method based on the principle of independent component analysis-principal component analysis (ICA-PCA) to extract both Gaussian and non-Gaussian information for fault detection and diagnosis.5 Rashid and Yu integrated multidimensional mutual information (MMI) with ICA to evaluate the dependency between the ICs of the monitored and normal benchmark data sets for monitoring performance improvement.12,27 More recently, Ge and Song introduced a performance-driven ensemble learning ICA to improve process monitoring performance.30 In the ICA-based Received: Revised: Accepted: Published: 14396

March 27, 2013 September 4, 2013 September 14, 2013 September 15, 2013 dx.doi.org/10.1021/ie4009666 | Ind. Eng. Chem. Res. 2013, 52, 14396−14405

Industrial & Engineering Chemistry Research

Article

plots.4,18,35 Contribution plots are easy to generate, without having to consider prior process knowledge; they can also show the contribution of each process variable to the observed statistics.4,18 For the proposed method, similar contribution plots are presented for fault diagnosis. The remainder of the article is organized as follows. The next section briefly reviews the conventional ICA-based process monitoring technique, as well as the use of KDE for probability density estimation. The details of the development of KDEDWICA-based process monitoring method are then presented. Next, the performance of the KDE-DWICA method is illustrated using both the numerical process and the Tennessee−Eastman (TE) chemical process. A comparison between regular and variant PCA and ICA methods is also provided. Conclusions are presented at the end of the article.

monitoring scheme, kernel density estimation (KDE) is usually used to determine the control limit of the monitored statistics because the statistics do not follow Gaussian distribution.9,18,21 Furthermore, the contribution plots for fault diagnosis derived from ICA are more powerful than the plots from PCA or PCA variant-based methods.18 Although MSPM has been relatively successful, the previously mentioned MSPM methods may not always function well. Several problems still need to be addressed. In general, the extracted lower-dimensional representation of normal status has a critical role in subsequent fault detection and diagnosis. Without loss of generality, the deviation between the normal status and the abnormal samples captured by each latent variable is different from each other, and the degree of deviation of each latent variable is not determined by the sequential position. In some cases, there exists a situation in which the predetermined latent variables at the first several sequential positions, but not within the first significant deviation degrees, can hide useful information embedded in the current sample data. Therefore, the weighting strategy can be adopted to increase the fault sensitivity of the traditional MSPM methods. Several studies concerning the weighting strategy used in MSPM methods have been reported. Wold introduced a dynamically updated model using exponential weighting observations, and recent observations were weighted more heavily than the earlier ones.31 He et al. proposed a variableweighted kernel Fisher discriminant analysis method that highlighted the fault information by giving more weight to related variables.32 Ferreira et al. also suggested a sample-wise weighted PCA for multicampaign process monitoring.33 These methods directly operate on measured variables and do not further analyze the importance of different latent variables. In our previous work, the instances wherein useful information is being hidden are discussed, and weighted principal component analysis (WPCA) is proposed to solve the problem.34 WPCA evaluates the importance of each principal component according to the variation rate of T2 statistics and sets different weighting values on the components to highlight useful information for online monitoring. The WPCA-based method improved the PCA-based monitoring performance in both fault detection and identification aspects. However, the inherent Gaussian distribution assumption limits the application of the said method, and the monitoring results may fluctuate unpredictably because the effects of the previous status on the current status are not being considered. In this study, a double-weighted ICA algorithm is proposed to improve ICA based on process monitoring performance. First, KDE is used to estimate the probability of each IC for the samples measured online. These probabilities are then used to evaluate the importance of the corresponding IC, that is, to determine the ICs that can represent significant deviations between normal and current status. Second, the weight of each IC is obtained based on automatic and adaptive probability values. Finally, to further ensure better monitoring performance, a moving window technique is introduced to consider the effect of the previous status on the current status to weight the statistics smoothly. Given that the proposed method is composed of two different weighted strategies, it is called KDE-based double-weighted ICA (KDE-DWICA). When a fault has been detected, fault diagnosis is the next issue for process monitoring because finding the underlying cause of the fault is necessary. Currently, the most widely used approach in PCA and ICA models comprises contribution

2. PRELIMINARIES In this section, the ICA-based process monitoring technique is briefly reviewed, as well as the use of KDE for probability density estimation. 2.1. Independent Component Analysis. ICA is a multivariate statistical technique used to reveal hidden factors underlying sets of variables or signals.18,26 We assume that l measured variables at sample k, which is denoted by x(k) = [x1(k), x2(k), ..., xl(k)]T. In the ICA algorithm, we assume that there are r unknown independent components, that is, s(k) = [s1(k), s2(k), ..., sr(k)]T (r ≤ l), which can represent x(k) through linear combinations as follows:18 x(k) = As(k) + e(k)

(1)

where A = [a1, a2, ..., ar] ∈ Rl×r is the mixing matrix and e(k) is the residual. The mixing matrix A and the independent components s are usually estimated by the FastICA algorithm from the sample data x.18,26 To eliminate the cross-correlation between the measurement variables, whitening is the initial step in the FastICA algorithm, and eigenvalue decomposition is one of the popular methods used.19,26 Thereafter, the problem of finding a full-rank matrix A is simplified to be the problem of finding an orthogonal matrix B as follows:18,19 s(̂ k) = BTQx(k)

(2)

where Q = Λ−1/2UT, and U and Λ can be obtained from the eigenvalue decomposition of Rx = E{x(k)x(k)T} = UΛUT. The relationship between the demixing matrix W and B is then given by W = BTQ

(3)

An optimal number of ICs are selected to capture the dominant characteristics of the process based on the assumption that the rows of W with the largest norm have the largest effect on the variation of S.9,18,26 For process monitoring purposes, two statistical parameters known as I2 and SPE were constructed to monitor the independent component subspace (ICS) and the residual subspace, respectively.9,18 The statistical parameters I2 and SPE are defined as follows:9,18 I 2(k) = s(̂ k)T s(̂ k)

(4)

and

SPE(k) = e(k)T e(k) 14397

(5)

dx.doi.org/10.1021/ie4009666 | Ind. Eng. Chem. Res. 2013, 52, 14396−14405


Article

considered, namely, the larger α and the smaller β, as given by the following function:

where ŝ(k) = Wx(k) and e(k) = x(k) − x̂(k). The predicted value x̂(k) is calculated as follows:9,18 x̂(k) = QBTWx(k)

⎧ ⎪1/ α , wiK (k) = ⎨ ⎪ ⎩1/β ,

(6) 2

The control limits for I and SPE can be computed through a kernel density estimation,18,21 which will be presented in the following section. In this study, SPE statistics is not discussed; the main focus of the research is improving the usage of the I2 statistic. 2.2. Kernel Density Estimation. The kernel density estimation method was introduced by Rosenblatt36 and Parzen.37 The general idea of KDE is to compute the density of the statistical sample in a distribution using a presumed normal distribution. A univariate kernel estimator with kernel function K is defined by37,38 f ̂ (x ) =

1 nh

n

∑K i=1

{ x −h x }

1 1 n h 2π

i

n

⎛ (x − x )2 ⎞ i ⎟ 2 2h ⎝ ⎠

(7)

s W (k) = W K (k)s(̂ k) K

(9)

[diag(wK1 (k),

(10)

wK2 (k),

wKm(k))]1/2

where W (k) = ..., is a diagonal matrix. After the weighting procedures, the I2 statistics becomes I 2(k) = s(̂ k)T W K (k)2 s(̂ k)

(11)

To overcome drastic fluctuations in the monitoring result and to improve further the performance of the I2 statistic, a statistics weighting strategy is proposed. I 2 is defined as the comprehensive evaluation of a series of data within a moving window, which is described as follows:

∑ exp⎜− i=1

if p ̂(sî (k − 1)) ≤ β

where p̂(ŝi(k − 1)) is the probability of the ith IC on the (k− 1)th sample point (k refers to the current sample time), which can be estimated online using KDE. α, β are probability thresholds which are empirically determined in the study, where α is suggested to be (0.8−1) and β is advised to be (0.01−0.2). In practical applications, the values of α, β could be determined from offline modeling based on the normal testing data. One major principle is that the weighting strategy should not damage the testing of normal process data. That is, the false alarm rates should not be increased by introducing the weighting. It is suggested to find a proper empirical value for each specific case with the sufficient data in normal conditions. The weighted independent components then become

where x is the data point under consideration, xi is an observation value from the data set, h is the window width (also known as the smoothing parameter), and n is the number of observations.18,37 The kernel function K determines the shape of the bumps, and a number of possible kernel functions are present.38 In practice, the kernel function form is not very important, with the Gaussian kernel function being the most widely used.38 In this study, the Gaussian kernel is employed and the kernel estimator becomes38 f ̂ (x ) =

if p ̂(sî (k − 1)) > β

(8)

In KDE, the window width h usually has a crucial influence on the performance of the density estimator. The optimal choice of h depends on several factors, such as the number of data points, the data distribution and the choice of kernel function. Extensive studies on the choice of h have been reported, and finding a proper empirical value for each specific case has been suggested.38−40 Given that the training data set for normal status is easy to obtain with sufficient information, a satisfactory performance of KDE can be normally guaranteed.

2 ⎧ 2 I (k), if I ̂ (k − 1) ≤ CL ⎪ ⎪ 2 2 2 2 I ̂ (k ) = ⎨ 2 ̂ ̂ ̂ ⎪(I (k), I (k − 1), ..., I (k 2− n + 2), I ⎪ (k − n + 1))W s , if I ̂ (k − 1) > CL ⎩

(12) S

[wS1,

wS2,

wSn]T

where W = ..., is the weighting matrix functioning on the time series statistics within a moving window, n is the number of points in the moving window, and CL is the control limit obtained through KDE using a normal process data. The elements wSi in WS are defined as follows:

3. KDE-DWICA FOR PROCESS MONITORING In this section, the KDE-based weighted independent component analysis and the statistics weighting strategy is proposed. The KDE-DWICA for process monitoring is discussed in detail. 3.1. KDE-DWICA for Fault Detection. The probability density score of each independent component on the training data set can be estimated based on normal benchmark data using KDE. When utilized for online monitoring, the current sample point is initially projected onto the independent component subspace. The probability of the corresponding ICs can be evaluated subsequently based on the obtained probability density function for each independent component. Because different ICs provide different reflections of the deviation from the reference ICs, defining a real-time and dynamic weighting matrix WK is possible for each of the independent component scores on the currently monitored status to emphasize the important ICs that represent the main deviation between the normal and the current status. In this work, the probability function of each independent component is determined according to KDE. Only two values have been

wiS

i ⎧ (i = 1, ..., n − 2) ⎪1/2 ⎨ =⎪ i−1 (i = n − 1) ⎩1/2

∑in= 1

(13)

wSi

where n is the window width; thus, = 1. (The window width is suggested to be 4−8 because when n is larger, the weights are much smaller than the more recent points and would not significantly influence the statistics results.) The points in the window would be weighted discriminately, and the more recent samples would be weighted heavily. In this manner, the current process information would be highly addressed. The recent process history information would also be given proper consideration. The control limit of the weighted I2 statistics is determined through the kernel density estimation method. The univariate kernel density estimator is used to estimate the density function of the normal I2 values using normal data. The value occupying 99% of the density function area can be obtained, which is then assigned as the control limit of the normal operating data. For a 14398



Article

Figure 1. Illustration of the KDE-DWICA strategy for process monitoring.

contribution plot procedure in KDE-DWICA is summarized as follows: (1) Select the d components that are out-of-control. (2) Calculate the contribution of each variable xj to the outof control components conti,j = siwi,j(xj), where wi,j is the (i,j)th element of the loading matrix W. After KDE weighting, the contribution becomes

more detailed description of KDE, refer to the works of Parzen37 and Lee et al.18 The proposed double-weighting strategy is shown in Figure 1. The first weighting is carried out to emphasize certain ICs that capture the dominant variations of the current samples relative to the normal status. The second weighting is carried out to smoothly adjust the weighted I2 statistics for further improving the monitoring performance. The total monitoring procedure consists of offline modeling and online monitoring systems, the detailed steps of which are listed as follows. (1) Offline modeling procedure: Step 1: Collect two normalized data sets XA and XB under normal status from the industrial process. XA is employed for ICA modeling, whereas XB is used for online control limit determination; Step 2: Perform the ICA algorithm to obtain matrices W and B, as well as to project XB onto the ICS; Step 3: Use KDE to estimate the probability density function of each independent component based on the IC scores of XA; Step 4: Estimate the probability of the IC scores of each sample using the obtained probability density functions from Step 3, and then calculate the weighted I2 statistics of each sample in XB; Step 5: Use the KDE to decide the 99% or 95% confidence limit based on the weighted IC scores of XB. (2) Online monitoring procedure: Step 1: At the current time k, collect the current measurement data x(k) from the industrial process, and then normalize the current sampled data x(k) as described in the modeling steps; Step 2: Calculate the ICs of x(k); Step 3: Estimate the probability of each IC and determine the real-time and dynamic weighting matrix WK(k); Step 4: Calculate the I2(k) value for the current sample data, and then calculate the weighted statistics I2̂ (k) in the moving window; Step 5: If the I2̂ (k) statistics exceeds the confidence limit, a fault has been detected and fault diagnostic tools are used to analyze the root cause. Otherwise, return to Step 1 and continue monitoring. 3.2. Contribution Plots in KDE-DWICA for Fault Identification. When a fault has been detected, the next step is to identify the variables responsible for the out-ofcontrol status. Contribution plot method is usually used for fault identification in ICA-based process monitoring.18,35 The

wcont i , j = wiK siwi , j(xj) = wiK cont i , j

(14)

(3) Calculate the total weighted contribution of the jth process variable xj d

W CONTj = wiK ∑ (cont i , j)

(15)

i=1

(4) Calculate the weighted statistics contribution in a moving window with n samples, that is, W CONT( j k ) = [W CONT( j k ), W CONT( j k − 1), ... S , W CONT( j k − n + 1)]W (k) d

d

= [∑ (wcont i , j(k)), i=1

∑ (wconti ,j(k − 1)) i=1

d

, ...,

∑ (wconti ,j(k − n + 1))]W S(k) i=1

k

=

∑ a=k−n+1 k

=

∑ a=k−n+1

d

[∑ (wcont i , j(a))]wiS(a) i=1 d

[∑ (cont i , j(a))wiK (a)wiS(a)] i=1

(16)

In KDE-DWICA-based contribution plots, the important information is highlighted, providing a more significant guide for identifying the responsible variables. Given that the described method is also a comprehensive evaluation of the whole information within a time period, the results are more reliable for practical use.

4. ILLUSTRATIONS AND DISCUSSIONS In this section, the proposed KDE-DWICA method is tested using a numerical process and the well-known TE benchmark process.41 Comparisons with conventional methods, including PCA,4 ICA,18 KICA,11 ICA-PCA,5 Linear Gaussian State-Space (LGSS),42 and WPCA,34 for the TE benchmark process monitoring are also presented. 14399



Article

Figure 2. I2 statistics monitoring results of faults 1 and 2. (a) ICA for fault 1, (b) KDE-WICA for fault 1, (c) KDE-DWICA for fault 1, (d) ICA for fault 2, (e) KDE-WICA for fault 2, and (f) KDE-DWICA for fault 2.

Figure 3. Estimated probability of each IC: (a) fault 1 and (b) fault 2.

where y is a random input vector and each element is uniformly distributed over the interval (−2,2). g is the output equal to t plus a random noise vector v. Each element of v has zero mean and a variance of 0.1. The input u and output g are measured, but t and y are not. The data vector for analysis consists of x(i) = [gT(i)uT(i)]T, and the normal data with 200 samples are used for analysis. A total of five variables (g1, g2, g3, u1, u2) are scaled to zero mean and unit variance to eliminate the effects of the magnitudes of each variable.18,21 The tested fault types are listed as follows: Fault 1: A step change in y1 by 2.5 (lower by 0.5 compared with values in the references18,21) is introduced from sample 50. Fault 2: y2 is linearly increased by adding 0.05(i − 50) to the y2 value from sample 50, where i is the sample number.18,21 The I2 monitoring results of ICA, KDE-WICA, and KDEDWICA of the two faults are presented in Figure 2. Figure 2 panels a−c are the monitoring results of fault 1, whereas Figures 2 panels d−f are the monitoring results of fault 2. The comparisons of Figure 2 panels a, b, and c show that KDE-

4.1. Case Study on a Numerical Process. We consider the simple multivariate process suggested by Ku et al.43 and modified by Lee et al.18 ⎡ 0.018 −0.191 0.287 ⎤ ⎢ ⎥ t(i) = ⎢ 0.847 0.264 0.943 ⎥t(i − 1) ⎢⎣−0.333 0.514 − 0.217 ⎥⎦ ⎡1 2 ⎤ ⎢ ⎥ + ⎢ 3 −4 ⎥u(i − 1) ⎣− 2 1 ⎦ g (i ) = t (i ) + v (i ) ⎡ 0.193 0.689 ⎤ ⎡ 0.811 −0.226 ⎤ u(i) = ⎢ ⎥y ⎥u(i − 1) + ⎢ ⎣ 0.477 0.415 ⎦ ⎣−0.320 − 0.749 ⎦ (i − 1) 14400



Article

Figure 4. Contribution plots during the monitoring of faults 1 and 2. (a) ICA for fault 1, (b) KDE-WICA for fault 1, (c) KDE-DWICA for fault 1, (d) ICA for fault 2, (e) KDE-WICA for fault 2, and (f) KDE-DWICA for fault 2.

Figure 5. Monitoring results of fault 0 in the TE process: (a) ICA, (b) KDE-WICA, and (c) KDE-DWICA.

WICA performs much better than the conventional ICA method. However, the monitoring performance is further improved using the KDE-DWICA method. Monitoring fault 2 presents similar results. The best performance is observed using KDE-DWICA. To provide a visual analysis of the necessity for a weighting strategy, the estimated probability of the independent component scores are presented in Figure 3. Figure 3a shows that when fault 1 occurs, the probability of the second IC (IC 2) changes drastically, indicating that the information with the most deviation is captured by IC 2. Thus, IC 2 should be weighted with a higher value. For fault 2, the probability of all the three ICs changed drastically, and IC 1 and IC 3 were found to be outstanding (Figure 3b). The two aforementioned ICs evidently capture the dominant variations on the abnormal data; therefore, they should be given more weight. The contribution plots of ICA, KDE-WICA, and KDEDWICA for fault identification are illustrated in Figure 4. Figure 4 panels a−c are the contribution plots for fault 1, whereas Figures 4 panels d−f are the contribution plots for fault

2. The comparisons of Figures 4 panels a, b, and c show that all three plots can identify the corresponding variables accurately. 4.2. Application in TE Process. The TE process is a benchmark problem in process engineering developed by Downs and Vogel;41 it is widely used for process monitoring scenario testing. The base control scheme for the TE process is shown in Supporting Information Figure 1, and the simulation code for the open loop can be downloaded from http:// brahms.scs.uiuc.edu. The second plant-wide control structure described in Lyman and Georgakis’s work44 is implemented to simulate realistic conditions. The process has 22 continuous measurements, 19 compositions, and 12 manipulated variables, as listed in Supporting Information Table 1.4,41,44 The simulator can generate 21 types of different faults, as tabulated in Supporting Information Table 2.4,41,44 All the process measurements include the Gaussian noise; when a fault enters the process, it affects almost all state variables.4,41,44,45 In this work, 22 continuous process measurements, 19 compositions, and 11 manipulated variables (the agitation speed is not included because it was not manipulated) are 14401



Article

Figure 6. The probability along each IC during fault 4 monitoring.



shown in Figures 5 panels a−c, with false alarm rates of 0.046, 0.036, and 0.045, respectively. From a practical engineering aspect, these low false alarm rates can be neglected. Thus, the monitoring performance of the normal process was not degraded either by the introduction of the KDE weighting method or by the use of statistics weighting strategy. 4.2.2. Case Study on Fault 4. When fault 4 occurs, temperature in the reactor would increase, which would be compensated by the control loops.4,44 The other 50 measurement and manipulated variables will remain steady after the fault occurs. The mean and standard deviation of each variable

considered, with an overall total of 52 variables. The simulation data consist of 960 observations for each tested data set, and faults are introduced into the process on the 160th sample. The 99% confidence limits of all the statistics are determined by KDE. The KDE weighting threshold β is set to 0.01 and α to 0.9, whereas the moving window for statistics weighting is set to 6. 4.2.1. Case Study on Fault 0. Fault 0 represents the normal operating condition in the TE process, which is used for testing false alarm rates.4,44 The monitoring performances of conventional ICA, KDE-WICA, and KDE-DWICA for fault 0 are 14402



Article



21 using ICA, KDE-WICA, and KDE-DWICA is shown in Figure 10. Figures 8−10 show that monitoring performances using KDE-WICA and KDE-DWICA are improved significantly. The best performance is obtained using KDE-DWICA, with significantly reduced missed detections and detection delays. The monitoring results of PCA,4 conventional ICA, KICA by Lee et al.,21 PCA-ICA by Ge and Song,5 LGSS by Wen et al.,42 WPCA by Jiang and Yan,34 KDE-WICA, and KDE-DWICA for all the faults in the TE process are listed in Supporting Information Table 3. For each statistic, the missed detection rates for all 21 faults are calculated, and the minimum missed detection rates achieved for each fault are written in bold numbers. Faults 3, 9, and 15 are not used for comparison because the missed detection rates in all the methods are very high for these faults and no observed change in the mean or variance is detected by any of the methods used. As shown in Supporting Information Table 3, KDE-WICA and KDEDWICA perform better than PCA-based, ICA-based, and LGSS methods for most faults, and the missed detection rates are reduced significantly. The performance of KDE-DWICA is better than that of KDE-WICA, with the lowest missed detection rates in most cases. There are also many other new ICA-based methods reported in the past few years, such as the dynamic ICA by Lee et al.,21 the modified ICA by Lee et al.,9 the improved dynamic ICA by Hsu et al.,24 and the ensemble learning ICA by Ge and Song,30 etc. These methods addressed different aspects of the ICA and significantly improved the ICA based non-Gaussian process monitoring performance. The improvements are also illustrated with the TE benchmark process and the monitoring results are provided in the related articles. The results are cited and listed in the Appendix Table 4, but it is not suggested to make a direct comparison, because these methods are based on different assumptions and performed under different conditions.

differ by less than 2% between fault 4 and the normal condition, which makes fault detection and diagnosis rather challenging. The estimated probabilities of the ICs when monitoring fault 4 are presented in Figure 6. The probabilities of IC 4, IC 12, and IC 14 change significantly after the fault occurs, indicating that the scores shift from normal to abnormal conditions with higher probabilities. In KDE-WICA monitoring, the information along these ICs are highlighted. The monitoring results of fault 4 based on ICA, KDE-WICA, and KDE-DWICA are shown in Figure 7. The graphs in Figure 7 show that all of the methods can trigger the fault alarm in a timely manner. However, a number of abnormal points are retained under the control limit in the ICA-based method, leading to a high-missed detection rate. In the KDE-WICA-based method, the missed detection rate is reduced significantly; however, some points are still missed. In the KDE-DWICA method, the detection performance is further improved and all the abnormal points are detected. The KDEWICA and KDE-DWICA methods significantly improve the fault detection performance. 4.2.3. Case Study on Faults 10, 11, and 21. Fault 10 is a random change in the temperature of stream 4 (C feed) in the TE process, which is widely used for algorithm testing.4,44 The monitoring results of this fault by all the three methods are shown in Figure 8. Both the KDE-WICA and KDE-DWICA methods clearly perform better than conventional ICA. The missed detection rates of KDE-DWICA are the lowest among the three methods. Fault 11 in the TE process induces a random variation in the reactor cooling water inlet temperature, as well as large oscillations in the reactor cooling water flow rate, resulting in a fluctuation in the reactor temperature.4,44 The other variables remain around the set points and behave similarly as in normal operating conditions. Performance monitoring of fault 11 using ICA, KDE-WICA, and KDEDWICA is shown in Figure 9. Fault 21 in the TE process is defined as the situation wherein the valve for stream 4 is fixed at the steady-state position. The monitoring performance of fault 14403



Article

Figure 11. Contribution plots during fault 4 monitoring at sample 161: (a) ICA, (b) KDE-WICA, and (c) KDE-DWICA.

Figure 12. Contribution plots during fault 4 monitoring at sample 162: (a) ICA, (b) KDE-WICA, and (c) KDE-DWICA.

4.2.4. Contribution Plots for Fault 4. The contribution plots of ICA, KDE-WICA, and KDE-DWICA for fault 4 are presented in Figures 11 and 12. Figure 11 shows the contribution plot for the I2 value at sample 161 (the first point when the fault occurs) and Figure 12 shows the plots at sample 162. On the basis of Figure 11, we can conclude that variable 9 (reactor temperature) and variable 51 (reactor cooling water flow) have the largest contributions to the I2 statistics, providing a significant guide for the nest operation. Fault 4 is caused by a step change in the inlet temperature. However, the change is compensated by the control loop. Therefore, in the contribution plots of sample 162, the influence of variable 9 is removed in the ICA- and KDEWICA-based methods. However, the contribution plots of KDE-DWICA represent a comprehensive evaluation over a time period. The influence of variable 9 is retained in these plots, thus identifying the corresponding variables more accurately.

noted that this work makes an improvement on the classical ICA method; however, this procedure can be extended to nonlinear, dynamic versions or integrated with other improvements.

■

ASSOCIATED CONTENT

S Supporting Information *

Tables 1−4 as mentioned in the text and the base control scheme for the Tennessee Eastman process. This material is available free of charge via the Internet at http://pubs.acs.org.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Address: East China University of Science and Technology, P.O. Box 293, MeiLong Road No. 130, Shanghai 200237, P. R. China. Notes

The authors declare no competing financial interest.

■

5. CONCLUSIONS This paper proposed a KDE-based double-weighted ICA to improve the monitoring performance in non-Gaussian processes. First, the KDE-DWICA uses KDE to calculate the probability of ICs for online sampled data. The importance of each IC is then evaluated. Second, different weighting values are given to corresponding ICs to highlight the useful information along the IC. Finally, the weighting strategy is used to weight the I2 statistics smoothly by introducing a moving window. Compared with conventional monitoring methods (i.e., PCA, ICA, KICA, ICA-PCA, LGSS, and WPCA), the KDE-DWICA determines the ICs that represent the main deviation between normal and current status, and performs an adjusted comprehensive evaluation of a constant period. As demonstrated in the case study section, the process monitoring performance is improved significantly using the aforementioned method. The double-weighted ICA method presents a basic idea for weighting ICs to increase fault sensitivity. It should be

ACKNOWLEDGMENTS The authors gratefully acknowledge the support from the following foundations: 973 Project of China (2013CB733600), National Natural Science Foundation of China (21176073), and the Fundamental Research Funds for the Central Universities.

■

REFERENCES

(1) Qin, S. J. Statistical process monitoring: Basics and beyond. J. Chemom. 2003, 17 ((8−9)), 480−502. (2) Venkatasubramanian, V.; Rengaswamy, R.; Kavuri, S. N.; Yin, K. A review of process fault detection and diagnosis: Part III: Process history based methods. Comput. Chem. Eng. 2003, 27 (3), 327−346. (3) Zhao, C.; Gao, F. Between-phase-based statistical analysis and modeling for transition monitoring in multiphase batch processes. AIChE J. 2012, 58 (9), 2682−2696. (4) Chiang, L. H.; Russell, E.; Braatz, R. D. Fault Detection and Diagnosis in Industrial Systems; Springer Verlag: Berlin, 2001. 14404



Article

(5) Ge, Z.; Song, Z. Process monitoring based on independent component analysis-principal component analysis (ICA-PCA) and similarity factors. Ind. Eng. Chem. Res. 2007, 46 (7), 2054−2063. (6) Kourti, T.; MacGregor, J. F. Multivariate SPC methods for process and product monitoring. J. Qual. Technol. 1996, 28 (4), 409− 428. (7) Liu, X.; Xie, L.; Kruger, U.; Littler, T.; Wang, S. Statistical-based monitoring of multivariate non-Gaussian systems. AIChE J. 2008, 54 (9), 2379−2391. (8) Nomikos, P.; MacGregor, J. F. Multivariate SPC charts for monitoring batch processes. Technometrics 1995, 37 (1), 41−59. (9) Lee, J. M.; Qin, S. J.; Lee, I. B. Fault detection and diagnosis based on modified independent component analysis. AIChE J. 2006, 52 (10), 3501−3514. (10) Ge, Z.; Kruger, U.; Lamont, L.; Xie, L.; Song, Z. Fault detection in non-Gaussian vibration systems using dynamic statistical-based approaches. Mech. Syst. Signal Process. 2010, 24 (8), 2972−2984. (11) Lee, J. M.; Qin, S. J.; Lee, I. B. Fault detection of non-linear processes using kernel independent component analysis. Can. J. Chem. Eng. 2007, 85 (4), 526−536. (12) Rashid, M.; Yu, J. Nonlinear and non-Gaussian dynamic batch process monitoring using a new multiway kernel independent component analysis and multidimensional mutual information based dissimilarity method. Ind. Eng. Chem. Res. 2012, 51 (33), 10910− 10920. (13) Yin, S.; Ding, S. X.; Haghani, A.; Hao, H.; Zhang, P. A comparison study of basic data-driven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J. Process Control 2012, 22 (9), 1567−1581. (14) Dunia, R.; Qin, S. J.; Edgar, T. F.; McAvoy, T. J. Identification of faulty sensors using principal component analysis. AIChE J. 2004, 42 (10), 2797−2812. (15) Qin, S. J. Recursive PLS algorithms for adaptive data modeling. Comput. Chem. Eng. 1998, 22 (4), 503−514. (16) Kruger, U.; Dimitriadis, G. Diagnosis of process faults in chemical systems using a local partial least squares approach. AIChE J. 2008, 54 (10), 2581−2596. (17) Muradore, R.; Fiorini, P. A PLS-based statistical approach for fault detection and isolation of robotic manipulators. Ind. Electron., IEEE Trans. 2012, 59 (8), 3167−3175. (18) Lee, J. M.; Yoo, C. K.; Lee, I. B. Statistical process monitoring with independent component analysis. J. Process Control 2004, 14 (5), 467−485. (19) Hyvrinen, A. Survey on independent component analysis. Neural Comput. Surveys 1999, 2 (4), 94−128. (20) Kano, M.; Tanaka, S.; Hasebe, S.; Hashimoto, I.; Ohno, H. Monitoring independent components for fault detection. AIChE J. 2004, 49 (4), 969−976. (21) Lee, J. M.; Yoo, C. K.; Lee, I. B. Statistical monitoring of dynamic processes based on dynamic independent component analysis. Chem. Eng. Sci. 2004, 59 (14), 2995−3006. (22) Li, R.; Wang, X. Dimension reduction of process dynamic trends using independent component analysis. Comput. Chem. Eng. 2002, 26 (3), 467−473. (23) Zhang, Y. Fault detection of non-Gaussian processes based on modified independent component analysis. Chem. Eng. Sci. 2010, 65 (16), 4630−4639. (24) Hsu, C.-C.; Chen, M.-C.; Chen, L.-S. A novel process monitoring approach with dynamic independent component analysis. Control Eng. Pract. 2010, 18 (3), 242−253. (25) Wang, L.; Shi, H. Multivariate statistical process monitoring using an improved independent component analysis. Chem. Eng. Res. Des. 2010, 88 (4), 403−414. (26) Hyvärinen, A.; Oja, E. Independent component analysis: Algorithms and applications. Neural Networks 2000, 13 (4), 411−430. (27) Rashid, M. M.; Yu, J. A new dissimilarity method integrating multidimensional mutual information and independent component analysis for non-Gaussian dynamic process monitoring. Chemom. Intell. Lab. Syst. 2012, 115, 44−58.

(28) Kano, M.; Hasebe, S.; Hashimoto, I.; Ohno, H. Evolution of multivariate statistical process control: Application of independent component analysis and external analysis. Comput. Chem. Eng. 2004, 28 (6), 1157−1166. (29) Yoo, C. K.; Lee, J. M.; Vanrolleghem, P. A.; Lee, I. B. On-line monitoring of batch processes using multiway independent component analysis. Chemom. Intell. Lab. Syst. 2004, 71 (2), 151−163. (30) Ge, Z.; Song, Z. Performance-driven ensemble learning ICA model for improved non-Gaussian process monitoring. Chemom. Intell. Lab. Syst. 2013, 123, 1−8. (31) Wold, S. Exponentially weighted moving principal components analysis and projections to latent structures. Chemom. Intell. Lab. Syst. 1994, 23 (1), 149−161. (32) He, X. B.; Yang, Y. P.; Yang, Y. H. Fault diagnosis based on variable-weighted kernel Fisher discriminant analysis. Chemom. Intell. Lab. Syst. 2008, 93 (1), 27−33. (33) Ferreira, D. L. S.; Kittiwachana, S.; Fido, L. A.; Thompson, D. R.; Escott, R. E. A.; Brereton, R. G. Multilevel simultaneous component analysis for fault detection in multicampaign process monitoring: application to on-line high performance liquid chromatography of a continuous process. Analyst 2009, 134 (8), 1571−1585. (34) Jiang, Q.; Yan, X. Chemical processes monitoring based on weighted principal component analysis and its application. Chemom. Intell. Lab. Syst. 2012, 119, 11−20. (35) Miller, P.; Swanson, R.; Heckler, C. E. Contribution plots: A missing link in multivariate quality control. Appl. Math. Comput. Sci. 1998, 8, 775−792. (36) Rosenblatt, M. Curve estimates. Ann. Math. Stat. 1971, 1815− 1842. (37) Parzen, E. On estimation of a probability density function and mode. Ann. Math. Stat. 1962, 33 (3), 1065−1076. (38) Webb, A. R.; Copsey, K. D.; Cawley, G. Statistical Pattern Recognition; Wiley: New York, 2011. (39) Silverman, B. W. Density Estimation for Statistics and Data Analysis; Chapman & Hall/CRC: London, 1986; Vol. 26. (40) Johnson, R. A. Applied Multivariate Statistical Analysis; PrenticeHall: NJ, 2007. (41) Downs, J. J.; Vogel, E. F. A plant-wide industrial process control problem. Comput. Chem. Eng. 1993, 17 (3), 245−255. (42) Wen, Q.; Ge, Z.; Song, Z. Data-based linear Gaussian statespace model for dynamic process monitoring. AIChE J. 2012, 58 (12), 3763−3776. (43) Ku, W.; Storer, R. H.; Georgakis, C. Disturbance detection and isolation by dynamic principal component analysis. Chemom. Intell. Lab. Syst. 1995, 30 (1), 179−196. (44) Lyman, P. R.; Georgakis, C. Plant-wide control of the Tennessee Eastman problem. Comput. Chem. Eng. 1995, 19 (3), 321−331. (45) McAvoy, T.; Ye, N. Base control for the Tennessee Eastman problem. Comput. Chem. Eng. 1994, 18 (5), 383−413.

14405


Double-Weighted Independent Component Analysis for Non

Recommend Documents