Angle-Based Multiblock Independent Component Analysis Method

Subscriber access provided by UNIV OF CALIFORNIA SAN DIEGO LIBRARIES

Article

Angle-based multi-block independent component analysis method with a new block dissimilarity statistic for non-Gaussian process monitoring Jian Huang, and Xuefeng Yan Ind. Eng. Chem. Res., Just Accepted Manuscript • DOI: 10.1021/acs.iecr.6b00093 • Publication Date (Web): 13 Apr 2016 Downloaded from http://pubs.acs.org on April 19, 2016

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

Industrial & Engineering Chemistry Research is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Industrial & Engineering Chemistry Research

Angle-based multi-block independent component analysis method with a new block dissimilarity statistic for non-Gaussian process monitoring Jian Huang, Xuefeng Yan* Key Laboratory of Advanced Control and Optimization for Chemical Processes of Ministry of Education, East China University of Science and Technology, Shanghai 200237, P. R. China Abstract In recent years, the multi-block method has attracted substantial attention. Conventional multi-block methods divide an entire dataset into several blocks, and the monitoring in each block is conducted separately. The multi-block method highlights local information but ignores the information among different blocks. In this paper, we propose an angle-based multi-block independent component analysis (MBICA) method and create a new block dissimilarity (BD) statistic to measure the changes between blocks. Hierarchical clustering is adopted to cluster variables with small angles into a block. ICA models are then built into each block. Support vector data description (SVDD) is introduced to yield a final monitoring decision. The changes of blocks are determined by the differences between the angles of the monitored data and the benchmark data, leading to BD statistics. The proposed MBICA-BD method is applied to the Tennessee Eastman process. The simulation results demonstrate the superiority of the MBICA-BD method. Keywords: multi-block independent component analysis, block dissimilarity, support vector data description, fault detection, angle

Corresponding Author: Prof. Xuefeng Yan Email: [email protected] Address: P.O. BOX 293, MeiLong Road NO. 130, Shanghai 200237, P. R. China Tel: 0086-21-64251036

ACS Paragon Plus Environment


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 2 of 23

1. Introduction With the increasing number of observations, process monitoring has gained great importance for detecting abnormal events to ensure safe and reliable process operation

1-4

. In modern chemical and

biological processes, the real-time storage and analysis of data benefit from the extensive application of computer science. The collected data represent the process state at each sampling time and can be utilized for monitoring. Multivariate statistic process monitoring (MSPM) has been adopted to handle the highly correlated data by projecting process data onto an uncorrelated subspace

5-7

.

Principal component analysis (PCA) and partial least squares (PLS) are the most commonly used MSPM methods for fault detection and diagnosis. Many modified MSPM methods have been proposed 8-11. In the PCA method, the thresholds of monitoring statistics are determined by assuming that the original data have a Gaussian distribution and linear correlation

8,11

. However, in real industrial

processes, the distributions of and relationships between variables are complicated. The assumptions of the PCA algorithm usually cannot be satisfied. To overcome the disadvantages associated with the distribution, independent component analysis (ICA) was developed and has been proven to be efficient for monitoring performance 12,13. Lee et al. analyzed the drawbacks of the original ICA and proposed a modified ICA algorithm

10

. Kernel ICA (KICA) was developed to extract independent

components capturing nonlinear relationships among process variables 14. Dynamic ICA (DICA) was proposed to solve non-Gaussian and dynamic problems in modern industries

15,16

. Ge and Song

proposed a PCA-ICA method to extract both Gaussian and non-Gaussian information 8. Because large-scale industrial processes are becoming increasingly complex, it is increasingly difficult to monitor entire processes with a single model because of the huge number of process variables. Recently, many scholars have proposed multi-block algorithms. The main idea of multi-block algorithms is to divide the whole model into several sub-models that utilize local information effectively and simultaneously to improve the interpretation of process analysis. A multi-block PLS strategy was proposed by MacGregor et al. to create monitoring models in blocks, and the proposed method can detect faults quickly and facilitate fault diagnosis 17. Westerhuis et al. compared different multi-block methods and recommended that variables be separated into sub-blocks and that conventional PCA and PLS be built into each sub-block 18. Qin et al. gave a more detailed analysis of the multi-block strategy

19

. Zhang et al. proposed a multi-block kernel PLS with the advantages of


Page 3 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


capturing the information between blocks, interpreting nonlinear relationships between variables, and enabling fault diagnosis 5. Ge and Song built a distributed PCA by constructing blocks through different directions of PCA principal components

20

. Jiang and Yan divided variables into blocks

based on the mutual information between variables and applied this strategy to industrial and multi-mode processes 21,22. The previous multi-block process monitoring methods mainly divide the whole dataset into several blocks and monitor each block separately. The local information in a block is enlarged, thereby facilitating superior monitoring performance. However, the latent information among blocks is ignored after the block-division operation. When the relationship between blocks changes, the multi-block methods cannot adapt to the change. To solve the aforementioned problem, we create a new statistic to monitor the changes in the relationships between blocks. An angle-based multi-block ICA and block-dissimilarity algorithm (MBICA-BD) is proposed. The angle between different variables shows a geometric orientation of variables, which reflects the relationship between the two variables. The geometrical interpretation of the angle between two p-component vectors is regarded as two lines in p-dimensional Euclidean space. The angle based method has been adopted to investigate the similarities of principal components and two datasets8,23-25. The angle based similarity has achieved many good results. But there is no report on the angle based similarity of variables in the benchmark data. If both variables are totally linearly correlated, the angle between the both variables is equal to zero. A small angle indicates that both variables are correlated. The angle can not only explore the correlation between the variables but also measure the distance between variables. The angles between variables are calculated first. Hierarchical clustering 26,27 is used to implement the block-division step based on the angles between the variables. Variables with small angles are put into the same block. The hierarchical clustering divides the benchmark data into blocks, and ICA models are then built into each block. Support vector data description (SVDD) 28,29 is introduced to provide a final monitoring decision, which attempts to build the smallest possible hypersphere containing benchmark data. The changes in blocks are observed in terms of the difference in the angles between the monitored data and the benchmark data, which is recorded using a new statistic called block dissimilarity (BD). The rest of this paper is organized as follows. Section 2 reviews the ICA and SVDD algorithms. The block-division step, MBICA monitoring, and BD monitoring of the proposed MBICA-BD are described in detail and the step-by-step MBICA-BD fault detection scheme is given in section 3. The



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 4 of 23

proposed method is applied to monitor the Tennessee Eastman (TE) process and is demonstrated to exhibit efficient monitoring performance in section 4. Section 5 concludes. 2. Preliminary ICA and SVDD monitoring schemes are reviewed in this section. 2.1 Independent component analysis An observation vector x ( k ) at sample k with m variables can be written as linear combinations of d independent components (ICs). The ICA model is expressed as d

x (k ) = ∑ ai si (k ) + e (k ) = As(k ) + e(k )

(1)

i =1

where A = [ a1 , a2 , L , am ] ∈ R m×d represents the mixing matrix, e ( k ) is the residual vector, and

s(k ) = [ s1 (k ), s2 (k ), ⋅⋅⋅, sd (k )]T is the independent component vector. The problem of ICA is then to evaluate the mixing matrix and ICs. The FastICA 13,30 algorithm provides a solution to estimate A ∧

and s ( k ) by introducing a demixing matrix W . The estimated IC s (k ) can be expressed as ∧

s(k ) = Wx (k )

(2)

The reconstructed ICs are statistically independent. In the FastICA algorithm, the initial operation is to eliminate the cross-correlation between variables, which is called the whitening step. The whitening transformation is expressed as (3)

z ( k ) = Qx ( k )

The whitening matrix Q is derived by the eigendecomposition of the covariance matrix −1/2 T C = U ΛU T , which is expressed as Q = Λ U . Equation (3) can also be written as

z ( k ) = Qx ( k ) = QAs ( k ) = Bs ( k )

(4) ∧

where matrix B can be proven to be orthogonal. The estimated IC s (k ) is obtained by ∧

s (k ) = BT z (k ) = BT Qx (k )

(5)

By comparing Eqs. (2) and (5), the relationship between W and B is clearly W = BT Q . The IC selection is based on the L2 norm method. The rows of W with a large sum of


Page 5 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


squares coefficient is considered to capture more information on benchmark data. The selected k rows of W are expressed as Wk . Bk can be given as Bk = (Wk Q −1 )T . The reconstructed IC vector snew is calculated by s new = W k x . I 2 and SPE monitoring statistics are generated as follows: T I 2 = snew snew

(6)

∧

∧

SPE = eT e = ( x − x )T ( x − x )

(7)

∧

where x = Q −1 BkW k x . 2.2 Support vector data description Generally, the main idea of SVDD is to find a hypersphere with minimum volume that is able to hold all the training data

28,29

. First, benchmark data are projected onto the feature space by

introducing a nonlinear transformation Φ : x → F . The nonlinear transformation is implemented by 2

using the Gaussian kernel function K ( x, y) = exp(− x − y / σ ) . The problem to find a minimum volume hypersphere then changes into the following optimization problem: n

min R 2 + C ∑ ξi R , a ,ξ

(8)

i =1

2

2

s.t. Φ(yi ) − a ≤ R + ξi where a and R are the center and radius of the hypersphere, respectively. C denotes the tradeoff between the volume of the hypersphere and the number of errors. ξi is the slack variable, which allows a possibility that some of the training samples can be wrongly classified. The dual form of the optimization problem can be obtained as: n

n

n

min ∑ K ( xi , x j ) − ∑∑ α iα j K ( xi , x j ) αi

i =1

i =1 j =1

(9)

n

s.t. 0 ≤ α i ≤ C , ∑ α i = 1 i =1

where αi is a Lagrange multiplier and K ( xi , x j ) = Φ( xi ), Φ( x j )

represents a kernel function

that describes the inner product in the feature space. The selected support vectors (SVs) by SVDD are the samples corresponding to 0 ≤ αi ≤ C . The radius R can be computed as:



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

n

n

n

R = 1 − 2∑ α i K ( xi , x0 ) + ∑∑ α iα j K ( xi , x j ) i =1

Page 6 of 23

(10)

i =1 j =1

where x0 is one of the SVs. The current sample is expressed as x . The distance d between sample x and the center a is derived by: n

n

n

d = 1 − 2∑ α i K ( xi , x ) + ∑∑ α iα j K ( xi , x j ) i =1

(11)

i =1 j =1

If d is greater than R , sample x is projected outside the hypersphere, and thus, x is a fault. 3. The angle-based multi-block independent component analysis and block dissimilarity monitoring scheme 3.1 Block division rules The original data are expressed as X = [ x1 , x2 , ⋅⋅⋅, xm ] ∈ R n×m , where m and n represent the numbers of variables and samples, respectively. The angle θ ij between any two variables xi and

x j is calculated as cos θij =

xiT x j

(12)

xi x j

The angles between variables are in the range between 0 and

π 2

. A small angle indicates that the

corresponding two variables are correlated and should be placed in the same block. If the angle is close to

π 2

, then the both vectors are orthogonal, which means they are weakly correlated or

independent. The angle between different variables shows a geometric orientation of variables. The angle can not only explore the correlation between the variables but also measure the distance between variables. After the angles between the variables are calculated, we can get the angle vector ϑi for each variable xi such that

ϑi = [θ1i ,θ 2i , L , θ mi ]T

(13)

The related variables should be clusters into a same block on the basis of the angles. Hierarchical


Page 7 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


clustering 26,27, as a fundamental cluster algorithm, is used to divide m vectors into b blocks such that the vectors in a given block are similar to one another and simultaneously dissimilar to the vectors in other blocks. The main idea of hierarchical clustering is first to put the most similar variables into a group, then the second similarity, and so on. The step-by-step clustering ensures the similar observations are clustered into a group. Simultaneously, we do not need to stipulate a cluster number preliminarily. For variables clustering in process monitoring, we want to put the most similar variables into a group. In this paper, single linkage is adopted 31. The basic idea of single linkage is that the distance between blocks is equal to the minimum Euclidean distance between angle vectors in both blocks. Suppose that any two blocks can be denoted as Ωi and Ω j . The distance between

Ωi and Ω j is written as D( Ωi , Ω j ) = min x∈Ωi , x '∈Ω j D(ϑx ,ϑx ' )

(14)

where D (ϑ x , ϑ x ' ) is the Euclidean distance between angle vectors ϑx and ϑ x ' for variables x and

x ' . The concrete steps of hierarchical clustering are explained as follows: Step 1: Calculate the angles between any two variables and generate the angle vector ϑi for each variable xi . Each vector xi (i = 1, 2,L , m) belongs to each block Ωi at the beginning of clustering. That is, there are m blocks Ω1, Ω2 ,L Ωm for vectors x1 , x2 , ⋅⋅⋅, xm . Calculate the Euclidean distance between any two angle vectors D( Ωi , Ω j ) = D(ϑi , ϑ j ) ; Step 2: Blocks Ω p and Ωq merge into a new block Ω pq when the following equation is satisfied:

D( Ω p , Ωq ) = min D( Ωi , Ω j ) (i ≠ j)

(15)

The number of blocks decreases by 1. Recalculate the angles between blocks using Eq.(14); Step 3: If the convergence condition is met, the iteration is closed. Otherwise, repeat step 2; Step 4: Data matrix X is divided into blocks by hierarchical clustering. 3.2 Multi-block independent component analysis process monitoring Hierarchical clustering separates original dataset X into b blocks, which is expressed as

X = [ X1 , X 2 , ⋅⋅⋅, X b ]


(16)


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 8 of 23

where X i (i = 1, 2,L , b) represents each block generated by hierarchical clustering. The ICA algorithm is used to monitor each block. The ICA model in each block Xi is conducted by

X iT = Ai Si + Ei

(17)

We can obtain Ii2 and SPEi in each block. An online sample vector x is first divided into b vectors according to the original variable division results by the following:

x = [ x1 , x2 , ⋅⋅⋅, xb ]

(18)

Independent components si in block i can be acquired by

si = Wi xi

(19)

where Wi is the demixing matrix in block i . The input data Y of SVDD are constituted by Ii2

and SPEi in each benchmark block,

which can be written as

Y = [I12 , I 22 ,L , Ib2 ,SPE1 ,SPE 2 , L ,SPE b ] where Ii2

(20)

and SPEi are scaled to zero mean and unit variance. A corresponding monitored vector

ytest can be acquired. The distance d between monitored data and the center is expressed as n

n

n

d = 1 − 2∑ α i K ( yi , ytest ) + ∑ ∑ α iα j K ( yi , y j ) i =1

(21)

i =1 j =1

3.3 Block-dissimilarity strategy Conventional multi-block methods monitor each block independently without considering the information between blocks. A new statistic is introduced to detect changes in the relationships between blocks. Given two benchmark block matrixes with n samples

X I ∈ Rn×m1 and

X J ∈ R n×m 2 where m1 and m2 are the variable numbers, respectively, the angle between vector

xi (i = 1, 2,L , m1) in X I and vector x j ( j = 1, 2,L , m 2) in X J is calculated by


Page 9 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


cos θijIJ =

xiT x j

(22)

xi x j

For two monitored sets X 'I ∈ R w×m1 and X 'J ∈ R w×m 2 with w samples, the angle between monitored vectors x 'i and x ' j is expressed as θ 'ijIJ . The angle change of benchmark vectors and monitored vectors is reflected by the following: ∆θijIJ = θ 'ijIJ − θijIJ

(23)

where | | indicates the absolute value. The dissimilarity between blocks I and J can be written as

BDIJ =

m1 m 2 1 ∆θ ijIJ ∑∑ m1× m2 i =1 j =1

(24)

For simplicity, mean value vectors x I and x J of X I and X J are used to calculate the angle between blocks I and J as T

cos θ

IJ

xI xJ

=

(25)

xI xJ The corresponding angle for monitored data is expressed as θ 'IJ . The dissimilarity between blocks I and J can be rewritten as

BDIJ = θ 'IJ − θ IJ

(26)

The BD for the entire dataset is calculated by b

BD = ∑

b

∑ BD

IJ

(27)

I =1 J = I +1

The new statistic BD can monitor the relationship changes among blocks. We can obtain the normal BD values by exploring the normal benchmark data. Usually, the control limit is decided on the assumption of Gaussian distribution. However, the BD values cannot satisfy a defined distribution (Gaussian or other distributions). The control limit of BD is estimated by kernel density estimation (KDE). KDE 30 is given by ∧

f (BD) =

BD-BDi 1 n K( ) ∑ nh i =1 h


(28)


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 10 of 23

where n is the number of observations and h is the bandwidth. In this study, K represents the Gaussian kernel function

K (u ) =

1 ( − 12 u 2 ) e 2π

(29)

3.4 MBICA-BD fault detection scheme The proposed MBICA-BD fault-detection scheme is illustrated in Fig. 1. The detailed steps of the MBICA method and BD are shown as follows. MBICA offline modelling: Step 1: obtain the benchmark data matrix X and scale X to zero mean and unit variance; Step 2: calculate the angles between any two variables and generate the angle vector for each variable; Step 3: divide the original variables into b blocks by hierarchical clustering; Step 4: build a benchmark ICA model in each block and acquire I 2 and SPE statistics; and Step 5: take both statistics in each block as the input of SVDD and calculate the radius R of the benchmark data. MBICA online fault detection: Step 1: collect and divide monitored sample x into b parts based on the division result in offline step 3; Step 2: obtain I 2 and SPE statistics of the test data in each ICA model; Step 3: calculate the distance d according to both test I 2 and SPE statistics; and Step 4: if d is greater than R , the sample is a fault. The BD scheme is shown in Fig. 2, and the detailed steps are given as follows: Step 1: set the initial iteration i = 1 , and specify a moving window width w ; Step 2: collect the benchmark data X ∈ R n×m ; (i ) ∈ R w×m ; Step 3: obtain the moving window-based subset of the test data X test (i ) Step 4: divide X and X test into b blocks, respectively;

Step 5: calculate the difference of the angles between any two blocks, and generate the BD statistic; Step 6: increment i = i + 1 , and repeat step 3 until the current sample is the last test sample; and


Page 11 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Step 7: estimate the threshold of the BD statistic using the KDE method. 4. Case study 4.1 Introduction of the TE process The TE benchmark process is a widely used simulation experiment created by Downs and Vogel

32

and structured by Lyman and Georgakis

33

. The Supporting Information provides the

detailed description of the TE process and the control chart in Fig. S1, the variables in Table S1, and 21 simulation faults in Table S2. The benchmark dataset comprises 33 variables with 960 samples, and 21 different faults are generated from sample 161 to 960. The PCA, ICA, KICA, SVDD, and MBICA-BD algorithms are used to monitor the TE process, and the monitoring results are compared. The kernel parameter σ in both SVDD and MBICA-BD are set as the square of input dimension

m. 4.2 Block division of the TE process The block division of TE benchmark data is described as follows. First, an angle-based matrix is generated by calculating the angles between any two variables. Hierarchical clustering is then applied to cluster variables with small Euclidean distance of the angle vectors into a block. The cluster results are shown in Fig. 3. We have designed an index for choosing this convergence condition. The index is on the basis of the difference between the distances of the current and last clusters. The cluster number of current clustering is expressed as a . The minimal distance between current clusters can be calculated as Da = min D( Ωi , Ω j ) (i ≠ j i, j = 1, 2,L , a )

(30)

where D ( Ωi , Ω j ) is the distance between clusters i and j . We can easily know that the cluster number of last clustering is a + 1 . Then the minimal distance between last clusters can be denoted as Da +1 . If the difference between Da and Da +1 is sufficiently large, the clusters are clearly divided. The difference can be calculated as ∆Da = Da − Da +1 . If the number is too large, the global information may be ignored, and the computational complexity will increase, resulting in poor online-monitoring performance. The cluster number is varied from 2 to 10. For each number of clusters, ∆D is given in Fig.4. It can be seen that the ∆D in 4 clusters is the maximum value. The original variables are divided into 4 clusters. The detailed variable numbers in each block are given



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

in Table 1. 4.3 TE process monitoring BD and MBICA are each applied in TE. The width of the moving window is determined based on the benchmark data. If the width of moving window is too small, the angles are easily influenced by the noise. We take the benchmark data as the test data. Given a false alarm limit 3%, if the false alarm rate is lower than the limit, the width w is proper. In the BD part, the moving window w is chosen as 75. The fault detection rates of PCA, ICA, KICA, SVDD, and MBICA-BD are presented in Table 2. The value in boldface type for one fault indicates the highest fault detection rate. The proposed MBICA-BD performs similar fault detection rates to the other four methods when encountering faults 1, 2, 4, 5, 6, 7, 8, 12, 13, 14, 16, 18, and 19. Faults 3, 9, and 15 are small faults, and the fact that they are barely detected is acknowledged 34,35. These faults are quite small and have almost no effect on the overall process. The detection rates of faults 10, 11, 17, 20, and 21 are much higher than those of PCA, ICA, KICA, and SVDD. For fault 10, MBICA achieves efficient monitoring performance. The detection rate of MBICA in fault 10 are 0.88, whereas those of ICA and SVDD are less than 0.8. For fault 11, the MBICA statistic executes the best detection ability and detects 82% of the fault points. However, the detection rates of the other methods are only approximately 0.7. For fault 20, both MBICA and BD can detect faults efficiently with a high detection rates of 0.88 and 0.92. The both MBICA and BD detection rates of fault 21 reaches 0.88 and 0.59, but the other methods can detect only half of the fault samples. The angle-based MBICA-BD method has a reasonable division of original variables, which allows MBICA to detect more fault points. The new statistic BD evaluates the variations between blocks, which is a useful supplement to and extension of MBICA. The further analyses of faults 10, 11, 20, and 21 are given as follows. Fault 10 is a random temperature change of stream 4, which mainly affects the stripper pressure. The monitoring chart of ICA is shown in Fig. 5(a); SVDD in Fig. 5(b); BD in Fig. 5(c); and MBICA in Fig. 5(d). ICA and SVDD can detect only some of the fault samples with detection rates of 0.75, 0.76, and 0.62. The MBICA chart can detect more fault samples than others. Fault 11 is a random variation in the reactor cooling water inlet temperature. The large oscillations are spread by the control loop, resulting in a fluctuation in the reactor temperature. Fig. 6 shows the monitoring charts of fault 11. The MBICA chart shows better results than those of ICA and SVDD. MBICA divides the


Page 12 of 23

Page 13 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


whole model into several sub-models that utilize local information effectively and simultaneously to improve the interpretation of process analysis. Fault 20 is an unknown fault. Fig. 7 presents the monitoring charts of fault 20. The ICA algorithm can detect fault 20, but the fault-detection rates are only 0.78 and 0.74. SVDD also does not perform very well. The monitoring chart of the proposed MBICA shows a superior detection rate of 0.92. MBICA improves the monitoring results substantially. The new statistic BD evaluates the variations between blocks and the detection rate reaches 0.83. Both MBICA and BD have high fault-detection rates for fault 20. Fault 21 is a valve position constant fault of stream 4. At the beginning of this fault, the influence on the samples is inconspicuous. However, as time progresses, this fault will become obvious. To judge an algorithm’s efficiency is to determine whether it can detect this fault earlier in the process. The monitoring results of fault 21 are given in Fig. 8. It is obvious that BD detects the fault from the 230th point, whereas the others recognize the fault after the 600th point. The BD statistic detects the majority of fault samples and is more efficient than the other tested methods. The proposed BD statistic is therefore more sensitive to this slow-varying fault. The above faults are complex faults, which affects several variables in different blocks. The statistic BD can efficiently test the changes between blocks. 5. Conclusion In this study, a new angle-based MBICA and BD process monitoring strategy is proposed. The new method divides original data into several blocks based on the angles between the variables. Simultaneously, it also considers the changes in blocks, which is monitored using a new statistic, BD. The ICA algorithm is applied to monitor each block, and SVDD is conducted to combine the monitoring results. After the block division operation, the local information has been successfully utilized. Furthermore, the novel multi-block method improves the interpretation of process analysis. The new statistic BD evaluates the variations between blocks, which is a useful supplement to and extension of MBICA. Compared with PCA, ICA, KICA, and SVDD, the proposed method was demonstrated to be efficient using a case study of the TE process. However, the proposed method is a completely data-driven method, which lacks prior knowledge and expert experience. In the future, the applications of this method can be extended to image analysis and signal processing. Associated content



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Supporting information Detailed description of the variables, 21 faults, and the control scheme for TE process. This information is available free of charge via the Internet at http://pubs.acs.org/. Author information Corresponding Author *E-mail address: [email protected]. Mailing address: East China University of Science and Technology, P.O. Box 293, MeiLong Road no. 130, Shanghai 200237, P. R. China. Notes The authors declare no competing financial interest. Acknowledgments The authors gratefully acknowledge the support from the following foundations: the 973 project of China (2013CB733600) and the National Natural Science Foundation of China (21176073). References (1) Alcala, C. F.; Qin, S. J. Reconstruction-based Contribution For Process Monitoring with Kernel Principal Component Analysis. Ind. Eng. Chem. Res. 2010, 49, 7849-7857. (2) Jiang, Q. C.; Yan, X. F.; Zhao, W. X. Fault Detection and Diagnosis in Chemical Processes Using Sensitive Principal Component Analysis. Ind. Eng. Chem. Res. 2013, 52, 1635-1644. (3) Lv, Z. M.; Jiang, Q. C.; Yan, X. F. Batch Process Monitoring Based on Multisubspace Multiway Principal Component Analysis and Time-Series Bayesian Inference. Ind. Eng. Chem. Res. 2014, 53, 6457-6466. (4) Ding, S. X. Data-driven Design Of Monitoring and Diagnosis Systems for Dynamic Processes: A Review of Subspace Technique Based Schemes and Some Recent Results. J. Process Contr. 2014, 24, 431-449. (5) Zhang, Y.; Zhou, H.; Qin, S. J.; Chai, T. Decentralized Fault Diagnosis of Large-Scale Processes Using Multiblock Kernel Partial Least Squares. IEEE T. Ind. Inform.,2010, 6, 3-10. (6) Cheng, C. Y.; Hsu, C. C.; Chen, M. C. Adaptive Kernel Principal Component Analysis (KPCA) for Monitoring Small Disturbances of Nonlinear Processes. Ind. Eng. Chem. Res. 2010, 49, 2254-2262. (7) Chiang, L. H.; Braatz, R. D.; Russell, E. L. Fault Detection And Diagnosis In Industrial Systems; Springer, 2001. (8) Ge, Z. Q.; Song, Z. H. Process Monitoring Based on Independent Component Analysis-Principal Component Analysis (ICA-PCA) and Similarity Factors. Ind. Eng. Chem. Res. 2007, 46, 2054-2063. (9) Ge, Z. Q.; Yang, C. J.; Song, Z. H. Improved Kernel PCA-Based Monitoring Approach for Nonlinear Processes. Chem. Eng. Sci. 2009, 64, 2245-2255. (10) Lee, J. M.; Qin, S. J.; Lee, I. B. Fault Detection And Diagnosis Based on Modified Independent Component Analysis. AIChE J. 2006, 52, 3501-3514. (11) Huang, J.; Yan, X. F. Gaussian and non-Gaussian Double Subspace Statistical Process Monitoring Based on Principal Component Analysis and Independent Component Analysis. Ind. Eng. Chem. Res. 2015, 54, 1015-1027. (12) Hyvarinen, A. Survey on Independent Component Analysis. Neural computing surveys 1999, 2, 94-128. (13) Hyvärinen, A.; Oja, E. Independent Component Analysis: Algorithms and Applications. Neural networks 2000, 13, 411-430.


Page 14 of 23

Page 15 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(14) Lee, J. M.; Qin, S. J.; Lee, I. B. Fault Detection of non-Linear Processes Using Kernel Independent Component Analysis. Can. J. Chem. Eng. 2007, 85, 526-536. (15) Lee, J. M.; Yoo, C.; Lee, I. B. Statistical Monitoring of Dynamic Processes Based on Dynamic Independent Component Analysis. Chem. Eng. Sci. 2004, 59, 2995-3006. (16) Li, R. F.; Wang, X. Z. Dimension Reduction of Process Dynamic Trends Using Independent Component Analysis. Comput. Chem. Eng. 2002, 26, 467-473. (17) MacGregor, J. F.; Jaeckle, C.; Kiparissides, C.; Koutoudi, M. Process Monitoring and Diagnosis by Multiblock PLS Methods. Aiche J. 1994, 40, 826-838. (18) Westerhuis, J. A.; Kourti, T.; MacGregor, J. F. Analysis of Multiblock and Hierarchical PCA and PLS Models. J. Chemometr 1998, 12, 301-321. (19) Qin, S. J.; Valle, S.; Piovoso, M. J. On Unifying Multiblock Analysis with Application yo Decentralized Process Monitoring. J. Chemometr 2001, 15, 715-742. (20) Ge, Z. Q.; Song, Z. H. Distributed PCA Model for Plant-Wide Process Monitoring. Ind. Eng. Chem. Res. 2013, 52, 1947-1957. (21) Jiang, Q. C.; Yan, X. F. Plant-wide Process Monitoring Based on Mutual Information-Multiblock Principal Component Analysis. Isa T. 2014, 53, 1516-1527. (22) Jiang, Q. C.; Yan, X. F. Monitoring Multi-Mode Plant-Wide Processes by Using Mutual Information-based Multi-block PCA, Joint Probability, and Bayesian Inference. Chemometr. Intell. Lab. 2014, 136, 121-137. (23) Rashid, M. M.; Yu, J. A New Dissimilarity Method Integrating Multidimensional Mutual Information and Independent Component Analysis for non-Gaussian Dynamic Process Monitoring. Chemometr. Intell. Lab. 2012, 115, 44-58. (24) Krzanowski, W. J. Between-Groups Comparison of Principal Components. J. Am. Stat. Assoc. 1979, 74, 703. (25) Singhal, A.; Seborg, D. E. Pattern Matching in Historical Batch Data Using PCA. IEEE Contr. Syst. Mag. 2002, 22, 53-63. (26) Topchy, A.; Jain, A. K.; Punch, W. Clustering Ensembles: Models of Consensus and Weak Partitions. IEEE T. Pattern Anal. 2005, 27, 1866-1881. (27) Strehl, A.; Ghosh, J. Cluster Ensembles - A Knowledge Reuse Framework for Combining Partitionings. Eighteenth National Conference on Artificial Intelligence (Aaai-02)/Fourteenth Innovative Applications of Artificial Intelligence Conference (Iaai-02), Proceedings 2002, 93-98. (28) Tax, D. M. J.; Duin, R. P. W. Support Vector Data Description. Mach. Learn. 2004, 54, 45-66. (29) Tax, D. M. J.; Duin, R. P. W. Support Vector Domain Description. Pattern Recogn. Lett. 1999, 20, 1191-1199. (30) Lee, J. M.; Yoo, C. K.; Lee, I. B. Statistical Process Monitoring with Independent Component Analysis. J. Process Contr. 2004, 14, 467-485. (31) Leski, J. M.; Kotas, M. Hierarchical Clustering with Planar Segments As Prototypes. Pattern Recogn. Lett. 2015, 54, 1-10. (32) Downs, J. J.; Vogel, E. F. A Plant-wide Industrial Process Control Problem. Comput. Chem. Eng. 1993, 17, 245-255. (33) Lyman, P. R.; Georgakis, C. Plant-wide Control of the Tennessee Eastman Problem. Comput. Chem. Eng. 1995, 19, 321-331. (34) Wang, J.; He, Q. P. Multivariate Statistical Process Monitoring Based on Statistics Pattern Analysis. Ind. Eng. Chem. Res. 2010, 49, 7858-7869. (35) Tong, C. D.; Song, Y.; Yan, X. F. Distributed Statistical Process Monitoring Based on Four-Subspace Construction and Bayesian Inference. Ind. Eng. Chem. Res. 2013, 52, 9897-9907.



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

List of Tables Table 1. Block division results Table 2. Fault detection rates of the TE process


Page 16 of 23

Page 17 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 1. Block division results Block NO.

1

2

3

4

Variable

1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 16, 18, 19, 20, 21, 22, 23, 12,

15,

17,

NO.

24, 25, 26, 27, 28, 31, 32

30

33

29

Table 2. Fault detection rates of the TE process PCA

Fault

ICA

I2

SVDD

MBPCA-BD

T2

Q

1

0.99

1

1

1

1

1

1

0.53

1

2

0.98

0.99

0.99

0.99

0.98

0.98

0.99

0.96

0.99

3

0.00

0.01

0.03

0.08

0.01

0.03

0.15

0.20

0.13

4

0.54

0.96

0.78

0.89

0.81

1

0.96

0.31

1

5

0.23

0.25

1

1

0.25

0.28

0.38

0.43

1

6

0.99

1

1

1

1

1

1

0.96

1

7

1

1

1

1

1

1

1

0.78

1

8

0.98

0.98

0.97

0.99

0.97

0.98

0.99

0.58

0.99

9

0.01

0.02

0.01

0.03

0.01

0.03

0.06

0.00

0.07

10

0.33

0.34

0.75

0.76

0.81

0.78

0.62

0.57

0.88

11

0.21

0.64

0.70

0.55

0.58

0.77

0.74

0.24

0.82

12

0.97

0.98

1

1

0.99

0.99

1

0.62

1

13

0.94

0.96

0.95

0.96

0.95

0.95

0.95

0.93

0.96

14

1

0.99

1

1

1

1

1

0.41

1

15

0.01

0.03

0.04

0.15

0.03

0.05

0.22

0.08

0.26

16

0.16

0.25

0.80

0.75

0.77

0.87

0.56

0.65

0.85

17

0.74

0.89

0.88

0.90

0.91

0.97

0.93

0.73

0.97

18

0.89

0.90

0.91

0.90

0.89

0.91

0.91

0.82

0.91

19

0.14

0.28

0.65

0.37

0.70

0.85

0.15

0.06

0.88

20

0.32

0.60

0.78

0.74

0.50

0.65

0.65

0.83

0.92

21

0.26

0.43

0.43

0.49

/

/

0.51

0.88

0.59

NO.

I2

KICA Q

d/R

Q


BD

d/R


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

List of figures Figure 1. Flowchart of the proposed MBICA-BD method Figure 2. Illustration of block dissimilarity strategy Figure 3. The cluster results of TE process variables Figure 4. The ∆D value for each number of clusters Figure 5. Monitoring results of fault 10: (a) ICA (b) SVDD (c) BD (d) MBICA Figure 6. Monitoring results of fault 11: (a) ICA (b) SVDD (c) BD (d) MBICA Figure 7. Monitoring results of fault 20: (a) ICA (b) SVDD (c) BD (d) MBICA Figure 8. Monitoring results of fault 21: (a) ICA (b) SVDD (c) BD (d) MBICA


Page 18 of 23

Page 19 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


⋅⋅⋅

X1

I12 SPE

⋅⋅⋅

Ii 2 SPE

⋅⋅⋅

Xi

⋅⋅⋅

Xb

Ib 2

SPE

d/R Figure 1. Flowchart of the proposed MBICA-BD method

Figure 2. Illustration of block dissimilarity strategy



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure 3. The cluster results of TE process variables

Figure 4. The ∆ D value for each number of clusters


Page 20 of 23

Page 21 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(a)

(a)

(b)

(b)

(c)

(c)



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

(d)

(d)

Figure 5. Monitoring results of fault 10: (a) ICA


(b) SVDD (c) BD (d) MBICA


(a)

(a)

(b)

(b)


Page 22 of 23

Page 23 of 23

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(c)

(c)

(d)

(d)






Angle-Based Multiblock Independent Component Analysis Method

Recommend Documents