Unsupervised change point detection using a weight graph method for

Subscriber access provided by - Access paid by the | UCSB Libraries

Process Systems Engineering

Unsupervised change point detection using a weight graph method for process monitoring Ruqiao An, ChunJie Yang, and YIJUN PAN Ind. Eng. Chem. Res., Just Accepted Manuscript • DOI: 10.1021/acs.iecr.8b02455 • Publication Date (Web): 04 Jan 2019 Downloaded from http://pubs.acs.org on January 5, 2019

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Industrial & Engineering Chemistry Research

Unsupervised change point detection using a weight graph method for process monitoring Ruqiao An, Chunjie Yang,∗ and Yijun Pan Department of Control Science and Engineering, Zhejiang University, Hangzhou, China E-mail: [email protected]

Abstract Since industrial processes are complicated and time-varying in general, unsupervised and nonparametric process monitoring methods are necessary. Recently, a graph-based change point detection method with a developed scan statistic, which is unsupervised and nonparametric, has been introduced. Industrial processes are primarily continuous with considerable important information contained in the time relations of adjacent observations. This important information should be used for process monitoring, which could improve the power of change point detection. In this paper, the scan statistic based on the graph method is adopted for process monitoring, and the time intervals between observations are attempted for calculating the weights of the edges. There are two steps for detecting the change point: 1) construct the connection graph: a minimum spanning tree is used for constructing the connecting graph, and the weights of the edges are calculated based on the time intervals and Euclidean distances between the observations; and 2) calculate the scan statistic: the number of edges connecting the observations derived from two parts (before the change point and after the change point) is counted as a statistic for detecting the change point. In this paper, the Tennessee Eastman (TE) process and a blast furnace process are used to illustrate the power of the weight graph method for process monitoring. ∗ To

whom correspondence should be addressed

1

ACS Paragon Plus Environment

Industrial & Engineering Chemistry Research 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Introduction In the industrial processes, the production safety and product quality are important issues. Process monitoring could check the sensor and process faults during industrial processes to avoid economic loss; thus, this approach has attracted considerable attention in recent years.1−4 Change point detection, one of the approaches for process monitoring, attempts to indicate whether a change is occurring. The change point detection process has been researched widely for several years in applications including the annual discharge of rivers,5 genomic sequence analysis,6 degree of steadiness (DOS) assessment,7 and image analysis.8−10 There are two primary change point detection methods: supervised and unsupervised. Moreover, both supervised and unsupervised methods can be divided into two categories: parametric methods and nonparametric methods.11 Since industrial processes are controlled by operators in general, a data matrix without any faults is hard to obtain.12 Thus, the unsupervised method is attractive. Since distributed control systems are developing rapidly, the collected data are massive. The operating conditions are diverse.2 The nonparametric methods are more suitable than parametric methods with less required information and cost.13,14 Therefore, applying unsupervised and nonparametric methods for process monitoring is promising for use in the industrial processes. Many unsupervised and nonparametric change point detection methods have been researched during recent decades.11 Desobry et al. proposed a kernel change detection method, which compares two sets of descriptors extracted online from the signal at each time instant, i.e., the immediate past set and the immediate future set. They build a dissimilarity measure in feature space between those sets, without estimating densities as an intermediary step.15 A nonparametric method based on Wilcoxon rank statistic was developed by Lung-Yut-Fong et al., and they also concern the use of the proposed test statistic to perform retrospective change-point detection.16 Harchaoui et al. proposed a test statistic based upon the maximum kernel Fisher discriminant ratio as a measure of the homogeneity between segments. Moreover, they derive its limiting distribution under the null hypothesis (no change occurs) and establish the consistency under the alternative hypothesis (a change occurs).17 A new test statistic which has a known exact distribution and is exactly 2


Page 2 of 32

Page 3 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


distribution free was proposed by Rosenbaum et al. The new statistic compares two multivariate distributions by using distances between observations.18 Lu et al. proposed a new computation method to measure anomalies and a new statistical test is employed by using martingale for detecting a potential change in a given time series.19 A widely applied Bayesian online change point detection method has been presented by Adams et al.20 Recently, the graph-based change point detection method has been proposed.21 Compared with the methods mentioned above, the graph-based method is both unsupervised and nonparametric without many restrictions and is suitable for the process monitoring of industrial processes. This method constructs a graph, which represents the similarity between observations. The n observations are divided into two parts based on the position of the change point candidate, and a scan statistic is developed. It is an unsupervised method without a training matrix, which is also a powerful change point detection method for a high-dimensional data matrix without a parameter choice. This method has been utilized in the authorship debate and friendship network areas.21 However, to the best knowledge of the authors, the graph method has been barely investigated for use in process monitoring. Musulin et al. introduced a process monitoring method based on spectral graph analysis theory.22,23 Different from the above mentioned literature, a graph is constructed based on the Euclidean minimum spanning tree, and a scan statistic is proposed.21 In the literature,21 the observations were treated as equal nodes when the graph was constructed in the authorship debate and friendship network areas, which do not need to consider the time relations between observations. The time relations between observations indicate the important information contained in the adjacent observations. The graph-based method constructs a connecting graph based on the minimum spanning tree. All observations are in an equal status to calculate the Euclidean distances. In continuous industrial processes, the collected observations may contain important information regarding the time relations. The observations should connect to the near observations under normal conditions. The situation that the observations always attempt to connect to the one in the same part would be unusual, which may indicate that a disturbance occurred. The sentence "in the same

3



part" means that both of two observations are collected before the change point or after the change point. To reduce the influences of disturbances, a weight graph method which utilizes the time relations between observations is developed in this paper. There are two steps for detecting the change point. First, a graph should be constructed by the minimum spanning tree. The weights of the edges in the minimum spanning tree are first calculated by the Euclidean distances between the observations, and then are adjusted by the time intervals between the observations. The time interval means the time distance between two observations. The greater time interval corresponds to a larger weight. In this way, the observations can connect to the near ones, which will reduce the occurrence of confusion between the change points and disturbances. Finally, the number of edges connecting the two different parts is used as a scan statistic to detect a change point. If the number of edges connecting two different parts is small, there will be a change point. During industrial processes, the weight graph change point detection method can reduce the influences of noise and outliers, which means the method is robust. Moreover, the method could detect the faults occurring for a period of time. For oscillations, the process could be divided into some parts to get some change points to indicate the faults. In this paper, the time interval is a criterion to divide the observations and is also a way to control the fault alarm rate and fault detection rate. The time interval is not a constant, and it can be chosen based on the condition of the processes. An initial value of the time interval can be given, and adjusted based on some labeled data in each process. If there is an outlier, which occurs at a short time in general, then a small time interval would change the connection orders of the observations in the graph, which may lead to a high fault alarm rate. Different from the outliers, a fault occurs at a period time, which is much longer, and a large time interval leads to a low fault detection rate. Therefore, the appropriate time interval is important, which will reduce the influences of outliers and not affect the detection of faults. Since noise exists throughout these processes but does not greatly influence the detection of the change point which is introduced in the assumption 2, the influences of noise are not considered in this paper for convenience. There are no restricted conditions regarding data based on the weight graph method, such as

4


Page 4 of 32

Page 5 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


non-linear and non-Gaussian data, which is the same as the graph-based method.21 The method can be used for high-dimension data, and the situation that the number of variables are greater than the number of observations in data matrix, which is much more suitable than traditional datadriven methods for process monitoring. Moreover, a training matrix is not necessary, and thus, the computational cost and memory space are less. Therefore, the weight graph change point detection method, which is nonparametric and unsupervised, could be used for process monitoring in the industrial processes. The problem formulation is outlined in section 2. The graph-based method and the weight graph change point detection method are described in section 3. In section 4, the simulation results based on the weight graph method for the numerical simulation, the TE process and a blast furnace process are presented. Section 5 concludes this paper.

Problem Formulation Given a data matrix X ∈ Rm×n , xi is an observation with m variables, i = 1, 2, ..., n. The problem is to verify whether there is a change point τ. This is a two sample test problem. The null hypothesis is H0 : xi ∼ F0 , i = 1, 2, ..., n

(1)

The alternative hypothesis is H1 : xi =

   F0 , i ≤ τ

(2)

  F1 , i > τ where τ is a change point, 1 ≤ τ < n. F0 and F1 are two different data distributions. There are two assumptions that should be considered. Assumption 1. The observations collected from industrial processes are independent. Assumption 2. The noise distribution is the same throughout the processes. Assumption 1 that the observations collected from the industrial processes are independent is trivial. If the industrial processes are continuous, the observations with small sampling times will 5



Page 6 of 32

be dependent. The independence can be solved by increasing the sampling time. Moreover, the graph-based method is still powerful when the assumption is violated slightly.21 In the simulation section, it will be illustrated that the weight graph method is a powerful approach for process monitoring. Therefore, increasing the sampling time between observations can be used for addressing the dependence of the observations.

Unsupervised weight graph change point detection method Graph-based change point detection method A series of observations xi with m variables, i = 1, 2, ..., n, are ordered by time. This two-sample test problem with a null hypothesis H0 and an alternative hypothesis H1 is shown as follows H0 : xi ∼ F0 , i = 1, 2, ..., n    F0 , i ≤ τ H1 : xi =   F1 , i > τ

(3)

where τ is a change point, 1 ≤ τ < n. F0 and F1 are different distributions of data. Each observation xi is regarded as a node in the constructed graph. The primary three ways to construct the graph are based on the minimum spanning tree,24 minimum distance pairing25 and nearest neighbor graph.26 In this paper, the graph is constructed by the minimum spanning tree. The minimum spanning tree is a subset of the edges of a connected undirected graph, which connects all observations without cycles. The Euclidean distance between two observations is used for calculating the edge weight, and the connection rule attempts to connect two nodes with the smallest edge weight. Some definitions are introduced as follows. G is the graph constructed by the minimum spanning tree and the set of edges. Gi is a subgraph containing all edges that connect to observation xi . |Gi | is the number of edges in Gi . τ is a change point which divides all observations into two

6


Page 7 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


parts.21 The indicator function Ix is defined as follows,27

Ix =

   1,

if x is true

  0,

otherwise

(4)

Therefore, for any possible value t of τ, the number of edges connecting two parts (before τ and after τ) can be calculated as follows RG (t) =

∑

Igi (t)6=g j (t)

(i, j)∈G

(5)

gi (t) = Ii>t If RG (t) is small, which means the number of edges connecting two parts is small, then there will be a change point. If there is no change point, the connections of observations will be random, and the RG (t) will be irregular. A small RG (t) means that the observations tend to connect to the one in the same part, which is evidence against the null hypothesis.21

Unsupervised weight graph change point detection method Graph-based change point detection is an unsupervised method, which divides the observations into two parts and counts the number of edges connecting the two different parts as a statistic. The graph-based method does not need to set up parameters and collect the training matrix with a powerful performance guarantee. However, this method calculates the Euclidean distances between observations and does not consider the time relations between observations, which may contain considerable important information. Therefore, a graph-constructed method considering the time relations between observations is developed. The time intervals are used for addressing the time relations of the observations after calculating the Euclidean distances. The weight between two observations is calculated based on their time interval. Since the industrial processes are continuous without sudden change in general, the observations should connect to the near one under normal conditions. The calculated weights a7



Page 8 of 32

gree with the production principal of continuous industrial processes. The weight graph method reduces the influences of disturbances by considering the time relations in the industrial processes to improve the power of process monitoring. The detailed mathematical formulas are introduced as follows. The difference between the graph-based method in the literature21 and the weight graph method is an approach to calculate the weights of edges. The graph in the literature21 is constructed based on the Euclidean distances between observations. The two observations that have a smallest weight are connected. The Euclidean distances between observations are recorded in the matrix A, and ai j = kxi − x j k2 . After calculating the Euclidean distances, the time intervals also need to be considered. The novel weight matrix B is computed as follows

bi j = kxi − x j k2 × (d

|i − j| e)/n 0.25 × n

(6)

where xi , x j are the ith , jth observations, n is the number of observations, and de is a ceil function. The time interval is obtained from 0.25 × n based on the experience, which could be different in each condition.

Process monitoring with an unsupervised weight graph method Industrial processes are mainly complicated and unpredictable. The production states are changing all the time, which means the nonparametric methods are suitable for process monitoring in the industrial processes. Since the industrial processes are primarily controlled by operators, which are subjective, the training matrix without any fault is difficult to obtain. The corrupted training matrix may lead to poor process monitoring results. Therefore, an unsupervised and nonparametric method should be used in the industrial processes. Suppose that there are some time series xi , i = 1, 2, ..., n, where each xi is an observation with m variables, and τ is a change point. The steps of the unsupervised weight graph method for process monitoring are outlined as follows.

8


Page 9 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Step 1: Euclidean distance calculation Calculate the Euclidean distances between observations, and record them into a matrix A. The Euclidean distance of same observation is set to be infinite to avoid connecting to itself.

ai j = kxi − x j k2

(7)

where xi , x j are the ith , jth observations.28 Step 2: Weight matrix calculation Calculate the weights based on the time intervals between observations, and construct the weight Euclidean distance matrix B.

bi j = kxi − x j k2 × (d

|i − j| e)/n 0.25 × n

(8)

where xi , x j are the ith , jth observations and n is the number of total observations. Step 3: Graph construction According to the matrix B, the minimum spanning tree method is used for constructing the graph. The observations with the smallest weight calculated in step 2 are connected without cycles. Step 4: The number of edges count The number of edges RG (t) connecting two observations from different parts are counted. The change point candidate t of τ divides the observations into two parts. RG (t) =

∑

Igi (t)6=g j (t)

(i, j)∈G

(9)

gi (t) = Ii>t Step 5: Statistic calculation Since RG (t) depends on t, the standardization of RG (t) is computed, which means that the standardized RG (t) is comparable across t.21 The formula of standardization is shown as follows. The minus sign is used to conveniently observe the change point detection results. The large values

9



Page 10 of 32

of ZG (t) are evidence against the null hypothesis.21 RG (t) − E[RG (t)] ZG (t) = − p Var[RG (t)]

(10)

where E[RG (t)] = p1 (t)|G| 1 Var[RG (t)] = p2 (t)|G| + ( p1 (t) − p2 (t)) ∑ |Gi |2 + (p2 (t) − p21 (t))|G|2 2 i 2t(n − t) n(n − 1) 4t(t − 1)(n − t)(n − t − 1) p2 (t) = n(n − 1)(n − 2)(n − 3)

(11)

p1 (t) =

Step 6: Process monitoring If one of the elements in ZG (t) is obviously greater than others, then there will exist a change point. A fault occurs in the process, which needs to be controlled by operators.

Simulation In this part, the unsupervised weight graph change point detection method is tested in the numerical simulation, the TE process and a blast furnace process. The numerical simulation with proper settings is used for illustrating the power of the weight graph method. The TE process that simulates practical industrial processes and a blast furnace process are used for proving the effectiveness of the weight graph method for process monitoring in the industrial processes.

Numerical simulation In this section, the power for change point detection based on the weight graph method is illustrated. The weight graph method is unsupervised, and a series of observations should be constructed as a testing matrix with a change point. Therefore, the numerical simulation is constructed by 40 observations with a change point τ = 20. The first 20 observations are obtained from N(0, I2 ),

10


Page 11 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


0

and the other 20 observations come from N((2, 2) , I2 ).21 The outliers are added in the numerical simulation to verify that the weight graph method could reduce the influences of the disturbances. A total of 3% of the outliers are drawn from the uniform distribution with a zero mean and a 0.5 standard deviation. The time interval is 10 in this part chosen by 0.25 × n. n is the number of testing observations, which is equal to 40 in this part. The connection graphs constructed based on the Euclidean minimum spanning tree and the weight Euclidean minimum spanning tree are presented in Fig. 1 and Fig. 2. The connection orders of observations based on two methods are shown in Table 1, which are obtained from the workspace of Matlab. The change point detection results of two methods are shown in Fig. 3 and Fig. 4. minimum spanning tree

4

3

2

1

0

-1

-2 -2

-1

0

1

2

3

4

Figure 1: The connection graph of observations based on the graph method for a numerical example. The ’4’ represents the observations derived before the change point τ (including the observation at the change point), and the ’◦’ represents the observations derived after the change point τ. The RG (t) means the number of edges connecting two observations from different parts and the different parts are divided by a change point candidate t. Since a small change point candidate leads to the number of edges connecting two observations from different parts is small, the first 11



minimum spanning tree

4

3

2

1

0

-1

-2 -2

-1

0

1

2

3

4

Figure 2: The connection graph of observations based on the weight graph method for a numerical example.

Z(t)

5

4

3

2

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 12 of 32

1

0

-1

-2 0

5

10

15

20

25

30

35

40

t

Figure 3: The change point detection result based on the graph method for a numerical example.

12


Page 13 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 1: The connection orders of observations based on two methods No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

Graph 1→4 4→5 4 → 12 12 → 16 1→2 2→3 2→9 9→6 6 → 31 9 → 25 25 → 13 25 → 7 7 → 18 13 → 37 37 → 33 33 → 34 34 → 17 17 → 36 36 → 40 40 → 21 33 → 14 21 → 35 21 → 26 35 → 29 29 → 24 24 → 27 34 → 38 38 → 30 30 → 22 30 → 39 39 → 28 22 → 23 31 → 10 10 → 8 5 → 11 11 → 20 26 → 32 20 → 15 12 → 19

13

Weight graph 1→4 4→5 4 → 12 12 → 16 1→2 2→3 2→9 9→6 9→7 7 → 18 9 → 13 13 → 25 6 → 14 14 → 17 17 → 34 34 → 33 33 → 37 34 → 36 36 → 40 40 → 21 40 → 35 21 → 26 35 → 29 29 → 24 24 → 27 34 → 38 38 → 30 30 → 22 30 → 39 22 → 23 37 → 31 3 → 10 10 → 8 5 → 11 11 → 20 26 → 32 18 → 28 20 → 15 12 → 19



Z(t)

7 6 5 4 3

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 14 of 32

2 1 0 -1 -2 -3 0

5

10

15

20

25

30

35

40

t

Figure 4: The change point detection result based on the weight graph method for a numerical example. 5% and final 5% sampling time are not considered for deciding the change point. The dotted lines are used to indicate it. From Fig. 1, Fig. 2 and Table 1, compared with the graph-based method, in the weight graph method, the observation tends to connect to the near one based on the order of time. For example, in Table 1, the 9th connection order is the 6th observation trying to connect to the 31st observation based on the graph method, and the 9th observation trys to connect to the 7th observation based on the weight graph method. The industrial processes are primarily continuous, and the weight graph method is suitable for addressing the disturbances in the processes. A few outliers will not change the connection orders by calculating the weights. From Fig. 3 and Fig. 4, both the graph and the weight graph methods can detect the change point, but the weight graph method is more stable. There are few fluctuations based on the weight graph method by considering important information contained in the time relations between observations. Therefore, the power of the weight graph method for detecting the change point is illustrated by a numerical simulation, which could be utilized for process monitoring in the industrial processes.

14


Page 15 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Process monitoring in the industrial processes In the section, the TE process which simulates practical industrial processes and a blast furnace process are used for illustrating the process monitoring power of the weight graph change point detection method. The flow chart of the TE process is shown in Fig. 5; this example is widely used for testing the power of the process monitoring methods during the industrial processes.29−32 The data matrix collected from the TE process contains 41 measured variables and 11 manipulated variables, except a constant mixing speed. There are 21 preset faults in the TE process listed in Table 2. The data matrices are constructed by 22 different modes (including 1 normal operation mode and 21 fault collection modes), and every time the rand seed is changed. Each data matrix contains 960 observations, and the faults are introduced after the 160th observation for 21 fault collection modes.33,34 To illustrate the power of the weight graph method, a discussion regarding the graph method, the weight graph method and the PCA method is presented in this section. The training matrix of the PCA method contains 480 normal operation observations, and the other 480 normal operation observations are used for adjusting the control limit based on 5% fault alarm rate. The testing matrix of three methods is composed of the 141st-180th observations, and there is a change point at the 21st observation. The time interval is 10 in this part, chosen by 0.25 × n. n is the number of testing observations, which is equal to 40 in this part. Moreover, the PCA method utilizes 22 measured variables (XMEAS(1-22)) and 11 manipulated variables (XMV(1-11)) for process monitoring in the section, and 9 principal components are retained in the principal component space based on the 65% accumulative contribution rate derived from the PCA model.32 The control limit of T 2 statistic is 21.2, and the control limit of SPE statistic is 35.7 based on the PCA method (α = 0.01). The simulation results of three methods for fault 1 and fault 15 in the TE process are shown in Fig. 6-Fig. 11. The time of detecting the change points based on three methods are listed in Table 3. The true change points occur at the 21st sampling time. The difference between 21 and the value shown in Table 3 could be used for proving the power of the weight graph method. Based on the PCA method, if the process monitoring statistics of continuous 5 observations are all greater than the control limit, the first observation is determined as a change 15



point occurrence to reduce the fault alarm rate. Table 2: The descriptions of 21 faults in the TE process No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Variable A/C feed ratio, B composition constant B composition, A/C ratio constant D feed temperature Reactor cooling water inlet temperature Condenser cooling water inlet temperature A feed loss C header pressure loss-reduced availability A, B, and C feed composition D feed temperature C feed temperature Reactor cooling water inlet temperature Condenser cooling water inlet temperature Reaction kinetics Reactor cooling water valve Condenser cooling water valve Unknown Unknown Unknown Unknown Unknown The Valve fixed at steady state position

Type Step Step Step Step Step Step Step Random variation Random variation Random variation Random variation Random variation Slow drift Sticking Sticking Unknown Unknown Unknown Unknown Unknown Constant position

Fig. 6 and Fig. 7 are the simulation results of fault 1 based on two unsupervised methods in the TE process. Fault 1 is a step variation in the A/C feed ratio. The weight graph method could detect the change point sooner than the graph method. Fault 15, which is a sticking fault, is also used for illustrating the power of the weight graph method. From Fig. 9, the change point cannot be detected by the graph method, while the weight graph method immediately detects the change point. From Table 3, it can be concluded that the scan statistic based on the weight graph method exhibits better performance than that based on the graph method, except for fault 13. However, there is only one sampling time delay, which is acceptable. In some situations, the scan statistic based on the graph method could not detect the faults, such as fault 3 and 9. For most faults in the TE process, the weight graph method is found to have a better performance than the PCA method except for fault 6 and 14. For 21 testing matrices, the fault type and 16


Page 16 of 32

Page 17 of 32

Figure 5: The flow chart of the TE process.

Z(t)

8 7 6 5 4

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


3 2 1 0 -1 -2 0

5

10

15

20

25

30

35

40

t

Figure 6: The change point detection result using the graph method for fault 1 in the TE process.

17



Z(t)

8 7 6 5

Z

4 3 2 1 0 -1 -2 0

5

10

15

20

25

30

35

40

t

Figure 7: The change point detection result using the weight graph method for fault 1 in the TE process.

T2 statistic

T2 statistic

400

testing data control limit

300 200 100 0 0

5

10

15

20

25

30

35

40

sampling point SPE statistic

800

SPE statistic

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 18 of 32


600 400 200 0 0

5

10

15

20

25

30

35

40

sampling point

Figure 8: The process monitoring result using the PCA method for fault 1 in the TE process.

18


Page 19 of 32

Z(t)

5 4 3

Z

2 1 0 -1 -2 -3 0

5

10

15

20

25

30

35

40

t

Figure 9: The change point detection result using the graph method for fault 15 in the TE process.

Z(t)

7 6 5 4 3

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


2 1 0 -1 -2 0

5

10

15

20

25

30

35

40

t

Figure 10: The change point detection result using the weight graph method for fault 15 in the TE process.

19



T2 statistic

T2 statistic

30


20 10 0 0

5

10

15

20

25

30

35

40


40

SPE statistic

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 20 of 32


30 20 10 0 0

5

10

15

20

25

30

35

40

sampling point

Figure 11: The process monitoring result using the PCA method for fault 15 in the TE process. Table 3: Comparison of three methods 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

Graph 26 29 N/A N/A 31 N/A 21 N/A N/A 27 N/A 27 29 30 N/A N/A N/A N/A 35 N/A N/A

Weight graph 23 25 22 22 21 N/A 21 N/A 29 27 N/A 27 30 27 21 N/A 24 N/A 32 27 27 20

PCA(T 2 ) PCA(SPE) 27 23 35 37 N/A N/A N/A 23 33 27 24 21 21 21 N/A N/A N/A N/A N/A N/A N/A N/A N/A 28 N/A N/A 24 21 N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A


Page 21 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


data feature are various. The weight graph method compares the data distribution of two parts (divided by a change point candidate) for process monitoring. The different data distribution is regarded as a change point occurrence. The PCA method uses the T 2 statistic and the SPE statistic for process monitoring. The situation that one of statistics of a testing observation is greater than the control limit is detected as a change point candidate. The different detection method maybe the reason why the PCA exhibits the better simulation results for fault 6 and 14. Moreover, the PCA method is a conventional method with the T 2 statistic and the SPE statistic. The scan statistic based on the weight graph method is novel, which just counts the number of edges connecting two observations derived from different parts. In the further research, the k-nearest neighbor or clustering methods could be used for calculating the weights in the minimum spanning tree to improve the process monitoring power. Compared with the PCA method, the weight graph method is unsupervised and non-parameter, which is necessary for process monitoring in the industrial processes. Thus, although for fault 6 and 14, the PCA exhibits the better performance, the weight graph method is significant for research. In a blast furnace process, the solid materials including coke and ore are put from the top of a blast furnace, and the hot wind blows into the bottom of a blast furnace. The hot air is preheated to 1200 degrees containing abundant oxygen, oil, natural gas and coal dust. The reaction products are hot metal and slag.35 More details could be found in the literature.4 In a blast furnace process, the raw materials are put layer by layer, which may lead to the faults including low stock line and slip. The low stock line indicates that the stock line is lower than the normal condition, and the slip indicates the raw materials dropping off abruptly. Moreover, the temperature of a blast furnace is an important index reflecting the production condition. In this section, the faults including low stock line, slip and cooling are used for illustrating the power of the weight graph change point detection method. The 18 variables introduced in the literature12 are used for process monitoring. The 20 normal observations are used as the training matrix based on the PCA method. The other 20 observations collected under the normal condition are used for adjusting the control limit based on 5% fault

21



alarm rate. The testing matrix of three methods consists of 20 normal observations and 20 fault observations. Moreover, 3 principal components are retained in the principal component space based on the 80% accumulative contribution rate derived from the PCA model. The control limit of T 2 statistic is 21.4, and the control limit of SPE statistic is 78.8 based on the PCA method (α = 0.01). The change point is introduced in the 21st sampling time. The time interval is 10 in this part, chosen by 0.25 × n. n is the number of testing observations, which is equal to 40 in this part. The simulation results are presented in Fig. 12-Fig. 20. The time of detecting the change points based on three methods are listed in Table 4. The true change point occurs at the 21st sampling time. In Figure. 17, since the values of the T 2 and SPE statistic are large after the 21st observation, the range of Y-axis is preset for illustrating clearly. Z(t)

4

3

2

1

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 22 of 32

0

-1

-2

-3 0

5

10

15

20

25

30

35

40

t

Figure 12: The change point detection result using the graph method for low stock line in a blast furnace process.

Table 4: Comparison of three methods Fault Graph Weight graph PCA(T 2 ) Low stock line N/A 21 N/A Slip 21 21 21 Cooling N/A 21 N/A 22


PCA(SPE) N/A 21 21

Page 23 of 32

Z(t)

7 6 5 4

Z

3 2 1 0 -1 -2 0

5

10

15

20

25

30

35

40

t

Figure 13: The change point detection result using the weight graph method for low stock line in a blast furnace process.

T2 statistic

T2 statistic

60


40 20 0 0

5

10

15

20

25

30

35

40


600

SPE statistic

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60



400 200 0 0

5

10

15

20

25

30

35

40

sampling point

Figure 14: The process monitoring result using the PCA method for low stock line in a blast furnace process.

23



Z(t)

8

6

Z

4

2

0

-2

-4 0

5

10

15

20

25

30

35

40

t

Figure 15: The change point detection result using the graph method for slip in a blast furnace process.

Z(t)

8

6

4

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 24 of 32

2

0

-2

-4 0

5

10

15

20

25

30

35

40

t

Figure 16: The change point detection result using the weight graph method for slip in a blast furnace process.

24


Page 25 of 32

T2 statistic

T2 statistic

500


400 300 200 100 0 5

10

15

1000

SPE statistic

20

25

30

35

40

sampling point SPE statistic testing data control limit

500

0 5

10

15

20

25

30

35

40

sampling point

Figure 17: The process monitoring result using the PCA method for slip in a blast furnace process.

Z(t)

7 6 5 4 3

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


2 1 0 -1 -2 0

5

10

15

20

25

30

35

40

t

Figure 18: The change point detection result using the graph method for cooling in a blast furnace process.

25



Z(t)

7 6 5 4

Z

3 2 1 0 -1 -2 0

5

10

15

20

25

30

35

40

t

Figure 19: The change point detection result using the weight graph method for cooling in a blast furnace process.

T2 statistic

T2 statistic

40


30 20 10 0 0

5

10

15

20

25

30

35

40


1000

SPE statistic

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 26 of 32


500

0 0

5

10

15

20

25

30

35

40

sampling point

Figure 20: The process monitoring result using the PCA method for cooling in a blast furnace process.

26


Page 27 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


From Fig. 12-Fig. 20 and Table 4, it can be concluded that the weigh graph method could detect the change points exactly, while the graph method could only detect the slip fault. Compared with the weight graph method, the PCA method could not detect the change point for the low stock line fault. The blast furnace process is complicated. The sensor errors, environmental disturbances and operation change would lead to the outliers existing throughout whole process. The PCA method is sensitive to the outliers, while the weight graph method could reduce the influences of outliers, which is suitable for process monitoring in a blast furnace process. The weight graph method considers the time relations of observations, which are important information for process monitoring. The faults are introduced at the 21st observations, and under some conditions, the weight graph method could exactly detect the change point at the 21st sample point, which means the weight graph method is powerful, especially for a blast furnace process. For primarily large-scale industrial processes, if the abnormal condition can be detected as soon as possible, then the safety will increase significantly. The weight graph method utilizes the time relations between observations for accelerating the detection time, which is important for industrial processes.

Conclusion In this paper, an unsupervised weight graph process monitoring method is developed. The weights of the edges in the minimum spanning tree are calculated based on the time intervals and the Euclidean distances between observations, which consider the important information contained in the time relations. The scan statistic based on the graph change point detection method has already been used in the authorship debate and friendship network areas. This is the first time this approach has been attempted for use in process monitoring. In this paper, the steps of using the scan statistic based on the weight graph method to detect the change point are listed. The power of the weight graph method is tested in a numerical example, the TE process and a blast furnace process. It can be concluded that the weight graph method that considers the time relations between observations is

27



Page 28 of 32

more powerful than the graph-based method. However, the weight graph method is not a real-time process monitoring method, which needs some testing observations to compare the distributions of data collected before and after the change point candidate. In future research, this method will be investigated for use as a real-time method for process monitoring. For example, the k-nearest neighbor method or clustering methods could be used for calculating the weights in the minimum spanning tree for online change point detection based on the weight graph method.

Acknowledgement This work was supported by the National Natural Science Foundation of China (Grant No. 61290321) and the National Natural Science Foundation of China (Grant No. 61333007).

References (1) Qin, S. J. Survey on data-driven industrial process monitoring and diagnosis. Annu. Rev. Control 2012, 36, 220-234. (2) Ge, Z. Q.; Song, Z. H.; Gao, F. R. Review of recent research on data-based process monitoring. Ind. Eng. Chem. Res. 2013, 52, 3543-3562. (3) Ge, Z. Q.; Song, Z. H. Process monitoring based on independent component analysis-principal component analysis (ICA-PCA) and similarity factors. Ind. Eng. Chem. Res. 2007, 46, 20542063. (4) An, R. Q.; Yang, C. J.; Zhou, Z.; Wang, L.; Pan, Y. J. Comparison of different optimization methods with support vector machine for blast furnace multi-fault classification. 9th IFAC Symposium on Fault Detection, Supervision and Safety for Technical Processes, Paris, France, 2015; pp 1204-1209.10.1016/j.ifacol.2015.09.690 (5) Xiong, L. H.; Guo, S. L. Trend test and change-point detection for the annual discharge series of the Yangtze River at the Yichang hydrological station. Hydrol. Sci. J. 2004, 49, 99-112. 28


Page 29 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(6) Tsirigos, A.; Rigoutsos, I. A new computational method for the detection of horizontal gene transfer events. Nucleic Acids Res. 2005, 33, 922-933. (7) Liu, Y.; Hseuh, B. F.; Gao, Z. L.; Yuan, Y. Flooding Prognosis in Packed Columns by Assessing Degree of Steadiness (DOS) of Process Variable Trajectory. Ind. Eng. Chem. Res. 2016, 55, 10744-10750. (8) Radke, R. J.; Andra, S.; Al-Kofahi, O.; Roysam, B. Image change detection algorithms: a systematic survey. IEEE Trans. Image Process. 2005, 14, 294-307. (9) Tartakovsky, A. G.; Rozovskii, B. L.; Kim, R. B. H. A novel approach to detection of intrusions in computer networks via adaptive sequential and batch-sequential change-point detection methods. IEEE Trans. Signal Process. 2006, 54, 3372-3382. (10) Watkins, L. P.; Yang, H. Detection of intensity change points in time-resolved singlemolecule measurements. J. Phys. Chem. B. 2005, 109, 617-628. (11) Aminikhanghahi, S.; Cook, D. J. A survey of methods for time series change point detection. Knowl. Inf. Syst. 2017, 51, 339-367. (12) Pan, Y. J.; Yang, C. J.; An, R. Q.; Sun, Y. X. Robust principal component pursuit for fault detection in a blast furnace process. Ind. Eng. Chem. Res. 2018, 57, 283-291. (13) Zhou, M. Y.; Zhou, Q.; Geng, W. A new nonparametric control chart for monitoring variability. Qual. Reliab. Eng. Int. 2016, 32, 2471-2479. (14) Bickson, D.; Baron, D.; Ihler, A.; Avissar, H.; Dolev, D. Fault identification via nonparametric belief propagation. IEEE Trans. Signal Process. 2011, 59, 2602-2613. (15) Desobry, F.; Davy, M.; Doncarli, C. An online kernel change detection algorithm. IEEE Trans. Signal Process. 2005, 53, 2961-2974. (16) Lung-Yut-Fong, A.; Levy-Leduc, C.; Cappe, O. Homogeneity and change-point detection tests for multivariate data using rank statistics. https://arxiv.org/pdf/1107.1971v3, 2011. 29



(17) Harchaoui, Z.; Moulines, E.; Bach, F. R. Kernel change-point analysis. Adv. neural Inf. process. Syst. 2009, 609-616. (18) Rosenbaum, P. R. An exact distribution-free test comparing two multivariate distributions based on adjacency. J. R. Stat. Soc. 2005, 67, 515-530. (19) Lu, G. L.; Zhou, Y. Q.; Lu, C. H.; Li, X. Y. A novel framework of change-point detection for machine monitoring. Mech. Syst. Signal Process. 2017, 83, 533-548. (20) Adams, R. P.;

MacKay, D. J. C. Bayesian online changepoint detection. http-

s://arxiv.org/abs/0710.3742, 2007. (21) Chen, H.; Zhang, N. Graph-based change-point detection. Annu. Stat. 2015, 43, 139-176. (22) Musulin, E. Spectral Graph Analysis for Process Monitoring. Ind. Eng. Chem. Res. 2014, 53,10404-10416. (23) Musulin, E. Process Disturbances Detection via Spectral Graph Analysis. Comput. Aided Chem. Eng. 2014, 33, 1885-1890. (24) Friedman, J. H.; Rafsky, L. C. Multivariate generalizations of the Wald-Wolfowitz and Smirnov two-sample tests. Annu. Stat. 1979, 697-717. (25) Rosenbaum, P. R. An exact distribution-free test comparing two multivariate distributions based on adjacency. J. R. Stat. Soc. 2005, 67, 515-530. (26) Henze, N. A multivariate two-sample test based on the number of nearest neighbor type coincidences. Annu. Stat. 1988, 772-783. (27) Ye, K. Q. Indicator function and its application in two-level factorial designs. Annu. Stat. 2003, 984-994. (28) Zhou, Z.; Wen, C. L.; Yang, C. J. Fault isolation based on k-nearest neighbor rule for industrial processes. IEEE Trans. Ind. Electron. 2016, 63, 2578-2586. 30


Page 30 of 32

Page 31 of 32 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(29) Pan, Y. J.; Yang, C. J.; An, R. Q.; Sun, Y. X. Fault detection with improved principal component pursuit method. Chemom. Intell. Lab. Syst. 2016, 157, 111-119. (30) Yin, S.; Zhu, X. P.; Kaynak, O. Improved PLS focused on key-performance-indicator-related fault diagnosis. IEEE Trans. Ind. Electron. 2015, 62, 1651-1658. (31) Wang, L.; Yang, C. J.; Sun, Y. X. Multi-mode Process Monitoring Approach Based on Moving Window Hidden Markov Model. Ind. Eng. Chem. Res. 2018, 57, 292-301. (32) Yin, S.; Ding, S. X.; Haghani, A.; Hao, H. Y.; Zhang, P. A comparison study of basic datadriven fault diagnosis and process monitoring methods on the benchmark Tennessee Eastman process. J. Process Control 2012, 22, 1567-1581. (33) Chiang, L. H.; Russell, E. L.; Braatz, R. D. Fault detection and diagnosis in industrial systems; Springer: London, 2000. (34) Downs, J. J.; Vogel, E. F. A plant-wide industrial process control problem. Comput. Chem. Eng. 1993, 17, 245-255. (35) Jian, L.; Gao, C. H.; Xia, Z. H. Constructing multiple kernel learning framework for blast furnace automation. IEEE Trans. Autom. Sci. Eng. 2012, 9, 763-777.

TOC graphic

31



Z(t)

7

Z(t)

7

6

6

5

5

4

4

3 3 2

Z

Z

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 32 of 32

2 1 1

0

0

-1

-1

-2 -3

-2 0

5

10

15

20

25

30

35

40

0

5

10

15

20

t

t

(a)

(b)

32


25

30

35

40

Unsupervised change point detection using a weight graph method for

Recommend Documents