Spectral Graph Analysis for Process Monitoring - Industrial

Article pubs.acs.org/IECR

Spectral Graph Analysis for Process Monitoring Estanislao Musulin* Centro Internacional Franco Argentino de Ciencias de la Información y de Sistemas, Ocampo y Esmeralda, S2000BTP Rosario, Argentina ABSTRACT: Process monitoring is a fundamental task to support operator decisions under abnormal situations. In this article, spectral graph analysis monitoring (SGAM) is introduced. The approach is based on the spectral graph analysis theory. First, a weighted graph representation of process measurements is developed. Second, the process behavior is parametrized by means of graph spectral features, in particular, the algebraic connectivity and the spectral energy. The developed methodology is illustrated in autocorrelated and nonlinear synthetic cases and applied to the well-known Tennessee Eastman process benchmark with promising results.

1. INTRODUCTION In complex modern processes, the interest in monitoring systems has increased following the demand for better management of plants because of more restrictive economic and environmental conditions. In the past several years, many process monitoring methods have been developed to detect process disturbances in a timely fashion.1−4 In particular, data-driven techniques have attracted the greatest interest because of the high availability of online data.5−9 Many process monitoring applications rely on dimensionality-reduction techniques: Process measurements are projected onto a low-dimensional space where most of the normal data variability is contained. Therefore, the monitoring performance is directly influenced by the quality of the projection model. The most widespread of these techniques is principal component analysis (PCA)10,11 and its extensions.3,12−15 PCA examines the measurement covariance and selects a new orthogonal base of reduced dimensionality that explains most of the data variance. Thus, PCA can be considered as a globalitybased data projection method that does not consider the local structure of data. That is, the data neighborhood can be altered after projection. Recently, several methods of dimensionality reduction were proposed in the pattern recognition area16−20 and were reviewed and systematically compared in ref 21. In contrast to PCA, those methods, known as manifold learning, are based on the local structure of the data. Moreover, most of these approaches are nonlinear and computationally expensive. Of these methods, the locality preserving projections (LPP) method19 is of particular interest; LPP is a linear technique that defines the neighborhood relationship between data samples and finds the projection that preserves the intrinsic geometry structure of the data set. However, the outer shape of the data set can be modified, and therefore, the global data structure can be distorted. Recently, global−local structure analysis (GLSA)22 and local and global PCA (LGPCA)23 were proposed as compromise solutions. They construct dual objective functions aiming to preserve both the local and global structures of the data set. In this work, an alternative process monitoring technique based on spectral graph analysis (SGA) is proposed. As in manifold learning techniques, a similarity graph is constructed based on the normal data set; however, instead of using this information © 2014 American Chemical Society

to project data into a dimensionality-reduced space, the monitoring is performed by directly studying the spectral characteristics of the graph. Before going into further detail, we first introduce some general graph concepts. Many real-world situations can conveniently be represented by a diagram consisting of a set of vertices joined by a set of edges. For example, the vertices could be people, communication centers, or process variables, and the corresponding edges could represent friendship links, communications links, and degrees of interaction, respectively. A mathematical abstraction of these situations leads to the concept of graph.24 A graph can be represented by its adjacency matrix A; the analysis of A based on eigenvalues and eigenvectors is called the theory of graph spectra.25,26 This theory attempts to utilize linear algebra, including the well-developed theory of matrices, for the purposes of graph theory and its applications.27 Spectral graph theory has acquired great relevance in the past decade, particularly in the field of computer science. An excellent survey of these applications can be found in ref 28. Other areas of research such as process engineering have very few applications of this spectral approach. Nevertheless, there are some works related to graph theory, such as ref 29, in which new methods for the analysis of complex processes have been suggested. The authors formulated a flexible framework to help designers in comprehending a process by representing structural and functional relationships. Yang and co-workers30 suggested a fusion of information from process data and process connectivity. Signed directed graphs are used to capture the process topology and connectivity, thus depicting the causal relationships between process variables. Gutierrez-Perez and co-workers31 introduced a methodology based on spectral measurements of graphs to establish the relative importance of regions in water supply networks. These regions were then analyzed using a flexible method of semisupervised clustering. Finally, Zumoffen and Musulin32 built a graph using the process static sensitivity matrix Received: Revised: Accepted: Published: 10404

November 22, 2013 April 28, 2014 May 28, 2014 May 28, 2014 dx.doi.org/10.1021/ie403966v | Ind. Eng. Chem. Res. 2014, 53, 10404−10416

Industrial & Engineering Chemistry Research

Article

Figure 1. Swiss-roll data set representation, with abnormalities depicted as square points. (a,e,i,m) Original data, (b,f,j,n) PCA and (c,g,k,o) LPP projections, and (d,h,l,p) SGAM graphs and their spectral features a(G) and (G). In the SGAM graphs, only the five strongest connection are depicted for each vertex.

undirected, it is assumed that each edge carries a nonzero symmetric weight (wij = wji). The elements of the weighted adjacency matrix Aw of the weighted graph G are defined as26

and showed the utility of spectral analysis in optimally selecting variables to be controlled. In this work, SGAM is introduced as an alternative method for process disturbance detection. It is shown that processes can be monitored by analyzing the spectral features of a properly defined weighted graph. This article is organized as follows: Section 2 provides some insight into spectral graph theory with particular reference to some results that motivated this work. In section 3, the new monitoring methodology is presented. Experimental results are reported in section 4. Finally, in section 5, conclusions are given, and some future works are proposed.

⎧ wij ≠ 0 if i = j aij = ⎨ ⎩0 otherwise ⎪

⎪

(1)

According to this definition, Aw is a real symmetric matrix with zero diagonal. 2.1. Graph Energy. The spectrum (eigenvalues of Aw) is a graph invariant. Let λ1, ..., λn be the eigenvalues of Aw. Then, the energy of G is defined as26,33 n

(G) =

2. SPECTRAL GRAPH ANALYSIS Let G = (V,w) be a finite undirected weighted graph of order n without loops or multiple edges, with vertices labeled 1, 2, ..., n. If vertices i and j are joined by an edge, it is said that i and j are adjacent, written as i ∼ j. Because the graph is weighted and

∑ |λ i | i=1

(2)

This quantity is well-known in chemical applications, because, in some cases, the energy defined in this way corresponds to the energy of a molecule.27,34 However, the graph invariant (G) can 10405

dx.doi.org/10.1021/ie403966v | Ind. Eng. Chem. Res. 2014, 53, 10404−10416

Article

be considered for any graph independently of the chemical context; recently, much work on graph energy also appeared in the pure mathematics literature.26,35 This new perspective provides new generally valid mathematical properties for (G). In this article, the result obtained by Gutman and Shao26 is of particular importance. Theorem 1. Let G1 = (G,w1) and G2 = (G,w2) be two weighted graphs on a graph G. Suppose that there is an edge e0 = ij of G satisfying the following conditions: • w1(e) = w2(e) for all e ∈  (G) with e ≠ e0. • w2(e0) > w1(e0) > 0. • (G1 − e0) <  (G1), where G1 − e0 is the weighted graph obtained from G1 by deleting the edge e0. Then,  (G2) > (G1). This theorem says that, if removing some edge e0 from a weighted graph G decreases the energy, then increasing the positive weight on that edge will increase the energy. 2.2. Algebraic Connectivity. Given Aw, the discrete Laplacian matrix L is defined as L = Aw − D, where D is the diagonal matrix of vertex degrees D = diag(ai), and [D]ii = ai = δ∑nj=1aij. L is a positive semidefinite matrix, so all its eigenvalues are non-negative. The second smallest eigenvalue of L is usually called the algebraic connectivity of G27 and is denoted by a(G) . Although a simple eigenvalue is not sufficient to describe the graph structure, the algebraic connectivity has some important relationships with several graph metrics. As a consequence, it has been used in separation, metric, and isoperimetric problems.27 Our interest in a(G) relies in the inequalities described in the next two theorems. Theorem 2.36 If G is a connected graph on n vertices, then

2a. Remove the first column of Aw corresponding to the older point xm(1). 2b. Add xm(n+1) to the graph by calculating the weighted distance to the other graph vertices, and update Aw. 2c. Recalculate  (G) and a(G). Continue the procedure from step 2 until the complete normal data set has been processed.

⎡ 4 ⎤ diam(G) ≥ ⎢ ⎥ ⎢ na(G) ⎥

(3)

where diam(G) is the graph diameter, defined as the maximum distance between any two vertices of G. Theorem 3.37 For any graph G on n vertices (n ≥ 2) i(G) ≥ a(G)/2

(4)

where i(G) is the isoperimetric number or conductance, defined as27 i(G) =

min

0 4σ will be almost disconnected (see eq 7). The maximum response time of this system was about 15 samples. Therefore, n was set to 80, which was sufficient to have a graph that comprised the process dynamics and most of the common cause variability. Panels a and b of Figure 3 show the resulting control charts for PCA and SGAM, respectively, under

normal conditions, and Figure 3c is a graphical representation of the adjacency matrix at sample 90 (i.e., it includes measurements from sample 11 to sample 90), where darker shading indicates a weaker relation (similarity) between points. To examine the monitoring performance, five disturbances of different types were introduced in the input variable w1 at sample 101 (see Table 3). 10412



Article

Figure 11. Tennessee Eastman process. PCA, LPP, and SGAM control charts for the normal operating conditions (NOC) and disturbances IDV(11), IDV(12), IDV(14), and IDV(15).

Figure 4 shows the monitoring results for step disturbance IDV(1). PCA detected the fault at time 107 (three consecutive T2 samples outside the 99% limit). Spectral detection was obtained at the same time, but it was much clearer, showing a decrement in the graph energy and in the algebraic connectivity. Note that, after the step, the spectral features returned to normal values, because SGAM monitors local structures of data. IDV(2) (see Figure 5) was a small drift disturbance, which was detected by the T2 statistic intermittently at samples 138, 178, and 185 and more clearly after sample 197. Using SGAM, the disturbance was distinctly detected in the aG-control chart from sample 137. Because the system does not return to a steady state, the spectral features does not completely return to the normality limits. IDV(3) and IDV(4) (see Figures 6 and 7) correspond to increments and decrements, respectively, in the noise levels. IDV(3) was detected by PCA at sample 126 and again and with more clarity at sample 190. Detection using the aG-control chart was faster and sharper at sample 105. Note that, because the noise level was kept higher than normal, the SGAM charts did not

return to their normal values. Regarding IDV(4), although the reduction of noise caused a reduction in the values of its statistics (see Figure 7), this disturbance could not be detected by PCA. On the other hand, IDV(4) caused an increment in the graph energy (the graph vertices were more concentrated) that could be observed in the EG-control chart from sample 132. The differences between the adjacency matrices in Figures 6c and 7c illustrate the changes in the connectivity of the graph vertices upon changes in the noise levels. As shown in Figure 4, after detection, if the system remained at an operating point with a similar common cause variance, the spectral features returned to normal values. This gives the system some adaptive properties that are useful in the detection of consecutive faults such as IDV(5) (see Figure 8). Here, a positive step introduced at sample 101 was detected by PCA and SGAM statistics at sample 107. After some time (related to the time window size n), the SGAM statistics returned to normal values. In this situation, a new step was introduced at sample 201 and was clearly detected by SGAM charts (at sample 207). On the other hand, the PCA statistics remained outside the limits 10413



Article

Figure 12. Tennessee Eastman process. PCA, LPP, and SGAM control charts for IDV(19), IDV(20), and IDV(21).

because of the detection of the first step at sample 107, and therefore, the second disturbance was overlooked. 4.2. Tennessee Eastman Benchmark Process. In this case study, the proposed approach is applied to the Tennessee Eastman (TE) process.39 The process has five main units (see Figure 9): an exothermic two-phase reactor, a product condenser, a flash separator, a reboiler stripper, and a recycle compressor. The TE has 42 measured variables and 12 manipulated variables. Several control strategies have been applied to this benchmark. Because monitoring results change depending on the selected measurements, the control strategy, and the sampled times, for the sake of standardization, in this work, the data set published in ref 40 was used. This data set was obtained using the plantwide control structure recommended in ref 41. It consists of 33 measured process variables collected with a 3-min sample time. All of the disturbances proposed by Downs and Vogel39 were introduced at sample 161. Each data set contained 960 samples. See Table 4 for detailed disturbance descriptions. To better illustrate the obtained results, a comparison with PCA and LPP monitoring is provided.

Using the autoscaled normal data set, SGAM was applied with parameters σ = 4 (because eq 10 returned a value of 4.08) and n = 100, which is equivalent to 5-h time windows. A PCA model was built using the same normal data set, and nine principal components were selected. LPP was aplied using σ = 4 as in SGAM and selecting nine projected dimensions, as in PCA. Fault alarms were considered to be triggered when three consecutive samples exceeded the confidence limit. To provide a fair comparison, confidence limits for all statistics were set to avoid false alarms during normal operating conditions. As explained before, SGAM is adaptive, and if the process stabilizes, its indicators return to normal values after detection. As a consequence, the direct application of reliability (defined as the percentage of points outside detection limits) measures cannot be used to compare the two techniques. Additionally, detection times are considered to be more important than detection reliability because, once detected, a fault alarm can easily be held until the operator acknowledges the abnormal state. Therefore, detection time was chosen as the criterion for evaluating and 10414



Article

comparing monitoring performance. Table 5 shows the results obtained for PCA and SGAM. It can be seen that most disturbances were large in magnitude and were easily detected by PCA, LPP, and SGAM at similar detection times [IDV(1), IDV(2), IDV(5)−IDV(7), IDV(12)]. IDV(8), IDV(17), and IDV(18) were detected some samples later, but also with similar detection times for any technique. With the exception of IDV(9), which was particlary well detected by LPP (see Figure 10h), and IDV(13) (see Figure 11d), which was detected slightly faster by PCA, disturbances were better detected by SGAM. Even in the case of similar detection times, the SGAM indicators tended to be more definitive. To illustrate this situation, PCA, LPP, and SGAM control charts are presented for some of the most interesting disturbances. Figure 10a−c shows the IDV(3) PCA, LPP, and SGAM control charts. The PCA and LPP statistics could not detect this disturbance with the selected set of sensors. IDV(3) was detected only by SGAM, in the form of small changes in the aG-control chart. IDV(4) (see Figure 10d,f) is a step fault introduced in the reactor cooling water inlet temperature. Note that the fault was detected by both PCA and SGAM. In particular, the aG-control chart showed a decrement in the algebraic connectivity that returned to normal values when the control stabilized the process. LPP detected this fault 60 samples later (see Figure 10e). Random variation disturbances, such as IDV(10) and IDV(11), were found to be detected faster and with more clarity in SGAM control charts, as is evident in Figures 10j−l and 11a−c. IDV(13) presents the only case in which the fault was detected first using PCA (for just eight samples). However, from Figure 11d−f, it can be noted that the SGAM detection was much clearer, as an abrupt reduction of the algebraic connectivity. IDV(15) (see Figure 11g−i) produced a slow drift that was detected by LPP at sample 628, by PCA at sample 758, and by SGAM 15 samples earlier. IDV(19) was also a random variation that was easily detected by SGA but not detected by PCA (see Figure 12a,c). This disturbance was also not detected by LPP (see Figure 12b). IDV(20) was detected by all techniques, but the SGAM charts are clearer (see Figure 12b,e,f). Finally, IDV(21) was an unknown fault that primarily caused a reduction of process variability and a posterior drift in process variables. The reduction of process variability was detected by SGAM as an increment in the graph energy (see Figure 12g,i) at sample 266, long before the occurrence of the drift in process variables. PCA detects this disturbance only after the drift at sample 639. SGAM is thus the only technique that was able to detect all of the presented faults. In addition, it provided good all-around results, as shown by the average detection time (see Table 5).

fault indicator returns to the normal state) until the operator acknowledges the fault. As a future work, the intrinsic adaptive capabilities of SGAM will be further investigated. Also, SGAM will be extended, and its potentialities as a fault diagnosis tool will be explored.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS Financial support from CONICET, ANPCyT, and FCEIA-UNR is fully acknowledged. I also thank Dr. David Zumoffen for the many insightful discussions during the development of this work.

■

NOMENCLATURE A = graph adjacency matrix Aw = graph weighted adjacency matrix a = adjacency matrix element a(G) = graph algebraic connectivity D = diagonal matrix of vertex degrees e = graph edge (G) = graph energy G = graph i(G) = graph isoperimetric number K = number of time windows in a data set k = sample time L = discrete Laplacian matrix m = number of variables n = number of graph vertices RN = normal data set radius TW(k) = moving time window V = graph vertex w = adjacency matrix weights X = measurement data set xm = measurement vector

Greek Letter

λ = adjacency matrix eigenvalue

Subscripts

i, j = graph vertices Acronyms

5. CONCLUSIONS AND FUTURE WORK In this work, spectral graph analysis monitoring (SGAM) has been presented. SGAM is a new process monitoring technique that is not based on a reduced space model. SGAM was illustrated in autocorrelated and nonlinear synthetic cases and applied to the well-known Tennessee Eastman process benchmark. Several types of process disturbances were evaluated, including steps, drifts, and random variations. The technique shows promising results to identify abnormal events. It can be used as an alternative or complement to PCA, LPP, and other online statistical monitoring approaches. Because statistics sometimes adapt to the new faulty state, a simple hold-on alarm management scheme would improve the obtained results. In such systems, alarms stay active (even if the

■

GLPCA = global local structure analysis LGPCA = local and global PCA LPP = locality preserving projection NOC = normal operating conditions PC = principal component PCA = principal component analysis SGAM = spectral graph analysis monitoring

REFERENCES

(1) Venkatasubramanian, V.; Rengaswamy, R.; Yin, K.; Kavuri, S. N. Comput. Chem. Eng. 2003, 27, 293−311. (2) Venkatasubramanian, V.; Rengaswamy, R.; Kavuri, S. N. Comput. Chem. Eng. 2003, 27, 313−326. (3) Venkatasubramanian, V.; Rengaswamy, R.; Kavuri, S. N.; Yin, K. Comput. Chem. Eng. 2003, 27, 327−346. (4) Musulin, E.; Roda, F.; Basualdo, M. Comput. Chem. Eng. 2013, 59, 164−177. (5) Kano, M.; Tanaka, S.; Hasebe, S.; Hashimoto, I.; Ohno, H. AIChE J. 2003, 49, 969−976. 10415



Article

(6) Ge, Z.; Song, Z. Ind. Eng. Chem. Res. 2007, 46, 2054−2063. (7) Qin, S. J. Annu. Rev. Control 2012, 36, 220−234. (8) Xie, X.; Shi, H. Ind. Eng. Chem. Res. 2012, 51, 5497−5505. (9) Tong, C.; Yan, X. Chemom. Intell. Lab. Syst. 2014, 130, 20−28. (10) Jackson, J. A User’s Guide to Principal Components; John Wiley: New York, 1991. (11) Nomikos, P.; MacGregor, J. Technometrics 1995, 37, 41−59. (12) Ku, W.; Storer, R. H.; Georgakis, C. Chemom. Intell. Lab. Syst. 1995, 30, 179−196. (13) Musulin, E.; Yelamos, I.; Puigjaner, L. Ind. Eng. Chem. Res. 2006, 45, 1739−1750. (14) Ge, Z.; Yang, C.; Song, Z. Chem. Eng. Sci. 2009, 64, 2245−2255. (15) Bakshi, B. AIChE J. 1998, 44, 1596−1610. (16) Tenenbaum, J. B.; De Silva, V.; Langford, J. C. Science 2000, 290, 2319−2323. (17) Roweis, S. T.; Saul, L. K. Science 2000, 290, 2323−2326. (18) Belkin, M.; Niyogi, P. Neural Comput. 2003, 15, 1373−1396. (19) He, X.; Niyogi, P. Locality preserving projections. In Proceedings of Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, 2004; Vol. 16, pp 153−160. (20) von Luxburg, U. Stat. Comput. 2007, 17, 395−416. (21) van der Maaten, L. J. P.; Postma, E. O.; van den Herik, H. J. J. Mach. Learn. Res. 2009, 10, 1−41. (22) Zhang, M.; Ge, Z.; Song, Z.; Fu, R. Ind. Eng. Chem. Res. 2011, 50, 6837−6848. (23) Yu, J. J. Process Control 2012, 22, 1358−1373. (24) Bondy, J. A.; Murty, U. S. R. Graph Theory with Applications, 5th ed.; Elsevier Science: Amsterdam, 1985. (25) Nikiforov, V. J. Math. Anal. Appl. 2007, 326, 1472−1475. (26) Gutman, I.; Shao, J. Linear Algebra Its Appl. 2011, 435, 2425− 2431. (27) Cvetković, D. M.; Rowlinson, P.; Simić, S. An Introduction to the Theory of Graph Spectra; Cambridge University Press: Cambridge, U.K., 2010. (28) Cvetković, D.; Simić, S. Linear Algebra Its Appl. 2011, 434, 1545− 1562. (29) Castaño Arranz, M.; Birk, W. J. Process Control 2012, 22, 280−295. (30) Yang, F.; Shah, S.; Xiao, D. Int. J. Appl. Math. Comput. Sci. 2012, 22, 41−53. (31) Gutierrez-Perez, J.; Herrera, M.; Perez-Garcia, R.; RamosMartinez, E. Math. Comput. Modell. 2013, 57, 1853−1859. (32) Zumoffen, D.; Musulin, E. Comput. Chem. Eng. 2013, 56, 80−88. (33) Gutman, I.; Li, X.; Zhang, J. Graph Energy. In Analysis of Complex Networks: From Biology to Linguistics; Dehmer, M., Emmert-Streib, F., Eds.; Wiley-VCH: Weinheim, Germany, 2009; Chapter 7, pp 145−174. (34) Gutman, I. J. Serb. Chem. Soc. 2005, 441−456. (35) Nikiforov, V. J. Math. Anal. Appl. 2007, 326, 1472−1475. (36) Mohar, B. Graphs Comb. 1991, 7, 53−64. (37) Mohar, B. J. Comb. Theory B 1989, 47, 274−291. (38) Trefethen, L. N.; Bau, D., III. Numerical Linear Algebra; Society for Industrial and Applied Mathematics (SIAM): Philadelphia, PA, 1997. (39) Downs, J.; Vogel, E. Comput. Chem. Eng. 1993, 17, 245−255. (40) Braatz, R. D. Tennessee Eastman Problem Simulation Data; available at http://web.mit.edu/braatzgroup/links.html (accessed Jun 2013). (41) Lyman, P.; Georgakis, C. Comput. Chem. Eng. 1995, 19, 321−331.

10416


Spectral Graph Analysis for Process Monitoring - Industrial

Recommend Documents