Recursive Mixture Factor Analyzer for Monitoring Multimode Time

Subscriber access provided by UNIV OF CALIFORNIA SAN DIEGO LIBRARIES

Article

Recursive mixture factor analyzer for monitoring multimode time-variant industrial processes Jinlin Zhu, Zhiqiang Ge, and Zhihuan Song Ind. Eng. Chem. Res., Just Accepted Manuscript • DOI: 10.1021/acs.iecr.5b03446 • Publication Date (Web): 06 Apr 2016 Downloaded from http://pubs.acs.org on April 10, 2016

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a free service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are accessible to all readers and citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

Industrial & Engineering Chemistry Research is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Industrial & Engineering Chemistry Research

Recursive mixture factor analyzer for monitoring multimode time-variant industrial processes Jinlin Zhu, Zhiqiang Ge∗, Zhihuan Song

State Key Laboratory of Industrial Control Technology, Institute of Industrial Process Control, Department of Control Science and Engineering, Zhejiang University, Hangzhou 310027, Zhejiang, P. R. China

Abstract A critical issue for real-time monitoring of industrial processes with data-based methods is how to adjust the constructed model during monitoring procedure so as to adapt to the change of the process. Traditional latent variable models like factor analyzer are static models and are simply based on the single Gaussian assumption. Therefore, when one comes to the multimode and time-variant process conditions, conventional strategies become cumbersome. In this work, a recursive mixture factor analyzer is proposed for multimode time-variant process modeling and monitoring. The developed model with Bayesian mechanism can automatically select and update the Gaussian components during modeling. Furthermore, a corresponding monitoring mechanism is proposed so that the recursive model can also elegantly adjust parameters according to the newly incoming data information and effectively employ more components for new operating modes. Feasibility and efficiency of the proposed method are illustrated through two case studies.

Keywords: Recursive probabilistic latent variable model; multimode process monitoring; time-variant adaptation; expectation maximization (EM); Factor analysis.

∗

Corresponding author: Tel.:+86-87951442, E-mail address: [email protected](Ge Z.)

1

ACS Paragon Plus Environment


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 2 of 42

1. Introduction Nowadays, modern industrial process tends to become more and more complex due to the ever growing requirement of product quality

1-3

. To ensure the operating safety and product quality, process

monitoring is indispensible 4. Previous process monitoring techniques mainly resort to the first principle mechanisms and specific kinetic formulas are demanded in order to make an overall grasp of process units 5, 6

. However, such strategy largely depends on the accurate acquisition of process mechanisms of which

should be time-consuming and sometimes unattainable. In this case, data based methods need little process knowledge and thus become more and more popular over the past few decades

7-9

. A large amount of

multivariate statistical modeling methods have been developed and applied for monitoring applications of industrial processes 10, 11. Among various multivariate statistical process monitoring (MSPM) methods, principal component analysis (PCA), probabilistic PCA (PPCA) and factor analysis (FA) are widely used fundamental methods that have been widely accepted and focused 12. All these methods are constructed with the latent variable framework and the observations are combined and condensed into a lower projection space during the modeling phase. For process monitoring, the Hotelling’s T 2 and squared predicted error (SPE) terms are then employed to distinguish the abnormal events from normal scenarios. In addition, PPCA and FA makes a further improvement since the probabilistic modeling scheme also considers the potential noise uncertainty by including Gaussian variations for projecting observations

13

. The major difference from

PPCA and FA is that the former one is designed with isotropic variance while the latter one assumes different variances for different measurement variables. One common issue is that traditional latent variable models cannot characterize the multimode property of industrial process data. Multimode should be one of the most remarkable attributes since multiple operating conditions can lead to various statistical

2


Page 3 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


allocations of process observations 14, 15. A common idea is to build several local models so as to capture and analyze each local behavior. A large variety of such modified latent variable models have been reported, such as multiple independent component analysis–principal component analysis (ICA–PCA) localized Fisher discriminant analysis

17

, multi-subspace PCA

18

, multi-block PCA

19

16

,

, etc. Recently, a

mixture fashion of FA (mixture FA or MFA) has been developed to cope with the multimode issue

20

.

Essentially, MFA can be viewed as a class of finite mixture model with each local dimension reduction structure defined by a specified factor analyzer. Compared with other models, many desirable benefits can be found for MFA such as the efficient Bayesian learning and inference mechanism with EM algorithm and the ability to handle missing entries. However, as a static model with fixed parameters, traditional MFA model as well as the above modified strategies cannot adaptively determine local components for modeling; while for monitoring, MFA cannot be readily applicable for monitoring the time-variant process environment. In that case, the operators have to manually regulate the model structure and parameters during both modeling and monitoring phases. All of these could be inefficient and arduous, especially for the real-time monitoring procedure for time-variant systems

21, 22

. The time-variant characteristics may usually occur due to many

practical reasons such as changed modeling condition, catalyst deactivation or equipment aging

23, 24

.

Under these conditions, the false alarm in traditional latent variable models may be easily triggered and the entire plant may even have to unnecessarily shut down for safety reasons. In order to keep the model up-to-date, one can make the traditional latent variable models re-constructed and re-trained many times during the monitoring phase. Regardless of the effectiveness, such repeating procedures can be computationally disastrous when a large amount of data have been collected and recorded. Therefore, some studies resort to a class of recursive (online) methods and the

3



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

traditional latent variable models are further modified with adaptive mechanisms，in order to make the model adaptive to the data streams25. Li et al. proposed two recursive PCA based on rank-one modification and Lanczos tridiagonalization for process monitoring 26. Wang et al. proposed a moving window PCA was designed and the model was updated by including the fresh sample while excluding the oldest one 27. In the work of Jeng et al., the recursive PCA and moving window PCA are developed and compared for online monitoring 28. More recently, the approximate linear dependence was proposed to have each new sample checked before deciding whether to update the PCA model 29. However, most of the above recursive latent variable models are designed for single operating condition and cannot deal with multimodal situations. To deal with this issue, an ideal solution should be making multiple local models with recursive strategies. For example, an aggregation k-means method is used for mode clustering and then a mode-unfolding scheme based PCA is developed for each mode and adaptive mechanisms are finally incorporated for model updating 30. In a more recent work, the fuzzy c-means clustering is applied for mode clustering and then a recursive kernel PCA is proposed for monitoring and updating 31. However, no probabilistic mechanisms have been considered to present uncertainties from observation noise. Recently, work by Feital et al proposed a maximum likelihood PCA method for extracting multimode information into lower dimension space and followed by an efficient Gaussian mixture method for modeling various modes that can learn the empirical model recursively 32. Other studies have also focused on developing adaptive/dynamic Gaussian mixture models (GMM)

23, 33

. However, such models are constructed directly based on high-dimensional

data and cannot perform feature extraction or dimension reduction. Actually, with a K mixture of Gaussian components for D-dimension data, one has a total of KD ( D + 1) / 2 + KD + K underestimated parameters which is quadratic with respect to data dimension. While for MFA with d-dimensional latent space, only DdK + 2 KD + K parameters are implicated for estimation which is linear to D. Therefore, for large-scale

4


Page 4 of 42

Page 5 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


industrial process with large data dimension, GMM may often fail during modeling if no dimension reduction mechanisms have been incorporated. The motivation of this work is to deal with the multimode modeling as well as time-variant process monitoring issues with recursive based MFA method. The theoretical foundation is constructed on the recursive modeling framework proposed by Titterington 34. In this work, a general online EM algorithm is developed for recursively estimating parameters using incomplete data

34

. However, such an online EM

scheme along cannot effectively deal with the mixture component selection issue. In this case, a Bayesian treatment is designed and a Dirichlet prior is defined for implicit mixture components to punish the empirical model. As a result, the derived recursive MFA can conduct the model selection and parameter learning simultaneously. Specifically, in the first step, the online expectation maximization scheme is applied for recursively updating parameters in each local FA model. Then, the Bayesian method is used to search the maximum posteriori solution from model space and select the appropriate mixture components. For real-time monitoring, a detailed monitoring workflow is proposed and each sample is first evaluated by the posterior assignments, based on which one can decide which local latent model should be updated. While for a new operating mode, the manual instruction needs to be further combined in order to decide whether one should add a new local model or report the appearance of abnormal events. Therefore, the proposed recursive method can help to reduce the operator intervention and deal with both modeling and monitoring process data in an effective and efficient manner. The rest of this paper is organized as follows. Section 2 gives a brief revisit of MFA model and the monitoring method. Then the recursive MFA modeling and monitoring mechanisms are proposed in section 3. In section 4, simulations are conducted to verify the proposed method. Finally, conclusions are drawn in the last section.

5



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 6 of 42

2. Mixture Factor Analysis 2.1 MFA for process modeling Given a set of data records Y = {y n | y n ∈ R D }nN=1 , the FA tries to project the observations into a lower dimensional space of which the generative model can be given as 20:

y n = Wxn + µ + e

(1)

Here W ∈ R D × d is the projection matrix, x n ∈ R d is the latent projection and d < D . The latent variable is assumed to be Gaussian regulated: x ~ N ( 0, I d ) , Id is the d-dimensional identity matrix. µ is offset and

e ∈ N ( 0, Σ) is Gaussian noise with zero mean and diagonal covariance Σ = diag (σ12 σ 22 L σ D2 ) , and notation diag keeps only diagonal terms while set all off-diagonal elements with zeros. For multimode circumstances, FA can be no longer suitable and hence the mixture counterpart was developed. For data Y , the MFA seeks to approximate the overall distribution by incorporating a mixture of K local FA components 20: K

p( Y) = ∑ π k p( Y | θ k )

(2)

k =1

where π k is mixture weight for component k and

∑

K k =1

π k = 1 , θk = {Wk , µk , Σk } is the parameter set

of the k th component. Hence, from the maximum likelihood (ML) perspective, the log-likelihood function can be derived as: N

K

log L ( Θ )MFA = ∑∑ log p ( y n , k | Θ )

(3)

n =1 k =1

Then the expectation formulation can be given as: N

K

E ( L ) = ∑∑ log p ( y n , k | Θnew ) n =1 k =1

p ( x , k |y n , Θold )

(4)

Therefore, the batch EM algorithm can be obtained. The details can be found in Appendix A.

6


Page 7 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


2.2 MFA for online process monitoring Once the MFA has been constructed, it can be used for multimode process monitoring. Basically, two Bayesian monitoring schemes can be found: the Bayesian soft monitoring (BSM) method and the Bayesian inference-based probability (BIP) index 14, 35. The BSM method converts the traditional T 2 and SPE into probabilistic fashion so that Bayesian rule can be employed to decide normal or faulty events. The BIP index is a global statistic that integrates each Gaussian local component by the Mahalanobis distance metric and the corresponding probabilistic assignment so that one can make the global discriminant by a single index. In this work, we use the BIP index for statistical process monitoring. Specifically, for a new coming data sample y new , the BIP index can be defined as (Yu & Qin, 2008): K

BIPnew = ∑ p ( k | y new , Θ ) PLk ( y new )

(5)

k =1

k where PL ( y new ) is defined as the Mahalanobis distance based probability of y new with respect to

component k (denoted as Ck ), and can be computed by 35: PLk ( y new ) = Pr {MD ( y, Ck ) | y ∈ Ck ≤ MD ( y new , Ck ) | y new ∈ Ck }

where MD ( y new , Ck ) = ( y new − µ k ) ( Wk WkT + Σ k ) T

−1

( y new − µk ) is

(6)

the squared Mahalanobis distance. The

Mahalanobis distance from the center of Ck follows the χ D2 distribution with degree of freedom D. Hence the above Eq.(6) can be easily solved by a simple integration. Besides, we have 0 ≤ BIPnew ≤ 1 . The BIP upper control limit can be given a priori, such as 0.95 or 0.99.

3. Recursive MFA method 3.1 Recursive formulation development Practically, one can collect a large amount of normal samples from various operating conditions of industrial plants, all of which can be used for offline modeling. In this case, a fully numerical computation

7



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 8 of 42

for batch parameter learning can be very expensive. Different with the batch algorithm which directly uses the whole dataset and update parameters iteratively, the recursive method updates model parameters sequentially according to the corresponding updating innovation provided by each data sample. The main advantage of the recursive method for process modeling is that it can treat a large volume of data one after another during a single round and no iterations are required. Therefore, the real-time requirement as well as high computational efficiency can be more satisfactory. In Ref 34, Titterington proposed a general online EM algorithm, stochastic approximation procedures are used for parameter estimation. Inspired by this idea, we propose a modified extension of the general online EM for the recursive development of MFA. According to Titterington’s work, given current observation y t +1 and estimation θtk , θ tk = {Wkt , µ tk , Σ tk } , θ tk+1

can be recursively estimated by

(Titterington, 1984): θtk+1 = θtk +

−1 1 I c ( θtk )  S ( y t +1 , θtk )  t +1 

(7)

where S ( y t +1 , θ tk ) is the first derivative of logarithm based conditional probability density function with respect to the estimating parameter and I c ( θtk ) denotes the Fisher information matrix corresponding to the current complete observation. The definitions are given as follows: 36 S ( y t +1 , θtk ) = ∇ θt log p ( y t +1 | Θ )

(8)

k

(

Ic ( θtk ) = Eθ S ( yt +1 , θtk ) S ( yt +1 , θtk )

T

)

(9)

Based on such definition, we can realize the recursive formulation of MFA. Specifically, by applying Eq.(7) and with some mathematical arrangements, the particular recursive rules for M-step of MFA in the t+1th sampling time can be induced as (please see Appendix B for details) Wkt +1 = Wkt +

(

−1 1 (π kt ) qk ( y t +1 ) ( y t +1 − µtk ) xTt +1,k − Wkt x t +1,k xTt +1,k  x t +1,k xTt +1, k t +1

µtk+1 = µtk +

(

−1 1 π kt ) qk ( yt +1 ) yt +1 − µtk − Wkt xt +1, k ( t +1

8


)

)

−1

(10) (11)

Page 9 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Σ tk+1 = Σ tk +

(

−1 1 π kt ) qk ( y t +1 )  y t +1 − µ tk − Wkt x t +1, k (  t +1

)

2

(

− Σtk  x t +1, k xTt +1, k 

)

−1

(12)

Notice that other expectation terms can be obtained similarly by Eq.(21)-(23) (in Appendix A) and we omit here for simplicity. The above recursive MFA provides an automatic mechanism for parameter learning and model updating except for the selection of local latent space dimensionality as well as the mixture component number. Both of which could be viewed as the issue of model selection that should be carried out simultaneously with parameter estimation.

3.2 Model selection In order to enhance the model representation ability to empirical datasets, some irrelevant local parts should be elaborately pruned out. Intuitively, the pruning action can be implemented by tuning those potentially excessive mixing weights equal to zeros when no enough evidences support such components. In a fully Bayesian framework, this can be achieved by introducing a proper prior distribution over the mixing weight variable. During learning procedure, the prior can be viewed as punishing regulator and will push those excessive or irrelevant models to extinction in an auto-driven manner. In statistic literatures, the mixing weight is usually defined by Dirichlet prior 37. Therefore, assume a total of K mixture components are included, we have: K

p (θ ( M ) ) ∝ ∏π kck

(13)

k =1

where ck refers to the hyper-parameter and is usually set as 0.5NPk where NPk is the number of parameters in component k. In this sense, the log-likelihood can be rewritten by incorporating mixing prior as N

K

log L ( Θ ) MFA = ∑∑ log p ( y n , k | θ ( M ) ) + log p (θ ( M ) )

(14)

n =1 k =1

One can obtain the updating rule for mixing weight by introducing the Lagrange multiplier method. After

9



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 10 of 42

some arrangement, the analytical solution at sampling time t should be π kt =

1  t   ∑ q k ( y i ) − ck  t − Kck  i =1 

(15)

Hence, the recursive formula can be easily induced as:

π kt +1 = =

1  t +1   ∑ q k ( y i ) − ck  t − Kck + 1  i =1 

1  t   ∑ qk ( y i ) + qk ( y t +1 ) − ck  t − Kck + 1  i =1 

(16)

1 1  t  qk ( y t +1 )  ∑ qk ( y i ) − ck  + t − Kck + 1  i =1 t Kc −  k +1 1 π kt − qk ( y t +1 )  = π kt + t − Kck + 1  =

However, one can judge that the innovation weight

1 turns to zero as t becomes lager. t − Kck + 1

Therefore, the major drawback for such derivation in practical applications is that the updating ability can be vanishing when more samples are engaged. In order to amend this problem, we use the same strategy as in Ref 38 and the mixing weights for MFA are modified as (see Appendix C for details): −1 −1 −1 π kt +1 = π kt + (1 + t ) (1 − KcT ) qk ( y t +1 ) − π kt − cT (1 − KcT ) 





(17)

where cT = c / T , T is a sufficient large number so that KcT < 1 . Such modification ensures the equal treatment of data samples. Please notice the fact that mixing weights should be non-negative. Therefore, one could discard unnecessary mixtures by setting those negative weights to zeros during learning process. For efficient updating, one can further add a speed control factor like adaptive GMM 38. Also notice that one should first disrupt sample sequential orders before modeling so as to avoid updating issues of Gaussian components since no splitting operations have been involved. Apart from component selection, the implementation for recursive MFA should also consider the determination of latent dimensionality. The local component dimensionality should be determined to make the latent space more concise and meaningful for each local model. This topic has already been discussed in some literatures. For instance, a cumulative percent variance (CPV) method has been included for

10


Page 11 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


recursive PCA 28. Besides, one can also resort to the fully Bayesian treatment by introducing a specific prior for the latent space, and the corresponding technique is the so called automatic relevance determination

37

. In this work, for the sake of convenience, we assume that the latent dimensionality for

each FA model is given a priori and fixed along the entire procedure.

3.3 Recursive MFA for online process monitoring After the establishment of recursive MFA, it still cannot be directly applied for process monitoring. Different from the previous recursive offline modeling, the online monitoring and recursive parameter updating are more complex due to the various process variations including slow drifting, random variation, new mode appearance and step errors. Generally speaking, parameter configuration for industrial process may be slow drifted in some extent after a certain period of continuous operations. Random variations may happen due to the changed production circumstances while step errors happen due to some non-stationary changes of process variables. Notice that for industrial processes, especially those with control units, some variations including step errors can be well regulated and the object system still can stay steady without necessity of taking extra measures. In other cases, a particular series of random variations can also happen due to the changing sampling environment of hardware sensors. In such cases, the above variations should not be regarded as faulty events once the actual system has turned into normal operation status. Apart from the above mentioned variations, in multimode industrial monitoring conditions, there can also be some extra new operating modes that should be introduced halfway and need to be considered for modeling and monitoring. In order to make the monitoring as well as recursive updating procedure more efficient, some regulations have to be defined so as to prevent blind updating. First, we assume that a new mode should be

11



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

previously notified before monitoring as well as updating. Besides, some controlled steady states of variations (different from previous states with no variations) can be treated as new modes and such information should also be notified by operators. Based on that, the detailed monitoring and update mechanisms can be formulated. Specifically, when a new sample y t +1 comes, the entire programmed monitoring procedures should be performed as: 1) If the data sample is from a new defined operating mode, add a new local FA and the related parameters are set as: π new = α new , µ new = y t +1 , Σnew = I D , Wnew = random matrix , algorithm returns BIP = 0 ; otherwise, compute the likelihood p ( yt +1 | k, Θt ) ; 2) If any p ( yt +1 | k, Θt ) ≠ 0 , further compute posterior p ( k | yt +1 , Θt ) with respect to each known modes; otherwise set p ( k | yt +1 , Θt ) =0 for all modes; 3) If p ( k | yt +1 , Θt ) =0 is satisfied for all local components, the global index is set as BIP = 1 and algorithm returns BIP = 1 ; otherwise, compute the BIP index with posterior; 4) If BIP ≥ 0.99 , then algorithm returns BIP; otherwise, update the mean, variance and projection matrix * of the corresponding local model with k = argmaxk p ( k | yt +1 , Θt ) and return BIP;

5) If several BIPs stay beyond the control limit, trigger alarm; otherwise, keep the current status; 6) If the process with variation is further diagnosed as under control, shut down the alarm unit by defining a new mode and go to step 1. With this algorithm, one can see that the recursive MFA can effectively deal with the multimode time-varying process monitoring issues and only a few necessary operator interventions are required. Notice that the new mixing weight αnew is a user defined parameter. For example, an empirical assignment can be set as α new = arg min k π k . One should also normalize the mixing weights so that the total mixing pieces sum up to 1. Also pay attention that the mixing weight should not be tuned during monitoring since

12


Page 12 of 42

Page 13 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


it can be easily inferred that most of the mixing weights may be punished to zeros if the same operating sequence runs long enough.

4. Case studies 4.1 A numerical example In this case, an open-loop simple example is designed to demonstrate the feasibility and superiority of the proposed method. The multivariate linear system is given as:  x1  0.3723  x  0.4890  2   x3  = 0.9842     x4   0.7011  x5  0.3205

0.6815  e1  e   0.2954 2  t1    0.1793   + e3   t 2    0.4231 e4  e5   0.9706

(18)

One should notice that projection matrix A ∈ R5×2 is a random matrix, [t1 t2 ]

T

are Gaussian distributed

data sources, and noise term is also assumed as zero-mean white noise. In the meantime, we further assume the data source is generated from following three different modes: Mode 1: t1 : N (10,0.8) ; t2 : N (12,1.3) Mode 2: t1 : N ( 5,0.6) ; t2 : N ( 20,0.7) Mode 3: t1 : N (16,1.5) ; t2 : N (10,2.5) For convenience, we also assume that the defined modes are operated with equal chances during the simulation process. During offline modeling, a total of 1800 samples are generated as historical records with 600 items for each mode. The recursive MFA is employed and the corresponding modeling result is described in Figure 1 where the contours in left figure denotes the initial assignments (10 components assigned) and the right plot shows the final estimated contours of local components. Notice that for simplicity, the latent dimensionality for each mode is set as 2. Moreover, covariance is remained as a full

13



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 14 of 42

matrix since we want to keep information from all directions during inference. The estimated priors for each local model are all 0.33. One can easily see that the developed recursive MFA can accurately locate the multimodal data distributions. It should be mentioned here that if some variables in all dimensions are slightly overlapped together, our method still can be effective. For example, if mode 1 is defined as t1 : N (12.5, 0.8 ) ; t 2 : N (12,1.3)

Then Mode 1 and Mode 3 will be overlapped together in all dimensions as shown in Figure 2a. One can see that the recursive method can still behave well for such overlapping condition as shown in Figure 2b and Figure 2c. [Figure 1-2 about here] In order to validate the feasibility of the proposed monitoring strategy, four scenarios are elaborately designed including 3 normal cases and 1 faulty case. For validation and comparison, we compare our recursive MFA with traditional MFA. Case 1: Process slow drifting. The defined process is operating in mode 1 during 1-1000th sampling

time and then the process switches within mode 2 during 1001-2000th sampling time. However, some entries of the process matrix A have continuous slow drifting during 1301-1500th sampling period due to catalyst deactivation or equipment ageing. Let A( m, n) denotes the mth row and nth column of process matrix and the drifting rule is designed as:  A ( 2,1) = A ( 2,1) + ( t − 1500 ) × 0.000002   A ( 4, 2 ) = A ( 4, 2 ) − ( t − 1500 ) × 0.000001

(19)

Simulation data for x2 and x4 are shown in Figure 3. Monitoring results for recursive MFA and MFA are given in Figure 4a and 4b respectively. [Figure 3-4 about here] One can easily judge from Figure 4 that both methods can be suitable for monitoring multimode

14


Page 15 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


process data. Moreover, the recursive MFA outperforms traditional MFA when slow process drifting occurs in process parameters. The recursive latent variable method can be adaptive to the drifting trajectories and still can conduct normal monitoring very well even the system has slightly changed to new configurations. In the meantime, one also infers that traditional MFA is inflexible and can be invalid for monitoring process drifting scenarios. Case 2: Random variations in output measurements. In this case, we assume that some random

variations are added with a certain output variable due to changed sampling noise. The process is running at mode 1 from 1-2000th sampling time, however, the first variable begins to have slight variations since 801st sampling moment as:

x1 = x1 + 1.07 × randn

(20)

where ‘randn’ denotes the MATLAB Gaussian random noise and 1.07 is the amplitude value. For clarity, the simulated data for x1 is given in Figure 5. [Figure 5-6 about here] As can be seen, when sampling noise becomes larger, the recursive MFA efficiently adjust the corresponding parameters to noise so that the monitoring index turns below control limit as more samples are included and updated. In contrast, MFA cannot follow the trends and show more unnecessary false alarms. Case 3: Adding a new operating mode. The original defined process runs subsequently from mode 1

to mode 3 and each mode consists of 1000 samples. Then a new operating mode numbered mode 4 is involved from 3001st sampling time. The new mode is defined as: Mode 4: t1 : N (16,1.5) ; t2 : N (10,2.5) For illustration, test set of the new defined mode and the previous 3 modes are scattered in Figure 7. It can

15



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

be seen that all these modes stay very close and the new defined data cluster has obvious intersections with mode 2. Like above, the comparative results from recursive MFA and MFA are depicted in Figure 8. [Figure 7-8 about here] One can clearly see from Figure 8 that the recursive MFA monitoring scheme can effectively model and monitor a new mode. On the contrary, traditional MFA strategy cannot efficiently incorporate new mode information and simply treat all data from mode 4 as faulty cases. Actually, MFA has to make extra offline modeling in such cases which can be inefficient and time consuming. Case 4: Step error. The original defined process runs subsequently from mode 1 to mode 3 and each

mode contains 1000 sampling records. Random variation of x1 occurs in mode 2 during 1301-2000 sampling period whereas in mode 3, a step error happens for x3 during 2501-3000 sampling time. It should be noted that compared with random variations that may not directly result in the change of product quality, the step error is assumed to be aroused by some malfunctions in process units such as pipe-burst caused feed loss or a sudden unknown change of environmental element like temperature. Therefore, in open loop conditions, such error can be unwanted variations and should be detected for alarm instead of adaptive modeling. The comparative study of monitoring results by recursive MFA and MFA is shown in Figure 9a and 9b. One can clearly infer that the recursive mechanism based modeling and monitoring method can be more advantageous to time-varying multimode situation since it can be easily adaptive to process data and make the right alarm decisions for step errors. Therefore, the automatic scheme also enables operators take only necessary measures in a real-time manner without manually adjust models all the time. [Figure 9 about here]

4.2 Application to closed-loop continuous stirred tank heater (CSTH)

16


Page 16 of 42

Page 17 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


In this case, the developed recursive MFA is applied to the CSTH benchmark. Different with the above open-loop numerical example, the closed-loop CSTH process with disturbances is a self contained hybrid Simulink model

39

. In the meantime, the disturbances are real data sequences measured

experimentally from the plant. The CSTH flowchart is given in Figure 10. As can be seen, the inputs are hot water, cold water and stream valve demands. In the meantime, the process outputs are measurements of level, temperature, hot water and cold water flow that are regulated by corresponding PID control loops. Similar to Ref 35, in this work, three controlled variables including level, temperature, cold water flow and the corresponding PID values are included for modeling and monitoring. A basic simulation model with control-loop for CSTH can be obtained from Thornhill’s website operating conditions, a total of five modes are designed

35

39

. In order to simulate the multimode

. Notice that we assume only 4 modes are

available during offline modeling and mode 5 is a new mode which should be added later halfway during monitoring. The corresponding set-points (SPs) are listed in Table 1. [Figure 10 about here] [Table 1-2 about here] For modeling, 200 samples are generated from each mode and a total of 800 entries are finally collected. The modeling results by recursive MFA is shown in Figure 11. It is evident that all local operating regions are well assigned by the relevant Gaussian components. For testing purpose, we have designed two monitoring cases for model validation and the detailed descriptions are given in Table 2. In case 1, the tank is initially running in mode 1 during 1-1600th sampling time, then process switches into mode 3 from 1601st sample. However, the temperature begins to show random variations during 2201-3200th sampling time. Afterwards, the CSTH operates in mode 2 during which the level step error happens at sample 3801 and lasts to the end. In case 2, the process is first running in mode 1 and switches

17



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

into mode 2 from 1601st moment, but followed by the hot water valve slow drifting from 2201-3000th samples. Subsequently, a new mode (mode 5) is defined from 3001st sample and is included into monitoring. It should be noted that all conditions during simulations are well controlled. The comparative results for both cases are exhibited in Figure 12 and 13 respectively. [Figure 11-13 about here] For modeling, one can see that the probabilistic distributions of multimodal data have been well estimated by recursive MFA. For case 1, it can be readily seen that our method can effectively capture the random variations while the step errors can be treated by first alarming errors and then adding a new stable mode in a reasonable manner. The step errors here have been later regarded as normal ones after alarms since the controller has good stabilizations and hence process comes into normal operating conditions. Although one can infer that both methods show similar efficiency for step errors, the conventional MFA show more false alarms for both normal operations and random variations. In fact, the false alarm for mode 3 may due to the shrink covariance matrix estimations which could result in the bad control limit eclipse. In case 2, both MFA and recursive MFA can perform well for normal operating situations. However, when hot water valve begins to slow drift and a new operating condition is later added, the MFA cannot distinguish the complicated operating schemes. As opposed to MFA, the designed recursive approach catches both data characteristics and can define a new situation which can be commonly encountered in practice. The different testing scenarios also clearly demonstrate the outstanding performance to monitor multimode time-varying processes. Finally, it is also worthwhile to note that for step error, although such error can be regulated by controller, the process finally comes to a new mode after stabilization which is quite different from the original one. If the operator further diagnoses such error by defining a new mode, then we would derive the above results. Otherwise, the monitoring results for recursive MFA would still be

18


Page 18 of 42

Page 19 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


regarded as abnormal events which can be similar to the numerical example.

5. Conclusions and future work In this work, a novel recursive MFA method is developed and the corresponding monitoring mechanism is designed for monitoring multimode time-variant industrial processes. For offline modeling, the recursive MFA can self-adaptively decide the proper local components. For monitoring, the recursive MFA can keep the statistic model up-to-date with latest samples so that process changes can be elegantly involved. In order to avoid blind updating which could mislead the well defined probabilistic latent variable model, an online process modeling and monitoring procedure has been proposed. Only those samples which show strong informative similarity with respect to certain local model can be considered for updating the corresponding local MFA. For model validation, the recursive MFA has been compared with MFA on a numerical example and the CSTH benchmark. Simulation results indicate that the proposed recursive method outperforms the traditional MFA for monitoring time-variant situations such as process shifting, random variation, new mode alliance. Therefore, the new method has shown more reasonable potentials for modeling and monitoring time-varying processes with multiple modes. For future study, four aspects can be found. First of all, it should be noticed that the recursive model in above simulations are constructed on multimode data with relatively stable operating characteristics. In other words, we assume that the process data should be approximately Gaussian distributed under various stable conditions. However, it still could be a big challenge for a recursive method to model and monitor those chemical processes with strong non-Gaussian characteristics. In this case, although more Gaussian components can effectively deal with such issue, the recursive model can be inefficient especially when excessive mixture components are selected and the overfitting is incurred. Therein, it should be sensible to

19



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

develop an efficient model selection mechanism for strong non-Gaussian modeling/monitoring situations. Second, merely process variables may be not enough for monitoring those under control faulty scenarios since the shifty fluctuations can bring about some uncertainties for monitoring processes. In this case, an incorporation of quality related information should be more desirable for recursive model development since the containment of products information can be more instructive and convincing while judging the real operating conditions of industrial process. Fortunately, the recursive model in this work is developed upon a general online learning framework. Hence, such model can be further defined as probabilistic regression method and should be readily realized. The third potential improvement for recursive model is the modeling robustness. Practically, recursive model with high efficiency can be particularly suitable for modeling background with a large volume of recorded and forthcoming data (or even big data). However, it is well known that most large data sets are mixed with outliers. Outliers can be dangerous and may result in model misspecification. Therefore, robust mechanisms should be considered so as to maintain a reliable and effectiveness online recursive mechanism which is also an important concern. Last, updating mechanisms could also be further modified so as to be adaptive to more complex situations. For example, some faulty scenarios including intra-mode and inter-mode variations should be further recognized respectively before updating models so as to avoid blind updating from abnormal data. All these can be considered in the future work so as to make the recursive method more desirable for monitoring multimode and time-varying process.

Appendix A Specifically, in the E-step, all related expectation terms are computed and we have 20:

20


Page 20 of 42

Page 21 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


p ( k | y n , Θold ) =

π k ,old p ( y n | θk ,old )

(21)

K

∑π k ,old p ( yn | θk ,old ) k =1

x n,k = A k ( y n − µ k ) x n , k xTn , k = B k + x n , k

where A k = WkT ( Wk WkT + Σ k ) , Bk = Id − Ak Wk , −1

(22)

xTn , k

(23)

x n , k is the expectation value of the k th latent space.

In the M-step, one calculates the update rules by take the derivative of Eq.(4) with respect to each parameter, the induced rules can be summarized as 20: 1 N ∑ p ( k | y n , Θold ) N n =1

π k ,new =

(24)

 N  N  Wk , new =  ∑ p ( k | y n , Θold )( y n − µk ) xTn, k  ∑ p ( k | y n , Θold ) xn,k xTn,k   n=1  n=1  N

µ k , new =

∑ p (k | y

n

−1

(25)

, Θold ) x n , k

n =1

(26)

N

∑ p ( k | y n , Θold ) n =1

Σ k , new =

1 N T diag  ∑  p ( k | y n , Θ old )( y n − µ k )( y n − µ k ) − Wk , new x n , k N  n =1 

(yn − µk )

T

  

(27)

The E-step and M-step alternates until convergence.

Appendix B Please notice that Eq.(8) can be further unfolded as: s t   Wk  ∇ θt log p ( y t +1 | Θ ) = qk ( y t +1 )  sµt  k  k  sΣt   k t

where

qk ( y t +1 ) =

π k p ( y t +1 | θ tk ) K

∑ π k p ( y t +1 | θtk )

is the soft-assignment of

(28)

y t +1 to local component k. Let

k =1

sW t =  sw t k  k ,1

T

sw t

k ,2

L swt  , sµt =  sµ t k ,D  k  k ,1

sµ t

k ,2

T  L sµ t  , sΣt = diag  s t 2 k ,D  k  (σ k ,1 )

first order derivative terms can be derived as

21


s

(σ )

2 t k ,2



, (σ ) 

L s

2 t k ,D

the


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

swt = (σ kt ,i ) ( yt +1,i − µ kt ,i ) x t +1, k − w tk ,i x t +1, k xTt +1, k  k ,i −2

yt +1,i − µkt ,i − w tk ,i x t +1, k

sµ t =

2 t k ,i

s

(σ ) t k ,2

2

=

1 2 (σ kt ,i )

2

 y − µ t − w t x k ,i k ,i t +1, k  t +1,i σ kt ,i  

(29)

(30)

(σ )

k ,i

Page 22 of 42

2    − 1   

(31)

Now consider the Fisher information terms. According to the expectation calculations, one can further obtain all implicated Fisher information as:

Ic ( wtk ,i ) =

π k xt +1,k xTt +1, k

(σ )

2 t k ,i

I c ( µkt ,i ) =

(

I c (σ kt ,2 )

2

)

=

πk

(33)

(σ )

2 t k ,i

π k x t +1, k xTt +1, k 2 (σ kt ,i )

(32)

4

(34)

Based on Eq.(29)-(34) and (7), the recursive formulation of MFA can be easily derived.

Appendix C Recursive equation derived from Eq.(15) is not appropriate since the adaptive could be vanished as more samples are involved. Rewrite this equation as π kt =

1 1 t  ∑ qk ( y i ) − ck / t  1 − Kck / t  t i =1

(35)

One can infer that the influence term ck / t introduced by imposed priors which should be used for selecting the model can be vanished for large t. In order to amend such issue, work by Ref 38 modify by setting cT = c / T . Although small bias is introduced, this bias will always be the same during estimation with an acceptable level 38. Under this framework, the recursive model can always keep refreshing for new coming data. For recursive formulation, the modified Eq.(35) can be further induced as

22


Page 23 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


1  1 t +1  ∑ qk ( y i ) − cT  1 − KcT  t + 1 i =1 1  1 t 1  = ∑ qk ( y i ) + t + 1 qk ( y t +1 ) − cT  1 − KcT  t + 1 i =1

π kt +1 =

=

1 1 − KcT

t  1 t 1 q (y ) − c  ∑ qk ( y i ) − ∑ 1 t t t + ( ) i =1 k i T  i =1

  1 qk ( y t +1 )   + 1 t +  

= π kt +

t  1  1 1 qk ( y t +1 ) −  ∑ qk ( y t )  t + 1 1 − KcT t (1 − KcT ) i =1 

= π kt +

cT  1  1 qk ( y t +1 ) − π kt −   1 − KcT  t + 1 1 − KcT

(36)

which is exactly Eq.(17).

Acknowledgement This work was supported in part by the National Natural Science Foundation of China (NSFC) (61273167), National Project 973 (2012CB720500), and the Alexander von Humboldt Foundation.

References 1.

Chiang, L. H.; Braatz, R. D.; Russell, E. L., Fault detection and diagnosis in industrial systems. Springer:

2001. 2.

Kruger, U.; Xie, L. Statistical monitoring of complex multivariate processes: JohnWiley & Sons Ltd, West

Sussex, UK, 2012. 3.

Ge, Z.; Song, Z. Multivariate Statistical Process Control: Process Monitoring Methods and Applications.

Springer, London, 2013. 4.

Gao, Z.; Cecati, C.; Ding, S. X., A survey of fault diagnosis and fault-tolerant techniques-Part I: fault

diagnosis With model-based and signal-based approaches. IEEE Trans. Ind. Electron. 2015, 62, (6), 3757-3767. 5.

Venkatsubramanian, V.; Rengaswamy, R.; Yin, K.; Kavuri, S. N., A review of process fault detection and

diagnosis Part I: Quantitative model-based methods. Comput. Chem. Eng. 2003, 27, (3), 293-311. 6.

Ding, S. X., Model-based fault diagnosis techniques: design schemes, algorithms, and tools. Springer

Science & Business Media: 2008. 7.

Qin, S. J., Survey on data-driven industrial process monitoring and diagnosis. Annu. Rev. Control. 2012, 36,

(2), 220-234. 8.

Khatibisepehr, S.; Huang, B.; Khare, S., Design of inferential sensors in the process industry: A review of

Bayesian methods. J. Process Control. 2013, 23, (10), 1575-1596. 9.

Zhu, J.; Ge, Z.; Song, Z., HMM-Driven Robust Probabilistic Principal Component Analyzer for Dynamic

Process Fault Classification. IEEE Trans. Ind. Electron. 2015, 62, (6), 3814-3821. 10. Li, G.; Qin, S. J.; Zhou, D., A New Method of Dynamic Latent-Variable Modeling for Process Monitoring. IEEE Trans. Ind. Electron. 2014, 61, (11), 6438-6445. 11. Ge, Z.; Song, Z.; Gao, F., Review of Recent Research on Data-Based Process Monitoring. Ind. Eng. Chem.Res. 2013, 52, (10), 3543-3562. 12. Kano, M.; Nakagawa, Y., Data-based process monitoring, process control, and quality improvement: Recent developments and applications in steel industry. Comput. Chem. Eng. 2008, 32, (1), 12-24. 13. Kim, D.; Lee, I.-B., Process monitoring based on probabilistic PCA. Chemometr. Intell. Lab. 2003, 67, (2),

23



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

109-123. 14. Ge, Z.; Song, Z., Mixture Bayesian regularization method of PPCA for multimode process monitoring. AIChE J. 2010, 56, (11), 2838-2849. 15. Choi, S. W.; Martin, E. B.; Morris, A. J.; Lee, I.-B., Fault detection based on a maximum-likelihood principal component analysis (PCA) mixture. Ind. Eng. Chem. Res. 2005, 44, (7), 2316-2327. 16. Zhu, Z.; Song, Z.; Palazoglu, A., Process pattern construction and multi-mode monitoring. J. Process Control. 2012, 22, (1), 247-262. 17. Yu, J., Localized Fisher discriminant analysis based complex chemical process monitoring. AIChE J. 2011, 57, (7), 1817-1828. 18. Song, B.; Shi, H.; Ma, Y.; Wang, J., Multisubspace Principal Component Analysis with Local Outlier Factor for Multimode Process Monitoring. Ind. Eng. Chem. Res. 2014, 53, (42), 16453-16464. 19. Jiang, Q.; Yan, X., Monitoring multi-mode plant-wide processes by using mutual information-based multi-block PCA, joint probability, and Bayesian inference. Chemometr. Intell. Lab. 2014, 136, (0), 121-137. 20. Ge, Z.; Song, Z., Maximum-likelihood mixture factor analysis model and its application for process monitoring. Chemometr. Intell. Lab. 2010, 102, (1), 53-61. 21. Zhang, Y.; An, J.; Zhang, H., Monitoring of time-varying processes using kernel independent component analysis. Chem. Eng. Sci. 2013, 88, 23-32. 22. Elshenawy, L. M.; Awad, H. A., Recursive fault detection and isolation approaches of time-varying processes. Ind.Eng. Chem. Res. 2012, 51, (29), 9812-9824. 23. Xie, X.; Shi, H., Dynamic multimode process modeling and monitoring using adaptive Gaussian mixture models. Ind.Eng. Chem. Res. 2012, 51, (15), 5497-5505. 24. Choi, S. W.; Martin, E. B.; Morris, A. J.; Lee, I.-B., Adaptive multivariate statistical process control for monitoring time-varying processes. Ind.Eng. Chem. Res. 2006, 45, (9), 3108-3118. 25. Zhe, L.; Kruger, U.; Lei, X.; Almansoori, A.; Hongye, S., Adaptive KPCA Modeling of Nonlinear Systems. IEEE Trans.Sigal Process. 2015, 63, (9), 2364-2376. 26. Li, W.; Yue, H. H.; Valle-Cervantes, S.; Qin, S. J., Recursive PCA for adaptive process monitoring. J. of Process Control 2000, 10, (5), 471-486. 27. Wang, X.; Kruger, U.; Irwin, G. W., Process monitoring approach using fast moving window PCA. Ind. Eng. Chem. Res. 2005, 44, (15), 5691-5702. 28. Jeng, J.-C., Adaptive process monitoring using efficient recursive PCA and moving window PCA algorithms. J.Taiwan Inst. Chem.Eng. 2010, 41, (4), 475-481. 29. Tang, J.; Yu, W.; Chai, T.; Zhao, L., On-line principal component analysis with application to process modeling. Neurocomputing 2012, 82, 167-178. 30. Tong, C.; Palazoglu, A.; Yan, X., An adaptive multimode process monitoring strategy based on mode clustering and mode unfolding. J. of Process Control 2013, 23, (10), 1497-1507. 31. Yong, G.; Xin, W.; Zhenlei, W., Fault detection for a class of industrial processes based on recursive multiple models. Neurocomputing 2015, 169, 430-438. 32. Feital, T.; Kruger, U.; Dutra, J.; Pinto, J. C.; Lima, E. L., Modeling and performance monitoring of multivariate multimodal processes. AIChE J. 2013, 59, (5), 1557-1569. 33. Yu, J., A particle filter driven dynamic Gaussian mixture model approach for complex process monitoring and fault diagnosis. J. of Process Control 2012, 22, (4), 778-788. 34. Titterington, D. M., Recursive parameter estimation using incomplete data. J. Royal Stat. Soc. Series B (Methodological) 1984, 257-267. 35. Yu, J.; Qin, S. J., Multimode process monitoring with Bayesian inference‐based finite Gaussian mixture

24


Page 24 of 42

Page 25 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


models. AIChE J. 2008, 54, (7), 1811-1829. 36. Porat, B.; Friedlander, B., Computation of the exact information matrix of Gaussian time series with stationary random components. IEEE Trans. Acoustics, Speech and Signal Processing. 1986, 34, (1), 118-130. 37. Beal, M. J. Variational algorithms for approximate Bayesian inference. Ph.D. thesis, University of London, 2003. 38. Zivkovic, Z.; van der Heijden, F., Recursive unsupervised learning of finite mixture models. IEEE Trans.Patten Anal.Mach.Intell. 2004, 26, (5), 651-656. 39. Thornhill, N. F.; Patwardhan, S. C.; Shah, S. L., A continuous stirred tank heater simulation model with applications. J. of Process Control 2008, 18, (3), 347-360.

25



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Figure captions Figure 1: A typical offline modeling procedure by recursive MFA: (a) Scatter plots of samples and mixture components before and after recursive learning (b) Change behavior of mixture component numbers during the recursive learning procedure Figure 2: Recursive MFA for overlapping data: (a) Scatter plots of samples (b) Modeling results (c) Change behavior of mixture component numbers during the recursive learning procedure Figure 3: Simulation data for case 1. Figure 4: Case 1 monitoring results by (a) Recursive MFA and (b) MFA. Figure 5: Simulated data for case 2. Figure 6: Case 2 monitoring results by (a) Recursive MFA and (b) MFA. Figure 7: Scatter plot for all defined modes. Figure 8: Monitoring results for case 3 by (a) recursive MFA and (b) MFA. Figure 9: Monitoring results for case 4 by (a) recursive MFA and (b) MFA. Figure 10: The continuous stirred tank heater. Figure 11: Modeling results by recursive MFA: (a) Scatter plots of samples and mixture components (b) Change behavior of mixture component numbers during the recursive learning procedure Figure 12: Monitoring results for case 1 by (a) recursive MFA and (b) MFA. Figure 13: Monitoring results for case 2 by (a) recursive MFA and (b) MFA.

26


Page 26 of 42

Page 27 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table captions Table 1. Five operating conditions for CSTH. Table 2. Descriptions of two simulated monitoring cases.

27



Initialization

After offline modeling

14

14

Mode1 Mode2 Mode3

13

13

12

12

11 11 X2

10 X2

10

9 9 8 8

7

7

6 5 8

10

12

14

16

6 9

18

10

11

12

X1

13

14

15

16

17

18

X1

(a) 10 Mixture components remained

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 28 of 42

9 8 7 6 5 4 3 0

200

400

600

800 1000 1200 1400 1600 1800 Samples

(b) Figure 1: A typical offline modeling procedure by recursive MFA: (a) Scatter plots of samples and mixture components before and after recursive learning (b) Change behavior of mixture component numbers during the recursive learning procedure

28


Page 29 of 42

X2

15 10 5 5

10

15

20

40

40

20

20

X3

0

0 5

10

15

5

20

10 X2

20

10

15

15

X4

20

10

10

0 5

10

15

5

20

10 X2

X1

0

15

10

20

30

X3

30

30

30

20

20

20

20

X5

30 X5

X5

15

20 X4

X4

X1

10

10 5

10

15

10 5

20

X5

X3

X1

10 X2

X1

15

0

10

20

30

10 10

X3

15 X4

20

13 12 11 10 9 8 7

8

10

12 X1

14

16

Mixture components remained

(a)

X2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


10 8 6 4 2

0

500

(b)

1000 Samples

1500

(c)

Figure 2: Recursive MFA for overlapping data: (a) Scatter plots of samples (b) Modeling results (c) Change behavior of mixture component numbers during the recursive learning procedure

29



11 10 X2

9 8 7 6 0

Process drifting

200

400

600

800

1000 Sample

1200

200

400

600

800

1000 Sample

1200

1400

1600

1800

2000

1600

1800

2000

16 14 X4

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 30 of 42

12 10 Process drifting

8 0

1400

Figure 3: Simulation data for case 1

30


Page 31 of 42

Drifting

BIP

1

0.5

0 0

200

400

600

800

1000 (a)

1200

1400

1600

1800

2000

1600

1800

2000

Drifting

1 BIP

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


0.5

0 0

200

400

600

800

1000 (b) Sample

1200

1400

Figure 4: Case 1 monitoring results by (a) Recursive MFA and (b) MFA

31



18 Random variation

16 14 X1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 32 of 42

12 10 8 6 0

200

400

600

800

1000 1200 Sample

1400

1600

Figure 5: Simulated data for case 2.

32


1800

2000

Page 33 of 42

Random variation

BIP

1

0.5

0 0

200

400

600

800

1000 (a)

1200

1400

1600

1800

2000

1600

1800

2000

Random variation

1 BIP

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


0.5

0 0

200

400

600

800

1000 (b) Sample

1200

1400

Figure 6: Case 2 monitoring results by (a) Recursive MFA and (b) MFA

33



14 Mode1 Mode2 Mode3 Mode4

13 12

11 X2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 34 of 42

10 9 New mode

8

7

6 8

10

12

14 X1

16

Figure 7: Scatter plot for all defined modes

34


18

20

Page 35 of 42

New mode

BIP

1

0.5

0 0

500

1000

1500

2000 (a)

2500

3000

3500

4000

New mode

1 BIP

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


0.5

0 0

500

1000

1500

2000 (b) Sample

2500

3000

3500

4000

Figure 8: Monitoring results for case 3 by (a) recursive MFA and (b) MFA

35



Random variations

1

Step error

BIP

0.8 0.6 0.4 0.2 0 0

500

1000

1500 (a)

2000

2500

Random variations

1 BIP

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 36 of 42

3000

Step error

0.5

0 0

500

1000

1500 Sample (b)

2000

2500


36


3000

Page 37 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 10: The continuous stirred tank heater.

37



Modeling results by recursive MFA 11 10

Flow

9 8 7 6 5 8

9

10

11

12

13

14

15

Mode1 Mode2 Mode3 Mode4 16 17

Level

(a)

Mixture components involved

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

10 9 8 7 6 5 4

0

200

400 600 Samples

800

1000

(b) Figure 11: Modeling results by recursive MFA: (a) Scatter plots of samples and mixture components (b) Change behavior of mixture component numbers during the recursive learning procedure

38


Page 38 of 42

Page 39 of 42

Random variation

BIP

1

Step error

Alarm 0.5

0 0

500

1000

1500

2000 (a)

2500

3000

3500

4000

Step error

Random variation

1 BIP

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


0.5

0 0

500

1000

1500

2000 Sample (b)

2500

3000

3500

4000


39



Hot water valve drifting

BIP

1

A new mode

0.5

0 0

500

1000

1500

2000 (a)

2500

3000

Hot water valve drifting

1 BIP

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 40 of 42

3500

A new mode

0.5

0 0

500

1000

1500

2000 Sample (b)

2500

3000

3500


40


Page 41 of 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 1: Five operating conditions for CSTH Variable

Mode 1

Mode 2

Mode 3

Mode 4

Mode 5

Level SP

12

16

9

12

12

Temperature SP

10.5

10.5

10.5

13.5

8

Hot water SP

5.5

5

4.5

6

4

41



1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 42 of 42

Table 2: Descriptions of two simulated monitoring cases Case

Mode

1

1

2

Event descriptions

Event starting time

Ending time

Steady operation

Initial

1600

3

Temperature random variation

2201

3200

2

Level step error

3801

4200

1

Steady operation

Initial

1600

2

Hot water valve slow drifting

2201

3000

5

A new mode added

3001

3800

42


Recursive Mixture Factor Analyzer for Monitoring Multimode Time

Recommend Documents