Localized, Adaptive Recursive Partial Least Squares Regression for

Article pubs.acs.org/IECR

Localized, Adaptive Recursive Partial Least Squares Regression for Dynamic System Modeling Wangdong Ni,†,* Soon Keat Tan,‡ Wun Jern Ng,§ and Steven D. Brown⊥ †

DHI-NTU Centre and Nanyang Environment and Water Research Institute, Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798 ‡ NTU-MPA Maritime Research Centre and School of Civil and Environmental Engineering, Nanyang Technological University, 50 Nanyang Ave, Singapore 639798 § Nanyang Environment and Water Research Institute and School of Civil and Environmental Engineering, Nanyang Technological University, 50 Nanyang Ave, Singapore 639798 ⊥ Department of Chemistry and Biochemistry, University of Delaware, Newark, Delaware 19716, United States ABSTRACT: A localized and adaptive recursive partial least squares algorithm (LARPLS), based on the local learning framework, is presented in this paper. The algorithm is used to address, among other issues in the recursive partial least-squares (RPLS) regression algorithm, the “forgetting factor” and sensitivity of variable scaling. Two levels of local adaptation, namely, (1) local model adaptation and (2) local time regions adaptation, and three adaptive strategies, (a) means and variances adaptation, (b) adaptive forgetting factor, and (c) adaptive extraction of local time regions, are provided using the LARPLS algorithm. Compared to RPLS, the LARPLS model is proven to be more adaptive in the face of process change, maintaining superior predictive performance, as demonstrated in the modeling of three different types of processes.

1. INTRODUCTION Online and direct measurements of physical, chemical, and biological properties are usually expensive and are typically not incorporated in process monitoring for economic reasons.1−4 Instead, such data are derived through data modeling techniques. Regression methods5−7 based on principal component analysis (PCA)8,9 are frequently adopted for estimating the principal process variables. PCA has the advantages of being founded in statistics, of offering ready interpretability of the model, and of having efficiency in handling noise and colinearity issues in data from an industrial process. Other models such as those based on artificial neural networks (ANN),10 neuro-fuzzy systems,11 and Gaussian process regression methods,11−14 are also widely accepted as useful techniques for online prediction. However, it has been observed many times that after the model has been established for a certain process, it may not be able to adapt to changes in process characteristics, and the estimation performance deteriorates. Such changes in process characteristics may arise from changes in the environmental conditions or in the consistency of the input (e.g., via catalyst deactivation). Helland15 and Dayal et al.16 presented different recursive partial least squares (RPLS) algorithms to model process dynamics with different updating mechanisms. Qin17 extended the RPLS algorithm by merging the old model with new data and introduced a forgetting factor to update the model. Others developed a model based on recursive PCA modeling,18,19 using a moving-window20,21 to adapt to changes in the process. These methods rely on adaptation schemes to correct the model and so accommodate changes in the process, but the predicted performance from these models is still limited either by the selection of forgetting factors as in the case of the recursive methods or by the length of the adaptation window, as in the case of the moving-window based techniques. In both cases, the © 2012 American Chemical Society

approach is generally one of selecting an initial, arbitrary value for the online prediction and adjusting it once sufficient samples have been collected and measured at the output. Therefore, local learning, which has been increasingly adopted for multivariate calibration,22−25 has been suggested in an effort to address the aforementioned limitations through examination of partitioned historical data sets.26,27 Fujiwara28 et al. proposed a local model. The authors split the historical data set into several partitions, where the best one was selected to train the model for online prediction. Although the model predicted abrupt changes in the process well, too many adaptations were needed in this algorithm, since a new local partition had to be extracted when the output of each new sample became available. In addition, the initial window used in the local partitioning was set arbitrarily. Kadlec and Gabrys29 reported the incremental local learning soft sensing algorithm (ILLSA) based on local modeling. From Kadlec’s work,29 ILLSA was, actually, an ensemble method and thus significantly improved predictive performance, but it is wellknown that an ensemble model suffers from issues with complexity and interpretability. In addition, once the historical data had been partitioned using this algorithm, each partition represented a single process state, and no new process states may be extracted for online prediction. This presents difficulty when measured online samples containing new process states become part of the historical data. In this paper, we report a recursive partial least-squares method (RPLS) with incorporation of novel, local modeling that we call localized and adaptive recursive PLS (LARPLS). LARPLS Received: Revised: Accepted: Published: 8025

December 26, 2011 April 10, 2012 May 16, 2012 May 16, 2012 dx.doi.org/10.1021/ie203043q | Ind. Eng. Chem. Res. 2012, 51, 8025−8039

Industrial & Engineering Chemistry Research

Article

is intended to perform two levels of model adaptation: (1) adaptation of local models and (2) adaptation of the local time region extracted from historical data. This algorithm first partitions the historical data into different local time regions and selects the best one to represent the current process state, instead of arbitrarily presetting the initial window. A PLS model is then trained based on the best local region for online prediction, and the means and variances of the local model are adapted to account for changes in means and variances over time. In addition, an adaptive forgetting factor is used to instantly track the process dynamics. Both adaptations directly address weaknesses in RPLS (which is sensitive to data scaling and requires a preset forgetting factor). After extended prediction using the local model, the local model should be adapted to extract the best local model representing the current process state, which may differ from the previous state. This level of adaptation permits the capture of any change in the process. Three different types of data sets, including two real data sets15,29 and one simulated data set,30,31 were employed to demonstrate the predictive performance from the LARPLS method. The catalyst activity data29 from a polymerization process was modeled by the LARPLS method to show its ability to handle slowly changed process dynamics, due to the catalyst decaying within one year. Implementation of the developed method to a simulated gas phase polyethylene process30,31 would demonstrate the predictive performance of handling different levels of noise in measurements and different data availabilities. Finally, a propylene polymerization process15 was employed to show the capability of the LARPLS method capturing process disturbance, in case the operating conditions changed significantly. The remaining sections of the paper are organized as follows: in Section 2, a brief overview is given of RPLS and LARPLS with different adaptation strategies. In Section 3, three application studies are employed to demonstrate the predictive performance of LARPLS models. Section 3.1 demonstrates the case study of catalyst activity prediction. The performance for the situations of different levels of noise and data availabilities from LARPLS is emphasized in Section 3.2 based on simulated data. Modeling process disturbance is explored in Section 3.3 for a propylene polymerization process. Finally, Section 4 presents the conclusions and the work to be explored in the future.

with error matrices EX and EY. The matrices T and U (score matrices), as well as the matrices P and Q (loading matrices), have a number of columns, with a ≤ min(m, p, n) being the number of latent variables in the PLS model. The x- and y-scores are connected by the inner linear regression, u1 = b1t1 + h1

with regression vector b1, and where t1 and u1 are the first PLS scores in the x-block and y-block, respectively. Subsequently, a second PLS component could be generated based on deflation of the residual matrices EX,1 and EY,1, as shown in eqs 5 and 6, through the same procedure as that was used to generate the first component. (5)

E Y ,1 = Y − b1t1q1T

(6)

PLS

{X, Y} ⎯⎯⎯→ {T, W, P, B , Q}

(7)

where T, W, P, B, Q, are the scores, weightings, loadings, regression coefficients, and loadings matrices. Once the PLS regression matrix has been generated, a new set of outputs may be predicted by using eq 1, on the basis of the new set of inputs. The number of latent variables a in the PLS model is usually optimized by using cross-validation. A powerful multivariate statistical tool, PLS works well for data streams with few changes, but the PLS model may become ineffective for modeling processes with time-varying, systematic changes in the data stream statistics. A recursive form of PLS called RPLS has been developed to deal with the latter cases (see Helland15 and Qin17). In RPLS, the PLS model is updated periodically through incremental integration of new data. Qin17 introduced three ways to adapt the PLS model for these applications: (1) sample-wise (performed sample by sample); (2) block-wise (performed when several new samples become available); and (3) by merging two PLS models. In this work, sample-wise adaptation is adopted. For the RPLS model, after the T, W, P, B, Q matrices have been generated based on the training data set, the PLS model can be updated by integrating the new data sample xt and yt into the model by applying a forgetting factor λ as follows: ⎡ λ PT ⎤ X t = ⎢ t − 1⎥ ⎢⎣ x t ⎥⎦

⎡ λB Q T ⎤ t − 1 t − 1⎥ Yt = ⎢ ⎢⎣ ⎥⎦ yt

(8)

where the forgetting factor defines the strength of adaptation. The lower the value of the forgetting factor, the larger the influence of the new data and the smaller the influence of the previous PLS model, resulting in faster adaptation of the PLS model to the new data. The expanded matrices described in eq 8 are used for prediction of target variables in yt+1 from the new sample xt+1, using a PLS model with r latent variables, r being the rank of Xt. Qin17 proved that predicted results from the PLS model based on eq 8 without the forgetting factor, λ, are the same as those from the PLS model based on the following equation:

(2)

⎡ X t − 1⎤ Xt = ⎢ ⎥ ⎣ xt ⎦

and Y = UQT + E Y

EX ,1 = X − t1p1T

Repeating this deflation a times, a PLS components are generated. In each deflation step, the score vectors tj (j = 1, ..., a) are normalized instead of the weight vector wj and loadings vector pj. Therefore, the PLS algorithm may be summarized as follows:

2. DYNAMIC PROCESS MODELING 2.1. Recursive Partial Least Squares (RPLS) Algorithm. Partial least-squares regression (PLS)32 is a popular linear regression technique that has been widely applied in areas such as multivariate analysis, process control, and dynamic system identification. Suppose X = (x1,...,xn)T ∈ Rn×m, where n is the number of available samples in the historical data, m is the number of measured variables, and p is the number of target variables are contained in Y(y1,...,yn)T ∈ Rn×p. The input and output matrices X and Y are linearly related as follows: (1) Y = XB + E where B is a matrix of regression coefficients and E is the residual matrix. Rather than finding this relation directly, both X and Y may be modeled by a set of linear latent variables according to the PLS regression model, X = TPT + EX

(4)

(3) 8026

⎡ Yt − 1⎤ ⎥ Yt = ⎢ ⎣ yt ⎦

(9)

dx.doi.org/10.1021/ie203043q | Ind. Eng. Chem. Res. 2012, 51, 8025−8039


Article

Figure 1. Schematic for the localized time regions and correlation coefficients in the catalyst activity data set: (A) sixth input variable and (B) target output variable; (C) correlation coefficients between the first sample of the test set and the entire training set.

Since the number of samples, n, in eq 9 is usually much larger than the rank of the input matrix in eq 8, use of the recursive PLS algorithm is more efficient than recalibrating a regular PLS model when a new datum is added to the historical data set. 2.2. Localized and Adaptive Recursive PLS Algorithm (LARPLS). The localized and adaptive recursive PLS algorithm (LARPLS) is based on the assumption that the current process state of any newly obtained sample is usually closest to that from the nearest sample in the historical data set. It is clearly depicted in Figure 1C that the farther away the data set is from the first sample in the test set, the smaller the correlation coefficient is for the historical samples (in the training set) for the catalyst activity data in general. Therefore, the last sample in the training set, which is closest to the first sample in the test, is employed to initialize localization and select the best localized region. As time progresses, more and more new data will be collected in the test set for online prediction, and the process state captured by current localization will be farther and farther away from the process state of newly measured data, and thus the predictive performance is expected to deteriorate. Once the predictive error is larger than the preset tolerance, a new localization process will be performed. For each localization, the process state of sample nearest to the newly collected data set is closest to that from the

new sample, which makes the LARPLS method track the process dynamic instantly and effectively. The incorporated two main processes and the entire procedure of LARPLS are, using catalyst activity data as an example, detailed as follows: (1) Historical samples stored in the database are divided into several local data sets, each of which contains successive samples included over a certain period of time. Local PLS models are developed and a best one is selected for online prediction. (2) Different strategies for the adaptation of the local model for online applications are evaluated and the best strategy is selected. 2.2.1. Localization and Training. Generally, a global model will not function well when the historical data contains significant noise and shows high redundancy. For this reason, the historical data is partitioned into local time regions, in an attempt to match any new datum xt to data in one of these regions, effectively tracking the process changes. Given the historical data set defined by the matrix of measured variables X = (x1, ..., xn−1)T and the target variables Y= (y1, ..., yn−1)T, the latest sample xn, taken when a measurement for yn is made, can be used as the validation set to select the window size for local time regions and to select the best 8027



Article

local region for online prediction. This process captures the current process dynamics effectively. To initialize the window size of local time regions, the historical data set X (x1, ..., xn−1)T and y(y1,...,yn−1)T are divided into k time regions, where k = 2, ..., i (i ≤ n−1), each of which contains the same number [(n−1)/k] of consecutive samples. Assume that the historical data set X(x1, ..., xn−1)T and y (y1, ..., yn−1)T has been divided into two local time regions initially. Then, PLS is applied to these two time regions to train local models, both of which are used to predict the target values ŷn for the latest sample xn. The predicted values ŷn from these two local PLS models are compared with the measured value yn to generate estimates of the predictive errors of each local PLS model. The local PLS model with the lower predictive error is retained. The historical data set X(x1,...,xn−1)T and y (y1,...,yn−1)T is then divided into three time regions to generate three local PLS models, and the one with the lowest predictive error is retained, as before. This localization process is repeated until the historical data set has been divided in to i local time regions. Theoretically, i could be set equal to n−1, where n represents the number of samples used in the training set and the last sample in the training set is used for validation. However, there is no need to split the training set into such a large number of local time regions. As was discussed by Kadlec et al.,29 an initial window containing 30 samples can represent the smallest process drift of catalyst activity in the same polymerization process. In this study, 20 local time regions, each containing 9 samples (the training set consisted of 176 samples), were found adequate to isolate the process dynamics. A set of i−1 local PLS models are developed, each of which represents different window sizes. Predictions from these i−1 local PLS models are then compared to select the best local model for online prediction. The best performing model, and the corresponding time region over which it has been trained, is treated as most representative of the current process dynamics. Figure 1 demonstrates the schematic for the localization of the historical data set, which differs from that in ref 29, where 3 different windows with widths of 30, 50, and 100 samples were employed to initialize the localization, separately. 2.2.2. Strategies for the Adaptation of the Local Model. To more effectively capture the process dynamics, three different strategies of adaptation of the local model are employed. Two of the strategies focus on adaptations of RPLS in local time regions, and the last adapts the window size (the best local region width) of the local model. Adaptation Strategy No. 1: Adaptation of Means and Variances. A dynamic system may be modeled by using many recorded process parameters, and many of these parameters may be of different orders of magnitude. As the process dynamics may not be steady, the means and standard deviations of the process variables will change over time. The PLS algorithm is sensitive to the scaling of every process variable. Thus, it is critical to appropriately normalize these variables and to use the normalized parameters in the PLS model. The historical data X and Y are normalized to have zero mean and unit variance. Incorporating new data xt and yt into X and Y may alter the mean and variance of these data. Qin17 proposed a method to introduce a vector 1 in the new data pair {xt, yt} to reduce the variance and to minimize the overall predictive error of the RPLS algorithm: ⎧⎡ ⎤ ⎫ 2 ⎨⎢X 1⎥ , Y⎬ ⎣ n−1 ⎦ ⎭ ⎩

where 1 is a vector of 1s and where the scaling factor 1/((n−1)1/2) is used to make the norm of (1/((n−1)1/2))1 comparable to the norm of the columns of X. However, simply introducing a vector 1 in the data set as in eq 10 may not fully account for changes in the means and variances. Because the newly collected samples may contain new process dynamics and thus incur poorer predictions from the PLS model, it is important to develop a new adaptation strategy that efficiently and instantly tracks the changes in the means and variances from the new measured samples. Consider the training data set and a window that contains n samples. The process variable parameters could be standardized in terms of mean x̅ and standard deviation σ. When the new datum is available, the means and variances will change incrementally (see eq 9). Given that xt̅ and σt are the mean and variance of n training samples, and when the sample of xt and yt at time t is acquired, the number of the samples is increased from n to n+1. The mean and variance of training set at time t+1 are recursively generated as follows: xt̅ + 1 =

n 1 xt + xt + 1 n+1 ̅ n+1

(11)

σt2+ 1 =

n−1 2 1 σt + (xt + 1 − xt̅ + 1)2 n n

(12)

The computation of mean and variance using eqs 11 and 12 has been widely used in process monitoring18,19,33 and online prediction,20,21 and the results and analysis have been theoretically demonstrated and described. Adaptation Strategy No. 2: Adaptation of the Forgetting Factor. The forgetting factor, which weakens the influence of the old model (data), is critical to adaptation in the RPLS algorithm. The regression matrices for the old model in eq 7 are weighted by this forgetting factor λ (0 < λ ≤ 1), which merges the old model with new data to update the model. In this way, the forgetting factor defines the strength of adaptation of the RPLS algorithm. The smaller λ is, the faster the RPLS model will adapt to new data and the faster the characteristics of the old data set are “forgotten”. If λ is too small, however, the adaptation is far too fast and may not generate a stable model. Hence, the forgetting factor λ is usually preset to a value ranging from 0.6 to 1, and this value has to be selected a priori in every model prediction exercise. The converse leads to slower adaptation to the process changes. Selection may be arbitrary, or it may be based on trial and error, but an inappropriate selection of the forgetting factor will inevitably lead to poor performance in modeling dynamic systems with RPLS. To address this weakness in the selection of λ, a new approach for adaptively setting the forgetting factor is proposed to allow RPLS to adapt the modeling dynamics according to the rate of change in the “drift”. When a new sample (xt in eq 8) is available, the RPLS model based on historical data Xt−1 can be applied to generate a predicted value ŷt for the target variable corresponding to xt. After the target variable yt is measured, the absolute value of the relative error (ARE) is calculated from xt in eq 8 to obtain the predicted target value of yt from the RPLS model. The absolute value of the relative error (ARE) is defined as ⎛ y − ŷ ⎞ t⎟ ARE = abs⎜⎜ t ⎟ y ⎝ t ⎠

(10) 8028

(13)



Article

• Step 1. Split the initial historical data set into a set of local time regions containing consecutive samples. • Step 2. Select the best local time region for model building. • Step 3. Build the initial PLS model on the local historical time region and reserve the resulting regression matrices {T, W, P, B, Q}. • Step 4. Apply the PLS model for online prediction, and set the threshold for local model adaptation. • Step 5. When the measured value of the output variable for the predicted sample is available, calculate the absolute value of the relative error of prediction. Add the new data pair {x1, y1} to the historical data matrices {X, Y}, so that the historical matrices become the augmented set {X1, Y1}. • Step 6. Update the local model. 6.1. Adaptation of local time region: If the absolute value of the relative error is bigger than the preset threshold δ, go to step 1 to relocalize the set of updated historical data matrices {X1, Y1}. Then, go to the next step. 6.2. Adaptation of local model: a. Review and modify the variable means and variances, according to eq 1) through eq 12. b. Review and modify the forgetting factor, according to eqs 13 and 14.

where abs( ) is the absolute value operator and ŷt is the value predicted by the RPLS model. The adaptive forgetting factor λ is generated as follows: λ = (exp( −ARE))2

(14)

When the output value measurement yt corresponding to the new sample xt+1 becomes available, the model based on Xt−1 and Yt−1 is updated with the new data pair xt, yt, and a new forgetting factor λ is automatically computed from eqs 13 and 14 as each new pair is incorporated. The absolute relative error used in eq 14 originates from eq 13, which represents the ability of the current model in capturing the process states (process dynamics). The operator of the exponential function will convert the ARE to a value ranging from 0 to 1, and the square function is employed to enhance the forgetting factor’s adaptability. Based on eqs 13 and 14, poor predictive performance will result in a higher ARE value. Consequently, a smaller forgetting factor will then be generated, thus leading to faster adaptation of new data for the RPLS model. This procedure adaptively generates a new forgetting factor as necessary, adapting the factor either to the rate of process changes or to the variations in a dynamic process. Other forms of generating forgetting factors may produce better adaptations, but the forgetting factors generated from eqs 13 and 14 have produced excellent adaptations for RPLS, at least for three different cases included in this work. Adaptation Strategy No. 3: Adaptation of the Local Model. After establishing the best local time region, the local PLS model is trained to predict the target data progressively, and the model is similarly progressively updated. Eventually, the simulation will extend beyond the latest process state and reach over to the new process state, which may differ significantly from the previous process state, leading to deterioration in the predictive performance. Therefore, the local model should be adaptive to account for the latest process change. We propose a strategy for this adaptation, as follows. When measured values are acquired for new output data, the predictive performance on the latest available data is employed to determine whether the local model needs to be updated. As discussed in the adaptation of the forgetting factor, the absolute value of the relative error (ARE) in eq 13 is used to decide on the local model adaptation. When the absolute value of the relative error is greater than a preset threshold, δ (tolerance), the historical data should be reapportioned into local time regions as in the initial localization, due to changes imposed by the newly predicted data added into the database. The preset threshold defines the frequency of adaptation, so the smaller the preset threshold is, the more frequent the adaptation, to better focus on small changes in the process states; a larger preset threshold produces less frequent adaptation, which accounts for more general process states. The threshold used for adaptation must be optimized for the specific dynamic system. Threshold values of 0.05, 0.1, and 0.2 were tested in this study. The LARPLS model with fixed forgetting factor λ = 0.8 and threshold value of 0.1 needed 39 relocalizations (adaptations). However, LARPLS models with the same forgetting factor required 21 adaptations when the threshold value used was 0.2, and 59 adaptations when the threshold value was preset at 0.05, respectively. Considering the frequency of adaptation and predictive performance, the threshold value of 0.1 was preset for all the localized models. The overall procedure for online model prediction and adaptation can be formulated as follows:

• Step 7. Go to step 4 for the upcoming data prediction. Other alternatives, such as k-means clustering,34 correlationbased just-in-time modeling (CoJIT),28 and receptive fields using t-tests,29 have been widely applied to localize the historical data into local regions. A common approach for partitioning is k-means. How does one appropriately select the k clusters?34 In any case, the sample discontinuity of local clusters generated from k-means clustering would appear to limit its application in dynamic process modeling. Fujiwara et al.28 proposed a localization based on CoJIT through investigation of the Q and T2 statistics of historical data to model two dynamic systems, but his method suffered from the need to make an initial selection of window size and localization when the output of every new sample was measured. However, LARPLS only relocalizes when the predicted error exceeds the threshold for model update. The issue of initial selection also surfaced in Kedlec’s ILLSA.29 As discussed in the Introduction, ILLSA employed a complex combination and update of local models. To simplify the modeling, a LARPLS is developed and demonstrated in this work, which is easy to understand and is readily implemented. Although it verges on a “blind-man” approach, LARPLS does not require prior investigation and diagnostics for the historical data, nor its scores or covariance structure. 2.3. Evaluating Performance of the Models for Dynamic Systems. The goodness of fit of the mathematical model to the system under investigation is measured by the degree of agreement between the actual process output and the model predicted output. In order to show the efficiency of dynamic models, the comparison may be performed by visual inspection of the responses of the model and the process, and through quantitative metrics based on the values of selected performance indicators. The number of latent variables in the PLS or RPLS model is optimized by cross-validation. The root mean squared error of cross-validation is employed to determine the appropriate number of latent variables involved in PLS model, as follows: 8029


Industrial & Engineering Chemistry Research 1 n

RMSECV =

Article

n

∑ (yi − yi ̂ )2 i=1

This data set contained 15 input variables, and the physical values represented by the variables in this data set are listed in Table 1. The data set was split into two parts: the training set (the

(15)

where yi and ŷi are the actual system output and the PLS model estimate of output of the ith sample in training set, respectively, and where n is the number of samples in the training data set. Then, we use either the root mean squared error of prediction (RMSEP) or the relative root mean squared error of prediction (RRMSEP), as appropriate, to compare prediction by the model to the output of the process: RMSEP =

RRMSEP =

1 n

Table 1. Variables of the Catalyst Activation Data Set29

n

∑ (yi − yi ̂ )2 i=1

1 n

⎛ y − y ̂ ⎞2 ∑ ⎜⎜ i i ⎟⎟ yi ⎠ i=1 ⎝

no.

description

units

1 2 3 4

time measured flow of air measured flow of combustible gas measured concn of combustible component in the combustible gas (mass fraction) total feed temp. cooling temp. temp. at 1/20 of reactor length temp. at 2/20 of reactor length temp. at 4/20 of reactor length temp. at 7/20 of reactor length temp. at 11/20 of reactor length temp. at 16/20 of reactor length temp. at 20/20 of reactor length product concn of oxygen (mass fraction) product concn of combustible component (mass fraction)

days kg/h kg/g unitless

5 6 7 8 9 10 11 12 13 14 15

(16)

n

(17)

where yi and ŷi are the actual system output and the RPLS model estimate of output at the ith step, respectively, and n is the number of samples in the online prediction data set.

3. APPLICATION STUDIES In this section, two industrial data sets and a simulated data set were employed to demonstrate the predictive performance of LARPLS modeling. The simulated gas polyethylene reactor data30,31 was used to illustrate the capability of the LARPLS model in handling noise. Implementation of LARPLS to an industrial propylene polymerization15 process demonstrated its ability to model abnormal process states (process disturbances). In this case, the operating conditions changed significantly after the 50th sample in the test set. Kadlec’s catalyst activity data29 was used as an example of modeling a slowly changing, timevarying process (involving catalytic activity that decayed within one year). 3.1. Application to Modeling Catalyst Activity in a Polymerization Process. The data set from a catalyst activation process in a polymerization reactor derived from Dr. Kadlec was employed as a case study. The data in the work was similar to the data that Dr. Kadlec examined in his paper.29 Both were from the same process. The predictive performance of LARPLS modeling was compared with that of traditional recursive PLS (RPLS), a method which was also adopted by Kadlec29 as a benchmark. It is demonstrated in this work that the two adaptive strategies in LARPLS make the RPLS model more adaptive to process changes and improve predictive performance. 3.1.1. Data Set. The data set used in this example was adapted from Kadlec et al.29 Catalyst activity affected this polymerization process, and the activity decayed to zero over time. The variations of the feed and the operating conditions were recorded as input variables, and the measurements of concentrations, flows, and several point temperatures were used to identify the state of the process. Many of the input variables in the collected data set covering one year of operation of the process plant contained high levels of colinearity and noise. In addition, there were large numbers of outliers, some of which can be discerned in Figure 1A. The data show other typical issues, too, such as missing values, changing sampling rates, and automatic value interpolation by the data recording system. The target output variable in this data set varied from 0.05 to 1.01 (as shown in Figure 1B).

°C °C °C °C °C °C °C °C °C unitless unitless

first 30% of the data set form the historical data), and the test set (the remaining 70% of the data, which represented the online data, and was delivered as a stream of samples), the same split as that previously reported,29 although the data set used in the study reported here is different from that used in ref 29, as noted. 3.1.2. Data Preprocessing. Because the data set contained many samples, about 6000 in all, and because certain input variables involved a large number of outliers and missing values, the data set was preprocessed prior to the LARPLS modeling in a way similar to that reported in ref 29 because these data sets model similar catalyst processes with similar process parameters. Specifically, the data set was down-sampled by a factor of 10; input variables 3, 4, and 15 were removed due to the presence of severe outliers and missing values; and all data samples with missing target values were removed prior to LARPLS modeling. After this preprocessing step, the training set contained 176 samples; the remaining 410 samples were used for the online simulation test set. 3.1.3. Results. The recursive PLS model was applied to this catalyst activation data set first as a benchmark to demonstrate the enhanced performance of the LARPLS modeling. For the ordinary RPLS model, the PLS algorithm was applied to the training set, which had been scaled to zero mean and unit variance, to train the initial model. This PLS model was optimized by 5-fold random cross-validation to determine the number of latent variables. A value of 3 latent variables was found best, which demonstrates the high colinearity of variables in this data set. Figure 2A illustrates the RMSECV (root mean squared error of cross-validation) metrics of PLS models with a varying number of latent variables (from 1 to 12). As can be seen in Figure 2B, the performance of the RPLS model (with a forgetting factor of 0.8) was poor. Since the predictive performance of RPLS model was highly related to the forgetting factor, the forgetting factors, varying from 0.6 to 1, should be preset in the RPLS model. Due to the slow changes in the process shown in Figure 1B and reported in ref 29, the forgetting factor can be assigned a value slightly less than 1. After investigating the performance of models based on forgetting factor values ranging from 0.6 to 1, a forgetting factor of 0.8 for 8030



Article

Figure 2. RPLS soft sensor: (A) cross-validation of the RPLS model and (B) predictive results.

Table 2. Predictive Results from RPLS and LARPLS Modelsa models

RPLS

LARPLS1

LARPLS2

LARPLS3

LARPLS4

LARPLS5

RMSEP RRMSEP

0.39 125%

0.039 14.07%

0.024 9.07%

0.039 14.15%

0.026 10.17%

0.017 7.62%

a

LARPLS1 represents the model without adaptation of the means and variances or the adaptive forgetting factor; LARPLS2 represents the model with adaptation of the means and variances only; LARPLS3 represents the model with adaptation of the adaptive forgetting factor; LARPLS4 represents the model with adaptation of the means and variances and the adaptive forgetting factor, and LARPLS5 represents the modified LARPLS model based on LARPLS4.

RPLS models was selected and preset in this study. The RPLS model generated RMSEP and RRMSEP values of 0.39 and 125%, respectively, as shown in Table 2. In the LARPLS algorithm, the historical data was segmented first to create subsets of the historical data set containing 36 successive samples each. The PLS algorithm was applied to this local time region, over which that portion of the data set was normalized and scaled to exhibit zero mean and unit variance. After the same cross-validation process, a local PLS model with three latent variables (the same number as that found in the conventional RPLS model) was found to produce the best predictive performance. The results are shown in Figure 3A. The forgetting factor was first preset to 0.8 as before and without adaptation of the forgetting factor or the means and variances. The local soft sensor needed to be adapted when the absolute value of the relative error of the latest sample was greater than the preset threshold, 0.1 (10%). The LARPLS model without adaptation of either the forgetting factor or the mean and the variance produced RMSEP and RRMSEP values of 0.039 and 14.07% (shown in Table 2), respectively, both significantly improved over the predictive results from the conventional RPLS model. Figure 3B shows that the predicted outputs and measured outputs track each other well, a result that again indicates the superiority of the LARPLS modeling as compared to RPLS. When the mean and variance were adaptively adjusted in the model, the LARPLS model produced RMSEP and RRMSEP of 0.024 and 9.07% (shown in Table 2), respectively, a result much better than that obtained with the RPLS model. There was an obvious improvement in the predictive performance, which may be seen by comparing the measured catalyst activity with the predicted results of LARPLS modeling with the means and variances adaptation, as shown in Figure 3C. There are a few

spikes in the trace of the output variable (catalyst activity) shown in Figure 3B near the 50th, 100th, 170th, 290th, 350th, and 380th samples. Most of these spikes were reduced, and there were some that even disappeared. Further development showed that the local model with an adaptively set forgetting factor produced RMSEP and RRMSEP of 0.039 and 14.15% (shown in Table 2), respectively, results which were almost identical to that obtained with the first local soft sensor described in this paper. A similar conclusion from the first LARPLS model can be inferred from Figure 3E. However, Figure 3D demonstrates that, for most online samples, there is no need for programmed forgetting (adaptation) in the local model, because most of the estimated forgetting factors are close to 1, except for certain regions that directly corresponded to the spikes in the predicted curve, as shown in Figure 3E. In the fourth LARPLS algorithm, the local model may also include adaptation of means and variances as well as the forgetting factor so as to generate more robust predictive performance, as demonstrated in Figure 3F. In terms of RMSEP and RRMSEP, the second and fourth LARPLS models produce almost the same predictive performance. As a result of the instant adaptation of the means and variances of the process, some spikes seen in Figure 3D are now reduced; see Figure 3F. This LARPLS model generated RMSEP and RRMSEP of 0.026 and 10.17%, respectively. Figure 3G shows the predictive curve. The RMSEP (0.026) from the LARPLS model was only 1/15 of that (0.39) from the RPLS model, which was also the benchmark value reported in Kadlec’s report,29 where the RMSEP (3.64) from ILLSA modeling was about 1/11 of that (0.318) from RPLS. These results indicate the LARPLS method in this study achieves slightly better performance; however, it has an advantage over Kadlec’s ILLSA method in terms of being easy 8031



Article

Figure 3. Results from LARPLS models: (A) the cross-validation of initial local model; (B) predictive results from LARPLS modeling without the adaptation of the means and variances or the adaptive forgetting factor; (C) predictive results from LARPLS modeling with the adaptation of the means and variances; (D) the variation of the adaptive forgetting factor from LARPLS modeling; (E) predictive results from LARPLS modeling with an adaptive forgetting factor; (F) variation of the adaptive forgetting factor from LARPLS model with the adaptation of the means and variances and the adaptive forgetting factor; (G) predictive results from LARPLS model with the adaptation of the means and variances and the adaptive forgetting factor; (H) the variation of the adaptive forgetting factor from the modified LARPLS model; and (I) predictive results from the modified LARPLS model.

8032



Article

to understand and being readily implemented other than the complicated fusion and adaptation steps used by Kadlec.29 Although the LARPLS modeling method can be used to effectively track the process dynamics, the slowly changing process of catalyst decay deserves further investigation, and in particular, the spikes in the predicted curve from LARPLS model with adaptation of means and variance and the adaptive forgetting factor require comments. For the LARPLS model, sometimes relocalization takes place, when the process state captured by the current local model is far away from that of newly obtained sample. Once the relocalization process takes place, the PLS model should be retrained and optimized by cross-validation as described in step 3 of the LARPLS method. The predictive performance of a PLS model depends on appropriate selection of latent variables (the bias-variance trade-off35). Initially the number of latent variables with the lowest cross-validation error, RMSECV, was automatically selected, and then, the model was used for online prediction. In some cases, this is feasible, but in other cases, overfitting occurs, especially for the spikes around the 90th and 390th samples, as shown in Figure 3G. When relocalization is observed, reselection of the appropriate number of latent variables is performed, leading to a modified model. The modified LARPLS model then generates the adaptive forgetting factor as shown in Figure 3H, and the predictive results shown in Figure 3I. Comparing the forgetting factors between Figure 3H and Figure 3F, the modified LARPLS is seen to perform better than LARPLS4, especially around the 90th and 390th samples. As depicted in Figure 3I, in that the spikes around the 90th and 390th samples in Figure 3G now disappear, and the predicted curve tracks the measured curve very well. A similar conclusion can be drawn from Table 2, as indicated by RRMSEP of 7.62% and RMSEP of 0.017 generated from the modified LARPLS. The adaptation strategies that are part of the LARPLS modeling are beneficial not only here, but also for traditional RPLS modeling. Table 3 summarizes the predicted results from

Figure 4. Schematic of industrial gas phase polyethylene reactor system30,31.

3.2.1. Data Set. The data set used in this example was provided by McAuley et al.30,31 and contained 305 samples generated from the simulation, of which 150 samples were used as historical data; the remaining samples were employed to mimic online prediction. The data set was downloaded from the Nonlinear Model Library (http://www.hedengren.net/ research/models.htm). Different levels of random noise, 2%, 5%, and 8%, were added to the output to demonstrate the capability of LARPLS modeling in handing noise effects. Different percentages of target values were also investigated using LARPLS modeling. 3.2.2. Results. Based on the results reported for the previous system, only RPLS and LARPLS with an adaptive forgetting factor and adaptation of means and variances were investigated in this simulated case. Different levels of noise, 0, 2%, 5%, and 8%, were added to the data, which were modeled by both RPLS and LARPLS to investigate the effects of noise on predictions from these two modeling methods. The predicted results from RPLS and LARPLS are presented in Figures 5 and 6. The predictive performance of LARPLS is much better than that of RPLS, especially for the noise-free case (shown in Figure 5A and Figure 6A). With increasing noise levels, the predictive performance deteriorates for both RPLS and LARPLS models, but in every corresponding case, LARPLS performs better than RPLS. Similar conclusions can be drawn from Table 4 in terms of predictive performance, as measured by RMSEP and RRMSEP. For the LARPLS models, as the level of noise added to the data is increased, the number of relocalizations (update in the local time region) is also increased, which indicates that the preset threshold for local model updating should be at least large as the noise level. Otherwise, the LARPLS model will relocalize at each newly measured sample. The LARPLS model applied to data with 8% random noise added generates an RRMSEP of 7.14%, which implies that most of the error in prediction arises from the noise in this case. Based on these results, the threshold preset as 0.1 (10%) as that in the previous case could ensure that the simulation can be performed, otherwise the LARPLS model will relocalize at every sample in the online test set. The effects of the preset threshold on performance and relocalization of the LARPLS has been covered in the discussion of adaptation strategy 3.

Table 3. Predictive Results from RPLS Soft Sensors with Different Adaptive Strategiesa models

RPLS1

RPLS2

RPLS3

RPLS4

RMSEP RRMSEP

0.46 192%

0.22 63.63%

0.21 58.92%

0.10 30.06%

a

RPLS1 represents the RPLS model adapted by the scaling correction given in eq 10 with a forgetting factor λ = 0.8; RPLS2 represents the RPLS model with means and variances adaptation as given in eqs 14 and 15, with a fixed forgetting factor λ = 0.8; RPLS3 represents the RPLS model with an adaptive forgetting factor, and RPLS4 represents the RPLS model with both means and variances adaptation and an adaptive forgetting factor.

RPLS models using different adaptive strategies. Using both types of adaptations, LARPLS modeling can significantly improve the predictive performance of an RPLS model, as shown in Table 3. Errors decreased by about 6-fold. 3.2. Application to Simulated Gas Phase Polyethylene Process. The design and model of a fluidized bed reactor had been described by McAuley et al.,30,31 the feed to which consists of ethylene, comonomer, hydrogen, inerts, and catalyst (as shown in Figure 4). The detailed dynamic model of this reactor is described in the Appendix. Four process variables, such as the concentration of inerts MIn, monomers MM, and of catalyst Yi (i = 1,2), were used to predict the reactor temperature T. 8033



Article

Figure 5. Predicted results from RPLS on simulated gas phase polyethylene data: (A) noise-free data (B) 2% noise added to data; (C) 5% noise added to data; and (D) 8% noise added to data.

CSTR was modeled as the target in this example. Eight process parameters were selected to control the system and to predict the melt flow rate (MFR) as the output; 360 data sets were collected. The first 100 samples were stored as the historical set and the remaining were used for the online test. 3.3.2. Results. As previously discussed, only RPLS and LARPLS models with adaptive forgetting factors and adaptation of means and variances were applied to the propylene polymerization data. Both were used to model a process with abnormal process states (process disturbances) produced by a significant change of operation conditions. The threshold was preset as 0.1 for LARPLS, a value which was also used by Li et al.15 as a control limit for the process. The performance is demonstrated in Figure 8 and Table 6. In terms of RMSEP and RRMSEP, LARPLS outperforms RPLS modeling significantly. This data set contained a significant process disturbance after the 50th sample in which a transition from grade F401 to J34015 took place. The operating conditions for manufacturing these two grades were different. However, the LARPLS modeling handles this abnormal process state well, with good predictive performance, even with a very sharp increment in the measured output. The LARPLS model is able to generate good predictive

Different data availabilities (e.g., 50% data availability means the outputs of 50% samples are available in online test set, and the outputs of the remaining cannot be obtained, which indicates that both RPLS and LARPLS models cannot be updated at samples without outputs) from 100% to 34% had been investigated to mimic the common situation in industrial practice that the measurement of output is delayed or even missed, so that the RPLS and LARPLS models cannot be updated instantly. A direct comparison of the results from RPLS and LARPLS models is depicted in Figure 7 and tabulated in Table 5. From Table 5 and Figure 7, it is clearly seen that, in terms of both RMSEP and RRMSEP, the LARPLS modeling outperforms that based on RPLS; note especially the negative predicted values from RPLS in Figure 7A−C. 3.3. Application to a Propylene Polymerization Process. The industrial propylene polymerization15 was used herein to show the ability of LARPLS to model abnormal process states (process disturbances), arising from the significant changes in operation conditions after the 50th sample in the test set. 3.3.1. Data Set. The data set used in this example was adapted from Li, et al.15 The process involved two continuous stirred tank reactors (CSTRs) and two fluid bed reactors (FBRs). The first 8034



Article

Figure 6. Predicted results from LARPLS on simulated gas phase polyethylene data: (A) noise free data with different levels of noise; (B) 2% noise added to data; (C) 5% noise added to data; and (D) 8% noise added to data.

Table 4. Predicted Results from RPLS and LARPLS Models on Data Containing Different Amounts of Noise RPLS

LARPLS

models

0% noise

2% noise

5% noise

8% noise

0% noise

2% noise

5% noise

8% noise

RMSEP RRSMEP

32.82 10.04%

34.24 10.32%

38.96% 11.69%

50.81 13.29%

9.79 2.23%

13.23 2.69%

25.10 4.80%

36.80 7.14%

found in the conventional RPLS model. Two levels of adaptation, including local model adaptation and local time region adaptation, and three types of adaptive strategies, including adaptation of means and variances, use of an adaptive forgetting factor, and adaptive extraction of local time regions, have been presented in this work. These models have been shown to maintain the predictive performance of process models, even when the system showed significant variation over time. Although the LARPLS algorithm is presented in this work is based on sample-wise, recursive PLS, the LARPLS method can be extended to blockwise adaptation. This extension will be explored in a future publication. Catalyst activation in an industrial polymerization process was used as a case study to illustrate the effectiveness of the newly developed LARPLS models. The findings of the study show that

performance for the process with a significant disturbance, which can be attributed to its ability to capture process dynamics immediately. This improvement of predictive performance is derived from the assumptions in the LARPLS model discussed in Section 2.2 and shown in Figure 1C. Base on the findings of this work, it may be deduced that, by using an approach such as that taken in the LARPLS model, after localization of the historical data set, any model (such as PLS, RPLS, or ANN) can be applied to the best local time region to estimate the target process variable online.

4. CONCLUSIONS This paper demonstrated the improvements that can be made to a local model for dynamic systems modeling by using a local adaptive algorithm, LARPLS, to address a number of issues 8035



Article

Figure 7. Predicted results from RPLS and LARPLS on simulated gas phase polyethylene data with different data availabilities: (A) 66% data availability for RPLS; (B) 50% data availability for RPLS; (C) 34% data availability for RPLS; (D) 66% data availability for LARPLS; (E) 50% data availability for LARPLS; and (F) 34% data availability for LARPLS.

superior predictive performance could be achieved readily, not just in the LARPLS model, but also in the RPLS model when the

adaptation methodology used in the LARPLS algorithm is also used to adapt the RPLS model. Compared to other local models 8036



Article

Table 5. Predicted Results from RPLS and LARPLS Models with Different Data Availabilities RPLS

LARPLS

models

66% data

50% data

34% data

66% data

50% data

34% data

RMSEP RRSMEP

44.15 14.00%

47.56 15.04%

65.63 20.91%

22.42% 5.66%

17.39 4.36%

13.43 4.03%

Figure 8. Predicted results from RPLS and LARPLS models on propylene polymerization data: (A) predicted curve from RPLS; (B) absolute relative error (ARE) from RPLS; (C) predicted curve from LARPLS; and (D) absolute relative error (ARE) from LARPLS;.

measurement noise and delay of measurements than RPLS models. It is also demonstrated that the LARPLS modeling could effectively capture the process dynamics through its implementation to the propylene polymerization process including a significant process disturbance, where the change in operating conditions occurs. This confirms our assumption that localization based on the sample closest to the newly obtained sample, as used in LARPLS can effectively track the process dynamics. Although the LARPLS modeling method achieves better predictive performance for online prediction for three different types of processes, including one with slow changes (decay in catalyst activity) and one with a fast disturbance (sudden change in operating conditions), several issues could be explored further in future work. The first is that other alternative localization methods, such as clustering and covariance structure could be

Table 6. Predicted Results from RPLS and LARPLS Models on Propylene Polymerization Data models

RPLS

LARPLS

RMSEP RRMSEP

0.931 26.12%

0.0971 2.6%

published in the literature,28,29 LARPLS is simpler, easier to interpolate, and more efficient (with no need to adapt the local time region every time new samples are measured). A simulated gas phase polyethylene process was employed to investigate the effects of different levels of noise in measurements of the output variable and of different data availabilities, mimicking the delay in measurements of output variable, on the LARPLS data. The predicted results based on this simulated data show that LARPLS models are better able to handle 8037



Article

⎡ E ⎛1 1 ⎞⎤ k pi = k p0 exp⎢ − a ⎜ − ⎟⎥ ⎢⎣ R ⎝ T Tf ⎠⎥⎦

applied to model the dynamic system. Although it has been demonstrated in this work that LARPLS modeling, including adaptation strategy 1 for correcting drifts in mean and variance, generated good predictive performance for online prediction for three different data sets, other alternatives exist for adaptation, such as an adaptation based on changes in covariance. This alternative may be worth investigating in future work. Although the adaptation threshold is suggested as 0.1 in this work for all three cases, and good predictive performance is generated based on this preset threshold, it would be desirable if this threshold could be automatically generated or could be made adaptive. Finally, we note that T2 statistics and squared prediction error (SPE) Q statistics are commonly used to diagnose or detect a process fault. It has been demonstrated in Kadlec’s review paper37 that using T2 statistics and Q statistics multivariate statistical methods, such as PCA,18,19 are commonly applied to process monitoring and fault detection. Furthermore, Fujiwara et al.28 used these two as a tool to partition the historical data into subsets. Liu et al.36 developed a fast, moving-window PLS algorithm that is coupled with these two metrics to perform online prediction and, at the same time, to perform process evaluation. This coupling could also be implemented in LARPLS. Application of LARPLS method to perform online prediction and fault detection simultaneously will be explored in the future.

Cpg =

(A12)

Hg0 = (Fg + bt)Cpg(T − Tf )

(A13)

Hg1 = FgCpg(Tg1 − Tf )

(A14)

Hτ = HreacM W1R M1

(A15)

magnitude and units

param.

magnitude and units

ac AU B1 Bw Cpin Cpm1 Cppol Cpw Cv Fg Fw Hreac kd kpo

0.548 mol/kg 1.14 × 106 cal/K s 198.0 mol/s 7 × 107 g 6.9 cal/(mol K) 11.0 cal/(mol K) 0.85 cal/(g K) 18 cal/(mol K) 7.5 8500 mol/s 3.11 × 105 mol/s −894 cal/g 0.001 s−1 85 L/(mol s)

Kp Mg MrCpr Mw Mw1 Pv Psp Tf Tfeed Vg ΓY τ1 υp

5737 s−1 6060.5 mol 1.4 × 107 cal/K 3.314 × 107 g 28.05 g/mol 17.0 atm 7.67 L atm/s 360 K 293 K 5 × 105 L 1.8 × 103 mol/s 1500 s 0.5

■

AUTHOR INFORMATION

Corresponding Author

(A3)

*Tel.: (65)67905321. Fax: (65)67906620. Email: WDNI@ntu. edu.sg. Notes

dT dt

The authors declare no competing financial interest.

= Hf + Hgl − Hgo − Hr − OpCppol(T − Tf )

■

ACKNOWLEDGMENTS Funding support from the Nanyang Environment and Water Research Institute, Nanyang Technological University and her research partners are gratefully acknowledged. The writers are especially grateful to Dr. Petr Kadlec for providing the catalyst activation data set of polymerization process used in this paper. Dr. Li and Dr. McAuley are also especially appreciated for providing the data from the propylene polymerization process and the data from the simulated gas phase polyethylene process, respectively.

(A4)

Integral error for ethylene partial pressure controller: dIE = Psp − [M1]RT dt

(A5)

Ethylene partial pressure controller: ⎛ 1 ⎞ ΓM1 = BI + K p⎜Psp − [M1]RT + IE⎟ τ1 ⎠ ⎝

■

(A6)

The algebraic equations for the reactor model are shown as follows: bt = υpCυ ([M1] + [In])RT −Pυ

Hf = ΓM1Cpm (Tfeed − Tf ) + ΓlnCpln(Tfeed − Tf )

param.

(A2)

Reactor energy balance: M rCpr + B W Cppol

(A11)

Table A1. Parameters and Units for the Gas Phase Polyethylene Process

Catalyst active sites balance: Op dYi = ΓYi − kdiYi dt R

[M1] [In] Cpm + Cpln 1 [M1] + [In] [M1] + [In]

Table A1 lists the parameters and their units used in the simulations.

(A1)

d[M1] [M1] = ΓM1 − bt − R M1 dt [M1] + [In]

(A10)

1

Mass balance on monomers: Vg

(A9)

Op = R M1M w1

■

d[In] [In] = ΓIn − bt dt [M1] + [In]

(A8)

R M1 = [M1](kP1Y1 + k p2Y2)

APPENDIX: SIMULATION MODEL AND CONDITIONS The simulated process has been described by McAuley30,31 et al. The simulated model consists of mass balance, catalyst activity sites balance, and energy balance, as follows: Mass balance on inerts: Vg

i = 1, 2

bt ≥ 0

REFERENCES

(1) McAvoy, T. Intelligent ‘‘control’’ applications in the process industries. Annu. Rev. Control 2002, 26, 75−86. (2) Fortuna, L.; Rizzo, A.; Sinatra, M.; Xibilia, M. G. Soft analyzers for a sulfur recovery unit. Control Eng. Practice 2003, 11, 1491−1500.

(A7) 8038



Article

(3) Assis, J.; de; Filho, R. M. Soft sensors development for on-line bioreactor state estimation. Comput. Chem. Eng. 2000, 24, 1099−1103. (4) Russell, S. A.; Kesavan, P.; Lee, J. H.; Ogunnaike, B. A. Recursive data-based prediction and control of batch product quality. AIChE J. 1998, 44, 2442−2458. (5) Zhang, J. Inferential feedback control of distillation composition based on PCR and PLS models. Proceedings of the 2001 American Control Conference, IEEE, 2001; Vol. 2, pp 1196−1201. (6) Kaneko, H.; Arakawa, M.; Funatsu, K. Development of a new soft sensor method using independent component analysis and partial least squares. AIChE J. 2009, 55, 87−98. (7) Li, C.; Ye, H.; Wang, G.; Zhang, J. A recursive nonlinear PLS algorithm for adaptive nonlinear process modeling. Chem. Eng. Technol. 2005, 28, 141−152. (8) Li, W. H.; Yue, H. H.; Valle-Cervantes, S.; Qin, S. J. Recursive PCA for adaptive process monitoring. J. Process Control. 2000, 10, 471−486. (9) Rotem, Y.; Wachs, A.; Lewin, D. R. Ethylene compressor monitoring using model-based PCA. AIChE J. 2000, 46, 1825−1836. (10) Radhakrishnan, V.; Mohamed, A. Neural networks for the identification and control of blast furnace hot metal quality. J. Process Control 2000, 10, 509−524. (11) Johansen, T. A.; Shorten, R.; Murray-Smith, R. On the interpretation and identification of dynamic Takagi−Sugeno fuzzy models. IEEE Trans. Fuzzy Syst. 2000, 8, 297−313. (12) Sciascio, F. di.; Amicarelli, A. N. Biomass estimation in batch biotechnological process by Bayesian Gaussian process regression. Comput. Chem. Eng. 2008, 32, 3264−3273. (13) Brahim-Belhouari, S.; Bermak, A. Gaussian process for nonstationary time series prediction. Comput. Stat. Data Anal. 2004, 47, 705− 712. (14) Ažman, K.; Kocijan, J. Application of Gaussian processes for black-box modeling of biosystems. ISA Trans. 2007, 46, 443−457. (15) Helland, H. E. Recursive algorithm for partial least squares regression. Chemom. Intell. Lab. Syst. 1992, 14, 129−137. (16) Dayal, B. S.; MacGregor, J. F. Recursive exponentially weighted PLS and its applications to adaptive control and prediction. J. Process Control. 1997, 7, 169−179. (17) Qin, S. J. Recursive PLS algorithms for adaptive data modeling. Comput. Chem. Eng. 1998, 22, 503−514. (18) Wang, X.; Kruger, U.; Irwin, G. W. Process monitoring approach using fast moving window PCA. Ind. Eng. Chem. Res. 2005, 44, 5691− 5702. (19) Choi, S. W.; Martin, E. B.; Morris, A. J.; Lee, I. B. Adaptive multivariate statistical process for monitoring time-varying processes. Ind. Eng. Chem. Res. 2006, 45, 3108−3118. (20) Mu, S.; Zeng, Y.; Liu, R.; Wu, P.; Su, H.; Chu, J. Online dual updating with recursive PLS model and its application in predicting crystal size of purified terephthalic acid (PTA) process. J. Process Control. 2006, 16, 557−566. (21) Ahmed, F.; Nazier, S.; Yeo, Y. K. A recursive PLS-based soft sensor for prediction of the melt index during grade change operations in HDPE plant. Korean J. Chem. Eng. 2009, 26, 14−20. (22) Ni, W.; Brown, S. D.; Man, R. Stacked partial least squares regression analysis for spectral calibration and prediction. J. Chemom. 2009, 23, 505−517. (23) Ni, W.; Brown, S. D.; Man, R. Stacked PLS for calibration transfer without standards. J. Chemom. 2011, 25, 130−137. (24) Ni, W.; Brown, S. D.; Man, R. Data fusion in multivariate calibration transfer. Anal. Chim. Acta 2010, 661, 133−142. (25) Tan, H.; Brown, S. D. Dual-domain regression analysis for spectral calibration models. J. Chemom. 2003, 17, 111−122. (26) Amirthalingam, R.; Lee, J. Subspace identification based inferential control applied to a continuous pulp digester. J. Process Control 1999, 9, 397−406. (27) Kano, M.; Lee, S.; Hasebe, S. Two-stage subspace identification for soft sensor design and disturbance estimation. J. Process Control 2009, 19, 179−186.

(28) Fujiwara, K.; Kano, M.; Hasebe, S.; Takinami, A. Soft-sensor development using correlation-based just-in-time modeling. AIChE J. 2009, 55, 1754−1765. (29) Kadlec, P.; Gabrys, B. Local learning-based adaptive soft sensor for catalyst activation prediction. AIChE J. 2011, 57, 1288−1301. (30) Dadebo, S. A.; Bell, M. L.; McLellan, P. J.; McAuley, K. B. Temperature control of industrial gas phase polyethylene reactors. J. Process Control 1997, 7, 83−95. (31) McAuley, K. B.; Macdonald, D. A.; McLellan, P. J. Effects of operating conditions on stability of gas-phase polyethylene reactors. AIChE J. 1995, 41, 868−879. (32) Geladi, P.; Kowalski, B. Partial least-squares regression: a tutorial. Anal. Chim. Acta 1986, 185, 1−17. (33) Yoon, S.; MacGregor, J. F. Fault diagnosis with multivariate statistical models part I: using steady state fault signatures. J. Process Control 2001, 11, 387−400. (34) Lee, I.; Yang, J. Common clustering algorithms. In Comprehensive Chemometrics. Chemical and Biochemical Data Analysis; Brown, S. D., Tauler, R., Walczak, B., Eds.; Elsevier: Amsterdam, 2009. (35) Kalivas, J. H. Calibration methodologies. Comprehensive Chemometrics. Chemical and Biochemical Data Analysis; Brown, S. D., Tauler, R., Walczak, B., Eds.; Elsevier: Amsterdam, 2009. (36) Liu, J.; Chen, D.; Shen, J. Development of self-validating soft sensors using fast moving window partial least squares. Ind. Eng. Chem. Res. 2010, 49, 11530−11546. (37) Kadlec, P.; Gabrys, B.; Strandt, S. Data-driven soft sensors in the process industry. Comput. Chem. Eng. 2009, 33, 795−814.

8039


Localized, Adaptive Recursive Partial Least Squares Regression for

Recommend Documents