Nonlinear Multivariate Quality Prediction Based On OSC-SVM-PLS

Subscriber access provided by UNIV OF LOUISIANA

Process Systems Engineering

Nonlinear Multivariate Quality Prediction Based On OSC-SVM-PLS Xiang Li, Feng Wu, Ridong Zhang, and Furong Gao Ind. Eng. Chem. Res., Just Accepted Manuscript • DOI: 10.1021/acs.iecr.8b06079 • Publication Date (Web): 27 Apr 2019 Downloaded from http://pubs.acs.org on April 28, 2019

Just Accepted “Just Accepted” manuscripts have been peer-reviewed and accepted for publication. They are posted online prior to technical editing, formatting for publication and author proofing. The American Chemical Society provides “Just Accepted” as a service to the research community to expedite the dissemination of scientific material as soon as possible after acceptance. “Just Accepted” manuscripts appear in full in PDF format accompanied by an HTML abstract. “Just Accepted” manuscripts have been fully peer reviewed, but should not be considered the official version of record. They are citable by the Digital Object Identifier (DOI®). “Just Accepted” is an optional service offered to authors. Therefore, the “Just Accepted” Web site may not include all articles that will be published in the journal. After a manuscript is technically edited and formatted, it will be removed from the “Just Accepted” Web site and published as an ASAP article. Note that technical editing may introduce minor changes to the manuscript text and/or graphics which could affect content, and all legal disclaimers and ethical guidelines that apply to the journal pertain. ACS cannot be held responsible for errors or consequences arising from the use of information contained in these “Just Accepted” manuscripts.

is published by the American Chemical Society. 1155 Sixteenth Street N.W., Washington, DC 20036 Published by American Chemical Society. Copyright © American Chemical Society. However, no copyright claim is made to original U.S. Government works, or works produced by employees of any Commonwealth realm Crown government in the course of their duties.

Page 1 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Industrial & Engineering Chemistry Research

Nonlinear Multivariate Quality Prediction Based On OSC-SVM-PLS Xiang Li1, Feng Wu1, Ridong Zhang1,2* , Furong Gao2

1.

The Belt and Road Information Research Institute, Hangzhou Dianzi University, Hangzhou 310018, P. R. China

2. Department of Chemical and Biomolecular Engineering, Hong Kong University of Science and Technology, Hong Kong Corresponding author: Ridong Zhang Email: [email protected]

Abstract: Predicting data with noise and high correlation using latent structural projection or partial least squares (PLS) is a commonly used linear regression method. In order to solve the nonlinear problem existing in the industrial processes, many methods for extending the PLS to the nonlinear space have been proposed. In this paper, the support vector machine (SVM) is integrated into the PLS to get a nonlinear PLS model. In order to reduce information in process variables that are independent of quality variables, a orthogonal signal correction (OSC) method has been introduced. Nonlinear data from industrial processes is used to build nonlinear PLS models and SVM methods are adopted to approximate nonlinear internal relationships between the input and the output. The model proposed in this paper can predict industrial process quality variables and improve prediction accuracy compared with traditional PLS. Key words ： Partial least squares, Orthogonal signal correction, Support vector machine, Process prediction

1

ACS Paragon Plus Environment

Industrial & Engineering Chemistry Research 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60

Page 2 of 23

1. Introduction As modern industries continue to develop, more and more sensors are being used in industrial processes, and we can know more and more process information. Nonetheless, in process control systems, there are still some important quality variables that cannot be measured in the real time fashion (such as the composition of the product in the distillation column, the temperature of the molten iron in the steel plant, and the concentration of the reactants in the chemical reaction). It is clear that the values of these quality variables can seriously affect the product quality and the production safety. Joseph and Brosillow proposed inferential control to solve this problem 1. The main idea of inferential control is to use the easily measurable variables to predict the output of the main quality variables that are not easily measurable, where they are correlated. This kind of idea has caused widespread concern, and thus derived a large number of online estimation and prediction methods that can be used in practical industrial processes, such as neural networks 2-3 and multivariate statistical methods. A major goal of industrial process data analysis is to build regression models using experimental or historical data and then predict product quality 4. The high dimensional nature of process data makes it difficult to accurately measure product quality and requires the use of process data to describe the final product quality. In order to deal with this situation, a multivariate prediction method, partial least squares (PLS), emerged 5. PLS is one of the most commonly used methods for predicting multivariate statistical processes. PLS is recognized as a powerful method for industrial process system model creation, process quality prediction and fault diagnosis due to its ability to process large quantities of industrial process data with variable correlations 6-8. The common PLS algorithms can be referred to in earlier literature

9-10.

However, the basic PLS can only establish a linear model, and industrial

processes are usually highly nonlinear whose dynamics change with time and contain uncertainty. It is not easy to establish a mathematical description of an industrial process by observing the process variables in real time. Due to the lack of sensors that meet the requirements, it is usually necessary to collect process data first and then perform 2


Page 3 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


offline modeling. However, due to the delay in offline modeling and the inability to sample in real time, this method cannot meet the real-time monitoring and control of actual industrial processes. In order to solve the nonlinear and highly correlated problems of industrial processes, many modified PLS methods have been produced in recent years

11-15.

For traditional linear PLS, a linear function can be used to describe the internal relationship 16-18.

between the input variable and the output variable

However, for industrial processes with strong

nonlinearities, this linear approximation usually does not produce satisfactory results. The nonlinear PLS model is more suitable for capturing the behavior of the process than the linear model. The nonlinear PLS proposed by Wold et al. uses quadratic or spline functions to simulate internal relations

19.

Unfortunately, Wold's nonlinear PLS

algorithm is only suitable for functions that can be approximated as linear, such as quadratic functions. For other non-linear relationships, such as logarithms, exponents, reciprocal functions, etc., nonlinear iterative partial least squares (NIPALS) cannot be executed. The use of artificial neural networks instead of linear internal relations is another nonlinear PLS modeling method 7. In fact, ANN can approximate all nonlinear functions. However, in practical applications, there are some shortcomings, such as local minimum a, the topology of ANNs and the phenomenon of "overtraining". ANN-based PLS is difficult to show good performance through fewer training samples 23-24.

20-22.

There is also a method for dealing with the nonlinearity of process data, called kernel PLS (KPLS)

KPLS is different from the above-mentioned method of nonlinearly approximation. It maps the original

process input data to the feature space of arbitrary dimension through nonlinear kernel function, and then creates a linear PLS model in the feature space. KPLS can effectively calculate the regression coefficients in high-dimensional space using nonlinear kernel functions. Compared with other nonlinear methods, KPLS can use the corresponding kernel function to avoid nonlinear optimization. Because different nonlinear kernel functions can be used, KPLS can handle different nonlinear data, and KPLS basically only needs to perform linear calculations. Based on these advantages, KPLS exhibits better performance than PLS for regression and prediction of Gaussian

3



Page 4 of 23

data of nonlinear systems. Kernel methods are becoming more widely used in industrial processes. However, when applied to process data with non-Gaussian property, there are still cases that do not meet expectations 25-29. SVM is a machine learning method using statistical learning theory. It is suitable for solving data with small sample size, high nonlinearity, high dimensionality and local minimum

30.

Due to its high precision and good

generalization capability, SVM has been used in machine learning, regression prediction, classification tasks, and pattern recognition 31. Based on the OSC-PLS model, this paper uses the SVM algorithm to approximate the nonlinear internal relationship between process variables and quality variables. The OSC-SVM-PLS model can be used for quality prediction in industrial processes. The method proposed in this paper is applied to a numerical simulation example and an actual industrial process to illustrate the superiority. This paper is organized as follows: The second part briefly introduces the basic PLS, and then introduces the newly proposed SVM-based OSC-PLS model in detail. In the third part, the proposed algorithm is applied to two examples, and the performance of the proposed method and the existing method are compared in detail. Finally, the fourth part is summarized.

2.Methods 2.1 PLS Algorithm The parameters of PLS models are usually calculated using nonlinear iterative partial least squares (NIPALS). The NIPALS algorithm is an iterative calculation method that can modify the feature vector and transform the elements to gradually improve the regression effect. This iteration is to calculate the feature vector of 𝑋 using the feature vector of 𝑌, and then calculate the feature vector of 𝑌using the feature vector of 𝑋. Assuming that the data matrices 𝑋 and 𝑌 have been normalized, the specific steps of the NIPALS are as follows:

4


Page 5 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Algorithm1:The PLS Algorithm Input : 𝑋,𝑌 Output : 𝑡,𝑢 1: X0 = 𝑋,𝑌0 = 𝑌 2: Set 𝑢 equal to the first column of 𝑌. 3: Get the input weight vector:𝑤 = 𝑋𝑇0𝑢/(𝑢𝑇𝑢) Normalize 𝑤 by 𝑤 = 𝑤/||𝑤|| 4: Calculate 𝑡 = 𝑋0𝑤 5: Calculate 𝑞 = 𝑌𝑇0𝑡/(𝑡𝑇𝑡) Normalize 𝑞 by 𝑞 = 𝑞/||𝑞|| 6: Calculate Output score vector:𝑢 = 𝑌0𝑞 7: Repeat steps 3-6 until Convergence. 8: Calculate the output load vector:𝑝 = 𝑋𝑇0𝑡/(𝑡𝑇𝑡) 9: Calculate internal regression coefficients: 𝑏 = 𝑡𝑇𝑢/(𝑡𝑇𝑡) 10: Calculate residual matrix:𝐸 = 𝑋0 ― 𝑡𝑝𝑇 11: Repeat the above process A times, A indicates the number of feature vectors. The number of iterations of the NIPALS algorithm represents the number of feature vectors taken, that is, the new data space dimension after projection. When using the PLS method to build a process model, since the modeling data contains redundant information, only the first A feature vectors need to be selected. Excessive feature vector selection amplifies noise and reduces process monitoring performance. 2.2. Orthogonal Signal Correction Algorithm Since Wold proposed orthogonal signal decomposition, the academic community has proposed a variety of

5



other orthogonal signal decomposition methods

32-38.

Page 6 of 23

This paper uses the original orthogonal signal decomposition

algorithm. Orthogonal signal decomposition is similar to NIPALS and is often used in principle component analysis (PCA) and PLS 39. The orthogonal signal decomposition is summarized as follows 40: Algorithm2:The OSC Algorithm Input : 𝑋,𝑌 Output : 𝑋𝑂𝑆𝐶 1: Standardize original data matrices 𝑋 and 𝑌. 2:Use PCA to get the principal component of 𝑋 and make 𝑡0 be the PC1. 3: Obtain 𝑡𝑛𝑒𝑤 from 𝑌 : 𝑡𝑛𝑒𝑤 = (𝐼 ― 𝑌(𝑌𝑇𝑌)

―1 𝑇

𝑌 )𝑡0

4: Obtain the PLS weight vector 𝑤:𝑋𝑤 = 𝑡𝑛𝑒𝑤 5: Calculate a new vector 𝑡0 from 𝑋 and 𝑤: 𝑡0 = 𝑋𝑤 6: Repeat steps 3, 4, and 5 until 𝑡0 until the convergence condition is met. 7: Get the loading vector 𝑝0:𝑝0 =

𝑋𝑇𝑡0 𝑡𝑇0𝑡0

8: Remove the “correction” from X to calculate the residuals 𝑋𝑂𝑆𝐶:XOSC = 𝑋 ― 𝑡0𝑝𝑇0 The above OSC algorithm only removes an OSC component. In an actual large-scale industrial process, removing a component may not meet the actual needs. Then, the above algorithm needs to be repeated multiple times to remove multiple components until the 𝑋𝑜𝑠𝑐 that meets the requirements is obtained. The resulting 𝑋𝑜𝑠𝑐 is then used for process modeling and the test data set is used to verify the model's predictive ability. The OSC effectively removes information that is independent of the target parameters and greatly reduces the number of potential variables required to build the calibration model. Therefore, OSC filtered data provides a simpler calibration model with fewer potential variables, so the interpretation of the model becomes easier. 2.3. Support Vector Machine

6


Page 7 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


The SVM is to nonlinearly map training data to high-dimensional feature space and construct an optimal classification hyperplane.

Figure 1. The idea of SVM

In order to obtain a nonlinear mapping function𝜱( ⋅ ), a kernel function needs to be introduced in the SVM. According to Mercer's theory, the sum function corresponds to the inner product of the Hilbert space. In SVM, we use kernel functions 𝑲(𝒙𝒊,𝒙𝒋)to project data from a nonlinear space into a linear space. The corresponding classification function is 41-42: n

f ( x)  sgn{  i yi K ( xi , x)  b}

(1)

i 1

There are a variety of kernel algorithms that can be used for SVM. Here are a few commonly used kernel functions: (1) Linear kernel function

K ( xi , x)  xi  x

(2)

(2) Radial basis kernel function 2

x  xi K ( xi , x)  exp( ) 2 2

(3)

K ( xi , x)  ( xi  x  1) d , d  1, 2,L , N

(4)

(3) Polynomial kernel function

(4) Perceptron kernel function

K ( xi , x)  tanh(  xi  x  b) 7


(5)


Page 8 of 23

3. Multivarite Quality Prediction based on OSC-SVM-PLS The OSC-PLS model decomposes the input matrix 𝑋(𝑛 × 𝑁)and output data matrices𝑌(𝑛 × 𝑀) into a linear combination of the score matrix𝑇(𝑛 × 𝐴) and 𝑈(𝑛 × 𝐴), the load matrixP(𝑁 × 𝐴) and 𝑄(𝑀 × 𝐴), and the residual matrix 𝐸(𝑛 × 𝑁) and𝐹(𝑛 × 𝑀): A

X   ta pa  E  TPT  E

(6)

a 1 A

Y   ua qa  F  UQT  F

(7)

a 1

where A is the number of latent variables retained in the model. The vector 𝑡𝑎 and 𝑢𝑎 represent the a-th potential variable. Both 𝑋 and 𝑌 are normalized. The internal relationship between the latent variable 𝑇and 𝑈 is:

U  f (T )  H  [ f1 (t1 ), f 2 (t2 ),L f a (ta )]  [h1 , h2 ,L ha ] ua  f a (ta )  ha , a  1, 2,L A

(8) (9)

where ℎ𝑎 is the residual vector For linear PLS, 𝑓𝑎( ⋅ ) is a linear function. In this paper, nonlinear SVM regression functions 𝑓𝑎( ⋅ ) are used as a basis to construct a SVM-based PLS model. The standard SVM regression function is expressed as: n

uˆa   ( a  aa* ) K (ta ,i , ta )  b

(10)

a 1

Here, 𝐾( ⋅ ) is a known kernel function, and the regression coefficient

(𝛼𝑎 ― 𝛼𝑎∗ )

and constant 𝑏 can be

determined using the SVM algorithm 43. Figure 2 illustrates the OSC-SVM-PLS modeling approach. The proposed method can be used to calculate the corresponding 𝑡𝑛𝑒𝑤,𝐸𝑛𝑒𝑤 , 𝑌new and 𝐹𝑛𝑒𝑤 to the new data 𝑋𝑛𝑒𝑤:

tnew  X oscnewW ( pT W ) 1

(11)

Enew  X oscnew  tnew PT

(12)

Yˆnew  f (tnew )QT 8


(13)

Page 9 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Fnew  Ynew  Y

Figure 2a. The 1st latent variable

Figure 2b. The 2nd latent variable   

9


(14)


Page 10 of 23

Figure 2c. The last latent variable Figure 2. Schematic illustration for OSC-SVM-PLS modeling

3.1. Establishment of a model under normal conditions The schematic diagram of online quality prediction and process monitoring is shown in Figure 3. According to the strategy in the figure, the model under normal working conditions is first established. (1). Raw data are normalized to standard data with a mean of zero and a variance of one. (2). The OSC algorithm is executed on the standardized process data, and 𝑋𝑜𝑠𝑐is obtained. (3). For the𝑋𝑜𝑠𝑐 implementation of the PLS algorithm, the latent variables 𝑢 of the potential variables 𝑡 and 𝑌 of 𝑋 are obtained. (4). Execute the SVM algorithm for 𝑡 and 𝑢 to get the prediction model 3.2 Online prediction (1) Standardize new samples 𝑥𝑛𝑒𝑤and 𝑦𝑛𝑒𝑤 processes. (2) For 𝑥𝑛𝑒𝑤executing the OSC algorithm, get 𝑥𝑜𝑠𝑐𝑛𝑒𝑤 (3) Calculated 𝑡𝑛𝑒𝑤 based on the parameters obtained by modeling:𝑡𝑛𝑒𝑤 = 𝑥𝑜𝑠𝑐𝑛𝑒𝑤𝑅 (4) Obtain a predicted value 𝑌new of the quality variable based on the prediction model

10


Page 11 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 3. Schematic description for on-line quality variable prediction and process monitoring

3.3 Predicted performance indicator To illustrate the effectiveness of prediction methods, many indicators have been proposed to measure the performance, such as Mean Absolute Error (MAE), Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Percentage Error (MAPE), etc., and this paper uses RMSE as an indicator to test the prediction performance of different prediction algorithms. The RMSE is defined as follows 44: n

 ( yˆ  y)

RMSE 

2

i 1

n

(15)

where 𝑦 represents the value collected from the actual industrial process, 𝑦 indicates the corresponding predicted value, and 𝑛 represents the number of samples collected.

4.Case Studies 4.1 A Numerical Example. We use the nonlinear numerical simulation example to assess the efficiency of the proposed method. It is defined as follows:

x1  t 2  t  1  r1 x2  sin t  r2 11



Page 12 of 23

x3  t 3  t  r3 y  x12  x1 x2  3cos x3  r4 where 𝑡 is uniformly distributed over [-1,1] and 𝑟𝑖,𝑖 = 1,2,3,4 are noise components uniformly distributed over [-0.1, 0.1]. Generate 300 samples and divide them into training and test sets. The first 200 samples were used for training and the last 100 were used for testing. They are shown in Figures 4 and 5.

Figure 4. Numerical simulation data set

12


Page 13 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 5. Data set in a numerical simulation example

It is seen from Figure 6 that the response variable is nonlinearly related to the input variable. Figure 7 shows the results predicted using PLS with the real values(Y), predicted values(𝑌) and residual values(𝑌) of the training and prediction data. Figure 8 shows the results of prediction using traditional KPLS. It can be seen that the prediction method of the traditional KPLS algorithm can be able to improve the prediction ability. Figure 9 shows the prediction results of the proposed method. It can be seen that the prediction performance of the proposed prediction method is much better than that of the traditional PLS, and also better than the traditional KPLS algorithm.

13



Figure 6. The relationship between variables in numerical simulation

Figure 7. PLS prediction in numerical simulation

14


Page 14 of 23

Page 15 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure 8. KPLS prediction in numerical simulation

Figure 9. OSC-SVM-PLS prediction in numerical simulation

Although it can be seen that the proposed method has better prediction performance than the traditional PLS and KPLS prediction methods, in order to more accurately describe the prediction ability of the prediction method, this paper compares the different RMSE indicators of these prediction methods for further look at the prediction

15



Page 16 of 23

accuracy. Table 1 shows the RMSE values of the different prediction methods for the numerical simulation example. It can be seen that the RMSE value of the traditional PLS prediction method is 0.3259; the RMSE value of the KPLS method is 0.2774; and the RMSE value of the OSC-SVM-PLS method is 0.0875. This is consistent with the results we see from the figures and also proves that the proposed method is superior to the traditional PLS method and the basic KPLS method. Table1. RMSE values for different prediction methods in numerical simulation examples PLS

0.3259

KPLS

0.2774

OSC-SVM-PLS

0.0875

4.2 TE Example The Tennessee-Eastman process is a complex nonlinear simulation model based on actual industrial processes that has been widely used in process modeling and process prediction. The process consists of five main units: reactor, condenser, compressor, separation, and stripper. The process consists of eight material components, namely gas components A, C, D, E, inert insoluble materials B, reaction by-product F and liquid products G and H. The process consists of 41 measured variables and 11 control variables. In this paper, 22 measured variables and 11 operating variables are selected as the process variable 𝑋, and the emptying material component G in the component variables is selected as the quality variable. The TE process simulation model was used to generate 500 samples, in which the first 300 are for training models and the last 200 are for testing model prediction capabilities.

16


Page 17 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Figure10.Predict result via PLS in TE process

Figure11.Predict result via KPLS in TE process

17



Page 18 of 23

Figure12.Predict result via OSC-SVM-PLS algorithm in TE process

The PLS is used to model the TE process data. After calculation, 27 potential variables are retained. Figure 10 shows the results predicted by the PLS method. It is seen from the figure that the PLS prediction results are different from the original data set, and the prediction model cannot track data changes. Large fluctuations result in residuals beyond the acceptable range. Figure 11 shows the prediction results of the traditional KPLS algorithm. It can be seen from the figure that the addition of the kernel algorithm can greatly improve the prediction ability of the traditional PLS algorithm. Figure 12 shows the prediction results of the model after introducing SVM into OSC-PLS. It is seen from the figure that this model can better track the variation of process quality variables, and the residuals fluctuate within a small range, reaching the expected effect. Table 2 shows the RMSE values of different prediction methods for the TE process. As can be seen from the table, the RMSE value of the traditional PLS prediction method is 0.0575, the RMSE value of the traditional KPLS is 0.0184 and the RMSE value of the OSC-SVM-PLS proposed in this paper is 0.0149, which further shows that the proposed method is better than the traditional PLS and KPLS algorithms.

18


Page 19 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


Table 2. RMSE values of different prediction methods for TE processes PLS

0.0575

KPLS

0.0184

OSC-SVM-PLS

0.0149

5.Conclusion This paper proposes a PLS-based nonlinear multivariate quality estimation and prediction method. The method is to perform a linear PLS algorithm after de-noising the data, and then execute the SVM algorithm for the obtained parameters. This method effectively captures the nonlinear relationship between input and output variables. Compared with PLS and other nonlinear relationships, OSC-SVM-PLS avoids the nonlinear optimization process. It involves calculations as simple as the linear PLS method. The OSC method also reduces the number of principal components that need to be calculated, which extremely reduces the amount of calculations. The comparison of the predicted performance of OSC-SVM-PLS and PLS shows that if the relationship between the variables of industrial process data is nonlinear, linear algorithms should not be used for modeling, because the use of linear algorithms to deal with nonlinear data has the danger of containing noise. The effectiveness of the proposed method in numerical simulation and TE process demonstrates the effectiveness of the proposed method.

When using OSC-SVM-PLS,

there are many factors that deserve special consideration. One of them is how to choose the best kernel function in the SVM and choose the appropriate kernel parameters. In this article, we only rely on experience to get the required value. How to solve this problem using the system method is still worth considering.

Literature Cited (1) Joseph, B.; Brosilow, C. Inferential control of processes: Part I. Steady state analysis and design. Aiche J. 2010, 24(3):485-492. (2) Qin, S.; Mcavoy, T. Nonlinear PLS modeling using neural networks. Comput. Chem. Eng. 1992, 16(4):379-391. (3) Radhakrishnan, V.; Mohamed, A. Neural networks for the identification and control of blast furnace hot metal quality. J. Process Contr. 2000, 10(6):509-524.

19



Page 20 of 23

(4) Wold, S.; Trygg, J.; Berglund, A.; Antti, H. Some recent developments in PLS modeling. Chemometrics Intell. Lab. Syst. 2001, 58(2):131-150. (5) Wold, S.; Antti, H.; Lindgren, F.; Ohman, J. Orthogonal Signal Correction of Near-Infrared Spectra. Chemometrics Intell. Lab. Syst. 1998, 44(1):175-185. (6) Khan, A.; Moyne, J.; Tilbury, D. Virtual metrology and feedback control for semiconductor manufacturing processes using recursive partial least squares. J. Process Contr. 2008, 18(10):961-974. (7) Kresta, J.; Macgregor, J.; Marlin, T. Multivariate statistical monitoring of process operating performance. Can. J. Chem. Eng. 1991. 69(1):35-47. (8) Wise, B. The process chemometrics approach to process monitoring and fault detection. J. Process Contr. 1996, 6(6):329-348. (9) Helland, I. On the structure of partial least squares regression. Commun. Stat. Simul. Comput. 1988, 17(2):27. (10) Hoskuldsson, A. PLS regression methods. J. Chemometr. 1988, 2(3):211-228. (11) Frank, I. A nonlinear PLS model. Chemometrics Intell. Lab. Syst. 1990, 8(2):109-119. (12) Wold, S. Nonlinear partial least squares modelling II. Spline inner relation. Chemometrics Intell. Lab. Syst. 1992, 14(1-3):71-84. (13) Malthouse, E.; Tamhane, A.; Mah, R. Nonlinear partial least squares. Comput. Chem. Eng. 1997, 21(8):875–890. (14) Holcomb, T.; Morari, M. PLS/neural networks. Comput. Chem. Eng. 1992, 16(4):393-411. (15) Nandi, S.; Kulkarni, S.; Chaudhary, A.; Tambe, S.; Kulkarni, B. Modeling and Monitoring of Batch Processes Using Principal Component Analysis (PCA) Assisted Generalized Regression Neural Networks (GRNN). Biochem. Eng. J. 2004, 18(3):193-210. (16) Nomikos, P.; Macgregor, J. Monitoring Batch Processes Using Multi-way Principal Component Analysis. Aiche J. 2010, 40(8):1361-1375. (17) Cho, H.; Kim, K. A Method for Predicting Future Observations in the Monitoring of a Batch Process. J. Qual. Technol. 2003, 35(1):59-69. (18) Meng, X.; Morris, A.; Martin, E. On-line monitoring of batch processes using a PARAFAC representation. J. Chemometr. 2010, 17(1):65-81. (19) Birol, G.; Undey, C.; Inar, A. A modular simulation package for fed-batch fermentation: penicillin production. Comput. Chem. Eng. 2002, 26(11):1553-1565. (20) Zhang, R.; Tao, J. Data-Driven Modeling Using Improved Multi-Objective Optimization Based Neural Network for Coke Furnace System. IEEE T. Ind. Electron. 2017, 64(4):3147-3155.

20


Page 21 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


(21) Rosipal, R.; Trejo, L.; Matthews, B. Kernel PLS-SVC for Linear and Nonlinear Discrimination. Twentieth International Conference on International Conference on Machine Learning. AAAI Press, 2003. (22) Zhang, R.; Tao, J.; Lu, R.; Jin, Q. Decoupled ARX and RBF Neural Network Modeling Using PCA and GA Optimization for Nonlinear Distributed Parameter Systems. IEEE T. Neur. Net. Lear. 2018, 29(2):457-469. (23) Kim, K.; Lee, J.; Lee, I. A novel multivariate regression approach based on kernel partial least squares with orthogonal signal correction. Chemometrics Intell. Lab. Syst. 2005,79(1):22-30. (24) Rnnar, S.; Lindgren, F.; Geladi, P.; Wold, S. A PLS kernel algorithm for data sets with many variables and fewer objects. Part 1: Theory and algorithm. J. Chemometr. 1994, 8(2):111-125. (25) Li, R.; Wang, X. Dimension reduction of process dynamic trends using independent component analysis. Comput. Chem. Eng. 2002, 26(3):467-473. (26) Albazzaz, H.; Wang, X. Statistical Process Control Charts for Batch Operations Based on Independent Component Analysis. Ind. Eng. Chem. Res. 2004, 43(21):6731-6741. (27) Kano, M.; Tanaka, S.; Hasebe, S.; Hashimoto, I.; Ohno, H. Monitoring independent components for fault detection. Aiche J. 2003, 49(4):969-976. (28) Lee, J.; Yoo, C.; Lee, I. Statistical monitoring of dynamic processes based on dynamic independent component analysis. Chem. Eng. Sci. 2004, 59(14):2995-3006. (29) Lee, J.; Yoo, C.; Lee, I. Statistical process monitoring with independent component analysis. J. Process Contr. 2004, 14(5):467-485. (30) Zhang, R.; Wang, S. Support vector machine based predictive functional control design for output temperature of coking furnace. J. Process Contr. 2008, 18(5):439-448. (31) Cho, H.; Kim, K. A Method for Predicting Future Observations in the Monitoring of a Batch Process. J. Qual. Technol. 2003, 35(1):59-69. (32) Wang, Z.; Yuan, J. Online Supervision of Penicillin Cultivations Based on Rolling MPCA. Chinese J. Chem. Eng. 2007, 15(1):92-96. (33) Trygg, J.; Wold, S. Orthogonal projections to latent structures (O-PLS). J. Chemometr. 2010, 16(3):119-128. (34) Sjoblom, J.; Svensson, O.; Josefson, M.; Kullberg, H.; Wold, S. An evaluation of orthogonal signal correction applied to calibration transfer of near infrared spectra. Chemometrics Intell. Lab. Syst. 1998, 44(1):229-244. (35) Andersson, C. Direct orthogonalization. Chemometrics Intell. Lab. Syst. 1999, 47(1):51-63. (36) Fearn, T. On orthogonal signal correction. Chemometrics Intell. Lab. Syst. 2000, 50(1):47-52. (37) Fernandez, J.; Massarta, D.; Noord, O.; Ricouxc, P. Direct Orthogonalization: some case studies. Chemometrics

21



Page 22 of 23

Intell. Lab. Syst. 2001, 55(1-2):101-108. (38) Blanco, M.; Coello, J.; Montoliu, I.; Romero, M. Orthogonal signal correction in near infrared calibration. Anal. Chim. Acta 2001, 434(1):125-132. (39) Feudale, R.; Robert, N.; Tan, H.; Brown, S. Piecewise orthogonal signal correction. Chemometrics Intell. Lab. Syst. 2002, 63(2):129-138. (40) Geladi, P.; Kowalski, B. Partial Least-Squares Regression: A Tutorial. Anal. Chim. Acta 1986, 185(1):1-17. (41) Svensson, O.; Kourti, T.; Macgregor, J. An investigation of orthogonal signal correction algorithms and their characteristics. J. Chemometr. 2010, 16(4):176-188. (42) Platt, J. Using Analytic QP and Sparseness to Speed Training of Support Vector Machines. Advances in neural information processing systems, 1999, 11. (43) Sain, S. The nature of statistical learning theory. Technometrics. 1996, 38(4):409 (44) Zhang, R.; Tao, J. A nonlinear fuzzy neural network modeling approach using improved genetic algorithm. IEEE T. Ind. Electron. 2018,65(7):5882-5892.

22


Page 23 of 23 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60


TOC

23


Nonlinear Multivariate Quality Prediction Based On OSC-SVM-PLS

Recommend Documents