Anal. Chem. 1983, 55, 643-648 (5) (6) (7) (8)
Hubaux, A.; Vos, G. Anal. Chem. 1970, 42, 849-855. Ingle, J. D. J . Chem. Educ. 1974, 57, 100-105. Smith, E. D.; Mathews, D. M. J . Chem. Educ. 1967, 44, 757-759. Franke, J. P.; de Zeeuw, R. A.; Hakkert, R. Anal. Chem. 1978, 50, 1374-1 380. (9) Schwartz, L. M. Anal. Chem. 1977, 49, 2062-2068. (IO) Schwartz, L. M. Anal. Chem. 1979, 51, 723-727. (11) Graden, J. S.; Mitchell, D. G.; Mills, W. N. Anal. Chem. 1980, 52, 2310-23 15. (12) Rodbard, D. Anal. Biochem. 1978, 9 0 , 1-12. (13) Flnney, D.J.; Phillips, P. Appl. StatisNcs 1977, 2 8 , 312-320.
643
(14) Tway, P. C.; Wood, J. S., Jr.; Downing, G. V . J . Agrlc. FoodChem. 1981, 2 9 , 1059-1063. (15) Helwlg, J. T.; Council, K. A. “SAS Users Guide”; SAS Institute: Raleigh, NC, 1979. (16) Draper, N. R.; Smith, H. “Applled Regression Analysis”, 2nd ed.; Wiley: New York, 1981; pp 33-42.
RECEIVEDfor review September 20,1982. Accepted January 6, 1983.
Partial LeasbSquares Method for Spectrofluorimetric Analysis of Mixtures of Humic Acid and Ligninsulfonate Walter Llndberg* and Jan-Ake Persson Department of Analytical Chemistry, Unlversity of Urn&, S-901 87 Umea, Sweden
Svante Wold Research Group for Chemometrlcs, University of Umeh, 6-90 1 87 Ume.4, Sweden
Quantitative determinations of multicomponent fluorescent mixture have been made. The test substances used were humic acid, iigninsuifonate, and an optical whitener from a detergent. The fluorescence spectra from these substances have slmllar features with severe overlap In the whole wavelength reglon. For resolving these spectra and quantlfying the substances, we used a numerlcal method, the partial least squares in latent variables (PLS). The results of its application and the theory of the method are presented and a comparison Is made with other numerical methods.
Molecular fluorescence has attained popularity as an analytical technique due to its high sensitivity and relative selectivity. In complex samples, however, spectral overlap is often a serious limitation. In order to avoid time-consuming cleanup procedures, attempts to resolve complex spectra by using instrumental approaches (1) or various numerical methods have been made. The numerical methods used can be divided into two classes depending on whether all substances in the sample are known or not. If all substances in the sample are known and no interactions causing nonlinearities occur, then multiple regression (2, 3) is applicable. Individual contributions from each substances to a spectrum can be obtained and thereby their concentrations can be determined. However, usually these conditions are not fulfilled for unknown samples. In such cases, principal component analysis (4)has been employed for resolving the spectra obtained. With a related method called the partial leastsquares models in latent variables (PLS) developed by Wold and co-workers (5),a new approach to the multivariate calibration problem is demonstrated. This method is preferable to principal component analysis since it not only describes the emission matrix but also at the same time correlates the measured intensities with the concentrationsof each substance in the standard solutions. Ligninsulfonate, a compound released into water from sulfite pulp mills, contributes to the general pollution of seawaters and may be fatal to fish resources. This compound 0008-27n0/83/0351i-Q643$01.50/0
has earlier been determined with some success by using fluorescence spectrometry (6). With this method, possible interferences arise from humic acid and detergents containing optical whiteners. Emission spectra of these compounds are severely overlapped and no spectral region with a single emitting compound can be found. The aim of this work was to evaluate the prospectives of the PLS method for quantitative determinations in various mixtures of these compounds.
MATHEMATICAL METHODS Problem Formulation and Notation. We use boldface capital letters for matrices, e.g., X and Y,primes for transposed matrices, e.g., X’, boldface small characters for vectors, x9q,and ym, and italic small characters for scalars, e.g., s, pa,and xcke The samples are divided into two matrices, the calibration matrix, the training set, and the test matrix. the validation set We use the first set containing the samples 1, 2 , ~ . .i,, ..., pt to establish a calibration model which relates the measured concentrations Y , k of the constituents 1,2,..., k ,..., m to the measured spectroscopic data xcj (wavelength j = 1, 2, ...,p ) . As a first step we subtract the means %, and Qk from the data xCjand Yckr respectively, and then scale each variable xi and Y k to unit variance. The thus centered and normalized data are collected in the n X p matrix X and the n X m matrix Y. The centering makes the following computations numerically well conditioned. The normalization gives each variable (Leo,emission wavelength) equal influence in the initial stage of the data analysis. If prior information about the information content of the variables is at hand, this can be utilized by weighting the variables proportionally to this information. We note that this is not directly related to the precision of the variables, but rather a function of the degree of “nonoverlap” between the chemical constituents in the variable in combination with its precision. A priori information about this information content was not at hand in the present example and hence the variables were normalized to unit variance. Assuming that the spectroscopic data, X,and the concentrations, Y,are linearly related, we have the calibration model (Figure 1) with the m X p coefficient matrix C still to be 0 1983 American Chemical Society
644
ANALYTICAL CHEMISTRY, VOL. 55, NO. 4, APRIL 1983 I
Table I. Compositions of the Calibration Solutions (yg mL-’)
Y=XC E, +
-c
I _
m
P
C
Y
The data for the n calibratlon samples are contained in the spectroscopic matrix X and their concentrations in matrix Y. The callbration coefficients C are to be determined by the statistical analysis. Flgure 1.
X=TB F +
T B The matrix X is decomposed as a product of two matrices T and B plus “random” errors contained in F. Figure 2.
determined (E contains the ”random” residuals not explained by the systematic model). Traditional Methods. If the variables xj were independent of each other (i.e., little correlated) and the number of samples, n,was large compared to the number of wavelengths, p , the estimation of C is best made by ordinary least squares (OLS) with multiple regression (MR). However, in the present type of problems, the x variables are strongly correlated. This is so because each constituent in the samples emits light over the whole spectral range, i.e., influences all x variables. Hence these variables all go up and down when the concentration of one constituent changes. Moreover, the number of wavelengths, p , is much larger than the number of major constituents in the samples (2 or 3) which also makes the effective rank of X much smaller than p . These correlations among the x variables make the OLS solution, which requires the inversion of X’X, almost infinite which, in turn, gives the calibrated model bad prediction properties. The best solution to this statistical-numerical problem is to represent X by a product of two small matrices, T and B, of dimensions n X a and a X p , respectively, with a lO+llull then back to step 2, else convergence is reached and step 7 is taken next. 7. Compute the coefficient uh which relates the latent variable of X to the latent variable of Y uh = Uh’th + d = th’Uh/llthl11’2 8. Compute the loading for X,bhx uh
bh, 9. Compute residuals
th’X
E, = x - thbh,
647
E, = Y - Uhthbhy 10. For the next dimension, use E, instead of X and E,, instead of Y. Start again with step 1 above. 11. The Number of Dimensions, a . The number of significant dimensions, a, is determined by cross validation. A quarter of the calibration samples is kept out of the calculations in the hth dimension. The Y values of these samples are then predicted as thbhy vh by the calculated model and the partial sum of squares (SS) of the differences between predicted and actual values are formed. Another quarter of the sample is then kept out, a second partial SS computed, etc., until each calibration sample has been kept out once and only once. If the resulting SS (sum of the four partial SS) is smaller than the residual SS (of Y)before the hth dimension, the hth dimension has predictive relevance and is deemed to be statistically significant. The procedure is continued until a dimension (a 1) is nonsignificant. 111. Predictions of y ifrom the Calibrated Model and a “New” Sample Data Vector, xi. (i) Degree of Fit. The fit of the “new” sample to the model is measured as the SD (standard deviation) of the residual e ei = xi - xiB’B (xiB’ = ti)
-
+
llell/b - a) If the fit is acceptable, i.e., s2 < s,2 (E,)-F,the predicted values s2 =
y can be reliably computed by the algorithm shown below.
Here s:(E,) is the residual SD of the X matrix after dimensions a and F the F statistic at probability level a and (p -a)/2 and (p - a)(n- a - 1)/2 degrees of freedom. Algorithm for Prediction. 1. Scale and center X as in the calibration. 2. h = 0 y = 9 (average) 3. h = h + l th = XWh’
Y =Y X
+ Uhthbhy
= X - thbh,
4. If h > a, then terminate (step 5) or else go back to step
3.
5. The predicted values of y are now obtained in scaled and centered form. They are transformed back to the original coordinates by applying the reverse of the centering and scaling done in the calibration phase. Registry No. HzO, 7732-18-5;ligninsulfonic acid, 8062-15-5.
LITERATURE CITED “Modern Fluorescence Spectroscopy 4;” Plenum Press: New York, 1981. Warner, I . M.; Davldson, E. R.; Christian, G. D. Anal. Chem. 1977, 4 -9 ,. 2155. .- - . Gold, H. S.; Rechsteiner, C. E., Jr.; Buck, R. P. Anal. Chlm. Acta 1978, 103, 167. Ho, C.-N.; Christian, G. D.; Davidson, E. R. Anal. Chem. 1980, 52, 1071. Wold, S.; Wold, H.; Dunn, W. J., 111; Ruhe, A. Report UMINF-83, Deptment of Chemistry, University of Umea, Sweden, 1980. Almgren, T.; Josefesson, B.; Nyquist, G. Anal. Chlm. Acta 1975, 78, 411. Wold, S. Technometrlcs 1978, 20, 397. Mallnowskl. E. R.; Howery, D. G. ”Factor Analysis In Chemistry”; Wlley: New York, 1980. Goiub, G. H.; Kahan, W. J. Numer. Anal. 1985, 82,205. Kruskal, J. “Factor Analysis and Principal Component Analysis, The Bilinear Methods”; Encyclopedia of Statistics; The Free Press: New York, 1978. Gnanadeslkan, R. “Methods for Statistical Data Analysis of Multivariate Observations”; Wlley: New York, 1977. Draper, N.; Smith, H. “Applied Regression Analysis”, 2nd ed.; Wlley: New York, 1981. Kanal, L. IEEE Trans. Inform. Theory 1974, I T - 2 0 , 697. Massart, D. L.; Dijkstra, A.; Kaufman, L. “Evaluation and Optimization of Laboratory Methods and Analytical Procedures”: Elsevler: Amsterdam, 1978.
Anal. Chern. 1983, 55, 648-653
840
(15) Wold, S.; Martens, H.; Wold, H. In “Proceedings on the Symposium Matrix Penclls, Pltea 1982”; Ruhe, A,, KAgstrom, B., Eds.; Lecture Notes in Mathematics. Springer Veriag: Berlin and Heidelberg, In press. (16) Nyquist, 0. Ph.D. Dissertation, Unlversity of Gothenburg, Sweden, 1979. (17) Wold, S.; Sjostram. M. In “Chemometrics: Theory and Application;”
Kowaiski, B. R., Ed.; American Chemical Society: Washlngton, DC, 1977; ACS Symposlum Series, No. 52.
RECEIVED for review September 13,1982. Accepted December 6,1982.
Digital Smoothing of Noisy Spectra Manfred U. A. Bromba* and Horst Zlegler FB 6-Angewandte
Physlk, Unlversltaf Paderborn, 0-4790 Paderborn, West Germany
I t Is shown what propertles a dlgltai low-pass filter should possess if only a few very general common features ilke symmetry, posltlvlty, and monotonlclty of the individual spectral lines are known a priori. The effect of smoothlng on such unknown line properties as shape, height, width, and posltlon Is Considered theoretlcaiiy to simpilfy the selection of sultable filter types In view of a good compromise between noise reduction and signal deformation.
it suffices to treat centered lines of unit height. Thus we define (sampled) Gaussiansg and Lorentzians h (of width /3 > 0) by (k = 0, fl, f2, ...)
&!PI= exp( h [ k ] = (1
Usually random noise is a permanent but nevertheless unpleasant componentof all kinds of spectra since it may alter important quantities such as the position, the height, or the width of a spectral line in an unpredictable way. Even spurious lines may be created by random noise. One way to simplify the interpretation of noisy spectra is smoothing by digital filtering. Besides deconvolution (resolution enhancement) and differentiation, smoothing seems to be the most frequent used type of filtering. The price to be paid for the reduction of noise (and hence a reduction of random errors) is the introduction of a systematic (predictable) line deformation error. There are numerous criteria concerning the total error (systematic& random) which allow the theoretical design of an optimal fiiter. Sometimes a parameter of a class of filters (e.g., Savitzky-Golay smoothing filters of fixed degree 2M (1-4)) may be adjusted to minimize the total error. Usually this parameter is called “filter width” and, in principle, controls the degree of smoothing. However, all these criteria need detailed a priori information about the signal, especially the line shape and the line width. (As a rule, line height and line position need not be considered betause of the linearity and the translational (or time) invariance of our filters.) Also some noise properties should be known. While in most cases the assumption of white noise meets the real world sufficiently well, the line width may vary over a wide range, dependent on the physics of the line, the scan speed, and the sampling rate. Different line shapes and line widths may even occur in the same spectrogram, thus making the choice of an optimal filter impossible. Furthermore, in several cases an a priori determination of the line shape and the line width is very difficult or time-consuming so that trial-and-error methods have gained a great popularity. The intention of this paper is to show what properties a digital filter should or should not possess if only a few very general common properties of the individual lines of a spectrum are known a priori. Lines having these properties will be called “spectrometric functions” throughout this paper. We only consider the effect of smoothing on single lines. Since digital filters are linear and translationally invariant, 0003-2700/83/0355-0648$01.50/0
+
;)
$)
Most spectral lines can be represented fairly well by Gaussians and Lorentzians. The Fourier transform and further formulas concerning sums and square sums of these functions arb reviewed in the Appendix. We have chosen the following common properties of Gaussians (f = g) and Lorentzians (f = h),which are valid for every width /3 > 0, to define a spectrometric function f : (a) f is symmetric f[k] = f [ - k ] every 12
(3)
(b) f is absolutely summable llflll
5
= k=-m If[kll
(4)
O0
(c) f is strictly monotonic
f[k]> f[k
+ 11
every k 1 0
(d) The Fourier transform o f f is strictly positive m
3 f ( w ) = C f [ k ]exp(-iok)
>0
every IwI 5
7r
k=-m
From properties (a)-(d) we can deduce further properties of spectrometric functions: (e) f is strictly positive
f [ k ] > 0 every k
(7)
(f) 3f has a maximum a t o = 0
S f ( 0 )> 3 f b ) (0
1 0 15
a)
(8)
(e) is a direct consequenceof (a)-(c). (f) results from (e) and can be shown using eq 6 to establish the low-frequency character of spectrometric functions. Moreover, regarding elementary properties of alternating series, we can derive the following inequalities from eq 6: f[OI - 2f[21 < 3 f ( P / 2 ) 0 1983 Amerlcan Chemical Soclety
f[OI
(9)
