1876
Anal. Chem. 1981, 53, 1676-1878
Signal-to-Noise Ratio in Higher Order Derivative Spectrometry T. C. O’Haver” and T. Begley Department of Chemistry, University of Maryland, College Park, Maryland 20742
The effect of higher order differentlation and sliding-average smoothing on the signal-to-noise ratio of noisy peak-type signals Is investigated by means of a simple error-propagation calculation. The importance of smoothing is found to be much greater In the higher order derlvatives than In the zeroth order. With sufficient smoothing, hlgh-order derivatives can be obtained with only modest signal-to-noise ratio degradation compared to the zeroth order. The effect of scan rate in derivative spectrophotometry Is discussed.
The mathematical differentiation of experimental signals is often used as a resolution enhancement technique, to facilitate the detection and location of poorly resolved components in a complex spectrum, and as a background compensation technique, to reduce the effect of variable spectral background in quantitative spectrophotometry. It is widely appreciated that differentiation degrades signal-to-noiseratio (SNR) and that some form of smoothing or low-pass filtering is required in conjunction with differentiation. Previous reports on the SNR of derivative spectra have not dealt explicitly with the optimization of smoothing (I, 2). This paper considers the effect of differentiation and sliding average smoothing of Gaussian bands with white noise. Although the sliding-average smooth is only one of several types of smoothing function which might be applied, its simplicity lends itself to a straightforward analysis and its computational speed causes it to be a practical choice for analytical laboratory applications. It is assumed here that the differentiation and smoothing operations are separately and successively applied to an array of stored data. The effect of repeat differentiation and smoothing is equivalent to the use of a single combined convolution function. Although the use of such a combined function is faster in execution, the use of separately applied sample operations is more versatile and is in fact the approach used by several commercial data processing systems available to the analyst. Consider that the experimental signal which is to be differentiated has been measured by a discrete sampling technique and consists of a series of amplitudes taken at discrete, equally spaced wavelength or time increments: a l , a,, a3,u4, ...,where al represents the amplitude at wavelength or time 1,and so on. In order to determine the effect of differentiation on the noise, consider that this series consists only of noise, i.e., that there is no signal (signal will be treated separately later). We further assume that the noise has a white spectral distribution and a Gaussian amplitude probability distribution with zero mean and that each element in the series is statistically independent of its neighbors. The noise in this series may therefore be expressed as the standard deviation of all the elements in the series, which we will give the symbol go, the subscript referring to the “zeroth” order derivative. The nth derivative of this signal, calculated by the simplest method of successive differences, consists of a linear combination of n + 1 elements from the original data series n n! din = mC=(-l)m ~ ( n - rn)!rn!ai+n
Table I. Relative Signal-to-NoiseRatios of Unsmoothed Derivatives band shape a derivative order Gaussian Lorentzian 0 1 2 3 4 5 6 7 8 a
1 2.02/w 3.26/W2 8.1/W3 17.7/w4 52/W5 141/W6 478/W7 1675/W8
1
1.84/W 4.1/w2 16.6/W3 64/W4 390/Ws 22041W 17065/W7 132275/W8
W = number of points in the fwhm of band.
where din is the ith element of the nth derivative. The standard deviation of the nth order derivative, un, can be calculated by the usual rules for error propagation and is simply the weighted quadratic sum of the standard deviation of its terms
In order to calculate the magnitude of the signal, it will be mathematically convenient to consider two representative simple band shapes, Gaussian and Lorentzian, since these shapes are typical of the type encountered in practical spectroscopy and chromatography, and since their derivatives up to order 10 have been evaluated analytically and are in the literature (3). We define the “signal” as the difference between the largest (most positive) and the smallest (most negative) value of the derivative. Relative signal-to-noise ratios defined in this way for the first eight derivatives of Gaussian and Lorentzian bands are listed in Table I where W is the number of points in the peak full-width at half maximum (fwhm). The effect of smoothing is to reduce both the noise and the signal ( 4 ) . The signal attenuation is easily evaluated by smoothing noiseless computer-generated model band shapes, and the noise reduction is cal.culated by the usual rules for statistical error propagation. A first derivative smoothed by one pass of an N-point sliding average smooth has the general form
and a standard deviation of 21/2uo/N. In general, n passes of an N-point sliding average smooth of an nth derivative is equivalent to differentiation with an increment of N points. This was the only type of noise reduction calculation considered in Cahill’s treatment (2). It seems probable that this does not represent the best trade-off between signal attenuation and SNR improvement. The noise reduction is related to the number of original data elements which are averaged in computing each element of the smoothed series, whereas the signal attenuation is related more to the width of the convolution function. For the first derivative, for example, and only two elements of the original data series (e.g., a l ) are used to compute each element of the smoothed first
0003-2700/81/0353-1876$01.25/00 1981 American Chemical Society
ANALYTICAL CHEMISTRY, VOL. 53, NO. 12, OCTOBER 1981
1877
1.0
4
I
0.8
9
8
7
6
5
4
.3
0.2
2
I
DISTORTION FACTOR
I
0
Flgure 1. Predicted effect of the number of passes of a sliinig average smooth on the signal-to-noise ratio and attenuation factor of tVle second derivative of a Gaussian lbond. Smoothing ratio is the hidden Independent variable. Note that three passes always give a bettelr trade-off between attenualion and signal-to-noise ratio than one or two passes.
1
1
.2
1
1
.4
1
1
.6
1
1
1
.8
1
1
1.0
SMOOTHING RATIO
Figure 2. Predicted effect of smoothing ratio on the attenuation factor of the zeroth through sixth derlvatives of a Gaussian band smoothed by n 1 passes of (1 sliding average smooth.
+
derivative, but they are separated by W elements. Thus, as N is increased, the attenuation increases but the number of
3
data elements used remains constant at two. It would seem better if all data elements within the width of the convolution function (e.g., between U N + and ~ al) weire used, as this would further reduce the noise without increasing the attenuation significantly. ?’hisis eady done by passing an N-point smooth through the series a total of n + 1 times. It can be shown that the standard deviation of an nth derivative smoothed by n + 1 passes of an N-point sliding average smooth is given by
A
t
LL
5
W
W
a
where n is the derivative order and N is the number of points in the smooth. By comparison to eq 1, the noise is seen to have been reduced by a factor of Nh+o.6relative to the unsmoothed derivatives. Enke and Nieman (4) have shown that the signal-to-noise improvements achieved by using multiple passes of u smooth were no greater than could be achieved by a single pass of a smooth of greater width. In order to determine whether this is true also for derivative signals, we prepared plots of signal attenuation vs. signal-to-noise ratio, with smoothing ratio as the “hidden” parameter. An example of such a plot is shown in Figure 1 for the second derivative of a Guassian band smoothed with one, two, and three passes of a sliding average smooth. It is clear that, even with very large smoothing ratios, one or two passes cannot equal three passes in terms of the SNR/attenuation trade-off. Four passes offer no significant improvement. In general, n + 1passes seem optimum, were n is the derivative order. The peak height attenuation for the drst six derivatives o€ a Gaussian band with n + 1passes of sliding average smooth are shown in Figure 2 as a function of smoothing ratio. The effect of smoothing on signal-to-noise ratio can now be calculated in the following general way. ‘The relative SNR of the nth derivative without smoothing is given by an expression of the form (SNR), = C, (SNR)o Wn
0
10.1
0.2 0 3 0.4 0.5 0 6 0 7 0.8 09 1.0
SMOOTHING RATIO
Figure 3. Predicted eiffect of smoothing ratio on the relative signalto-noise of the Oth, lst, 2nd, 4th, 6th, and 8th derivatives of a Gaussian band, relative to the signal-to-noise of the unsmoothed zeroth derivative for n 1 passes of EL sliding average smooth. The Gaussian has a fwhm of 8 points.
+
where C, is a constant which depends on the derivative order n and on the band shape (Table I) and where (SNR)ois the SNR of the unsmoothed zeroth derivative. Smoothing with a sliding average smooth reduces the noise by the factor Nn+‘.6, where N is the smooth width (eq 2), but reduces the signal by the attenuation factor CY, given in Figure 2. Thus the relative SNR of ths smoothed nth derivative is
(3) where CY, = attenuation factor of the nth derivative for a smoothing ratio r. Values of (SNR),/(SNR), as a function of smoothing ratio are given in Figure 3 for the derivatives of an eight point wide Gaussian band (W:= 8) smoothed by n + 1passes of a sliding average smooth. Note that the signal-to-noise ratios of all the derivatives including the zeroth tend to converge at high
1878
ANALYTICAL CHEMISTRY, VOL. 53, NO. 12, OCTOBER 1981
smoothing ratios. (In fact, the SNRs ultimately reach a maximum at smoothing ratios above 1.0.) Thus, it is possible to say that the commonly observed degradation of SNR accompanying differentiation is not a necessary consequence and can be avoided if sufficient smoothing is employed. In practice, however, such large smoothing ratios should never be used, because serious peak height attenuation results, particularly in the higher derivative orders. Just how significant this peak height attenuation is depends on the purpose of the derivative measurement. If it is desired for some reason to measure the absolute value of a derivative, then any attenuation is a direct error in the measurement and must be minimized by using small smoothing ratios. Absolute measurements are however not ordinarily made in analytical applications. In a typical application in quantitative analysis involving the use of calibration standards, the peak height attenuation would not in itself lead to a measurement error, inasmuch as the same degree of differentiation and smoothing would be used for both sample and standard measurements. Nevertheless, smoothing ratios much above about 0.5 should not ordinarily be used because of the peak broadening caused by smoothing. From an inspection of Figures 2 and 3 we can make the following observations: (1)The achievable relative SNR improvement (relative to the SNR without smoothing) is much greater for the derivatives than for the normal (zeroth derivative) spectrum. Had the peak width W been larger than 8, the difference would have been even more dramatic, as implied by eq 3. (2) With sufficient smoothing, the SNR of the derivatives can exceed that of the unsmoothed normal spectrum (relative SNR = l.O), although this occurs at progressively greater attenuation as the derivative order increases. (Ultimately, the SNR of the derivatives can exceed that of the normal spectrum smoothed to optimum SNR, but this occurs only a t rather drastic attenuations.) (3) Generally, the SNR decreases at progressively higher derivative orders, but the extent of the SNR degradation depends strongly on the smoothing ratio. The previously stated generalization (1) that the SNR decreases by a factor of 2 for each stage of differentiation is true only for a smoothing ratio of about 0.25, which is rather smaller than the largest smoothing ratio which may be practical in many cases.
Equation 3 also allows us to explain the effect of scan rate on signal-to-noise ratio when the derivative technique is used in dispersive spectrophotometry. The width W of the measured spectral peak, here expressed as the number of points in the fwhm, is determined by the spectral scan rate and by the data sampling rate. It can be seen from eq 3 that, if the smoothing ratio is held constant, the SNR is proportional to WI2for all deriuatiue orders. If the scan rate is changed at a constant data sampling rate, W changes inversely with scan rate and thus SNR is inversely proportional to (scan If the data rate is changed in proportion to the scan rate, W remains constant but the averaging time varies inversely with the data rate and noise varies with the square root of the averaging time. So in this case, too, SNR is inversely proportional to (scan rate)'J2. The same holds for real-time differentiators as well; as the scan rate is changed and the system response time is varied to keep the smoothing ratio constant, the bandwidth varies directly with scan rate, and the noise varies inversely with the bandwidth. This result is significant because it is contrary to the expectation based on signal amplitude alone; if the scan rate is increased, the derivative output signal increases, but if the smoothing ratio is kept constant, the SNR decreases. On the other hand, because one generally uses smoothing ratios below the maximum SNR, increasing the scan rate at a constant differentiation response time will increase SNR because the smoothing ratio is increased. We therefore conclude that, in derivative just as in conventional zeroth-order spectrometry, slow scan rates and correspondingly long response times improve signal-to-noise ratio. ACKNOWLEDGMENT
The authors thank A. F. Fell of the Herriot-Watt University, Peter Gans of the University of Leeds, and Timothy Nieman of the University of Illinois for their helpful comments and suggestions. LITERATURE CITED (1) O'Haver. T. C.; Green, G. L. Anal. Chem. 1976, 48, 312. (2) Cahill, J. E. Am. Lab. (Fairfield, Conn.) 1979, 11, 79. (3) Fell, A. F. UV Spectrom. Group Bull. 1980, No. 8 , 5. (4) Enke, C. G.; Nieman, T. A. Anal. Chem. 1976, 48, 705A.
RECEIVED for review March 18, 1981. Accepted July 8,1981.
