and used over long periods of time with the assumption that the relative response of the internal standard and analyte remain fixed. Use of an external standard electrode requires that conditions remain invariant during the time the sample and standard are run, while day-to-day or week-to-week variations could be expected to affect the response of the analyte US. internal standard (the RSF.) The current availability of
electrical detection and peak switching (IO) make the use of standard electrodes much more feasible even for larger numbers of elements. RECEIVED for review July 13, 1971. Accepted December 21, 1971. Work supported by LEAA Grant 154. (10) R. A. Bingham and R. M. Elliott, ANAL.CHEM., 43,43 (1971).
Digital Data Handling of Spectra Utilizing Fourier Transformations Gary Horlick Department of Chemistry, University of Alberta, Edmonton, Alberta Utilizing the information available upon Fourier transformation of spectra, several data handling operations are performed, including smoothing, differentiation, and resolution enhancement. These operations are carried out by appropriate simple modifications of the spacial frequency spectrum of the original spectrum. The spacial frequency spectrum is calculated by taking the Fourier transformation of the original spectrum. This calculation and the distribution of information in the spacial frequency spectrum are discussed and illustrated. Then the implementation of the above operations (smoothing, differentiation, and resolution enhancement) by utilization of the spacial frequency information i s described. In particular, this approach to spectral smoothing provides an effective way of maximizing the signal-to-noise ratio of a measurement.
ONE OF THE MAJOR developments in spectrometric measurements in recent years is the acquisition of spectral information in digital form. As technological developments make the so-called small computer more powerful, convenient, and inexpensive, this trend is sure to continue and expand. A major driving force in this development is the desire to perform various types of digital data handling on the digitized spectrum. Typical operations that have been performed on spectra include smoothing ( I ) , differentiation (2, 3), and resolution enhancement (4). A particularly powerful route for performing these and similar operations on spectra is through the utilization of Fourier transformations (5-9). In addition, this approach provides unique insight into the implementation and fundamental limitations of these techniques as the manner in which the available spectral information can be manipulated is clearly revealed. Data handling of signals utilizing Fourier transformations is not new. In particular, the concepts of smoothing were developed to a high degree of mathematical sophistication (1) A. Savitzky and M. J. E. Golay, ANAL.CHEM., 36,1627 (1964). (2) A. E. Martin, Specrrochim. Acta, 14,97 (1959). ( 3 ) J. R. Morrey, ANAL. CHEM., 40,905 (1968). (4) R. N. Jones, R. Venkataragharan, and J. W. Hopkins, Spectrochim. Acta, 23A, 925 (1967). ( 5 ) J. R. Izatt, H. Saki, and W. S. Benidict, J. Opt. SOC.Amer., 59, 19 (1969).
(6) Mihai Caprini, Sorin Cohn-Sfetcu, and Anca Maria Manof, IEEE Trans. Audio Electroacoustics, AU-18, 389 (1970). (7) T. Inouge, T. Harper, and N. C. Rasmussen, Nucl. Instrum. Methods, 67,125 (1969). (8) R. R. Ernst, “Advances in Magnetic Resonance, Vol. 2,” J. S. Waugh, Ed., Academic Press, New York, N.Y., 1966, p 1. 43, 1035 (9) D. W. Kirmse and A. W. Westerberg, ANAL.CHEM., ( 1971).
during the 1940’s by Norbert Wiener and applied to the design of radar receivers by many workers. However, the basic simplicity of data handling utilizing Fourier transformations is often lost in an excess of mathematical equations and in applications to arbitrary waveforms. It is important, in order to effectively utilize these techniques, to know and appreciate in a practical sense the types of information that are obtained upon Fourier transformation of real signals. The next section discusses this in detail for simple flame emission spectra. The main aim is to provide an intuitive feeling for the distribution of the spectral information in the Fourier domain. Smoothing, differentiation, and resolution enhancement of these spectra using Fourier transformations are then discussed and illustrated in the last section. EXPERIMENTAL All spectra were measured using a Heath EU-700 monochromator. The spFctral bandwidth of this monochromator is approximately 2 A at a slit width of 100 pm. The signal from the photomultiplier tube was converted to a voltage with a Heath EU-703-71 photometric readout module and the voltage was digitized with a DANA Model 5400/015 DVM. The spectra were sampled at 0.2-A intervals and the digitized points were punched directly onto cards. The Fourier transformations were calculated on an IBM 360 computer. All the transforms and plots are 128 points long. Thus the horizontal axis fqr the optical spectral plots have a normalized length of 25.6 A. The horizqntal axis for the spacial frequency plots extends from 0 to 2.5 A-I. Except where noted, the vertical axes are arbitrary. All the plots were drawn using the standard CALCOMP system. RESULTS AND DISCUSSION Information Obtained upon Fourier Transformation of a Spectrum. Fourier transformation is a technique for determining the frequency spectrum of a waveform. For the purposes of this paper, an optical spectrum is the waveform of interest. Carrying out a Fourier transformation on an optical spectrum results in a spacial frequency spectrum. The term “spacial frequency” refers to a frequency in the plane of the paper on which the optical spectrum is plotted and should be carefully distinguished from the term “optical frequency.” The spacial frequency spectrum has units that are the reciprocal of those used for the optical spectrum. Thus, if the original spectrum has units of A, the spacial frequency spectrum has units of A-1. ANALYTICAL CHEMISTRY, VOL. 44, NO. 6, MAY 1972
0
943
ORIGINAL OPTICAL SPECTRUM
-
l l
If
REAL OUTPUT
-
-
AMPLITUDE SPECTRUM OF THE SPACIAL FREQUENCIES
ll IMAGINARY CLTPUT
Figure 1. Outputs resulting from the Fourier transformation of a single line optical spectrum. (See text) The phase spectrum (1E)is modulo 2~
Most Fourier transformations are carried out using the so-called Fast Fourier Transform (FFT) (IO). This is simply an efficient algorithm for the calculation of the Fourier transformation of a set of points. Several versions of this algorithm have been programmed. The input to a typical program can be a set of real data (Le., a digitized optical spectrum) or a set of real and imaginary inputs. The output of a typical FFT program consists of two series, the real part of the transform [X(J)] and the imaginary part [Y(J)]. These two outputs can be used to generate two additional series, the amplitude spectrum of the spacial frequencies that make up the original optical spectrum and the phase spectrum of these spacial frequencies. The amplitudes of the spacial frequencies [A(J)]are calculated from the real and imaginary outputs by taking the root sum of squares of the two series, i.e.: The phases of these spacial frequencies [P(J)]are calculated using the following equation : P(J) = arctan CY(J)/X(J)] (2) All these outputs are illustrated in Figure 1 for a single line optical spectrum. Figure 1A is the original optical spectrum. This is a spectrum of the emission of 0.5 ppm Ca a t 4226.7 A in an OrH2 flame. The real output of the FFT for this input spectrum is shown in Figure 1B. It is simply a damped cosine wave. The frequency of this cosine wave depends on the position of the spectral peak with respect to the origin in the original spectrum, and the functional form of the damping depends on the line shape in the original spectrum (11). The imagi-
(10) R. S . Singleton, IEEE Trans. Audio Electroacoustics, AU-17, 166 (1969). (11) G . Horlick, ANAL.CHEM., 43 (8), 61A (1971). 944
ANALYTICAL CHEMISTRY, VOL. 44, NO. 6, MAY 1972
nary output (Figure 1C) is a damped sine wave with similar characteristics to the real output. The amplitude spectrum of the spacial frequencies is shown in Figure 1D. This amplitude spectrum indicates that the original optical spectrum is composed mainly of low frequency spacial frequencies including a relatively large dc level. The amplitudes of the higher frequency spacial frequencies are small but their presence is significant in that it is primarily these spacial frequencies that make up the noise in the original optical spectrum. This distribution of information in the spacial frequency spectrum becomes intuitively obvious when it is realized that the peak is spread out over several sampled points while the noise occurs from point to point. Thus the information about the peak occurs in a different region of the spacial frequency spectrum than does some of the noise information. Hence the noise information can be discriminated against with respect to signal information. This forms the basis for spectral smoothing operations. The phase spectrum of the spacial frequencies is shown in Figure 1E. Phase, in this case, simply refers to the phase of the individual spacial frequencies a t one specific point on the optical spectrum. The phase is most usefully calculated with respect to a point on the optical spectrum at which the spacial frequencies are in phase. For the spectrum illustrated in Figure l A , this point is the peak maximum. This, indeed, is the reason for the existence of the peak when the optical spectrum is interpreted as a Fourier summation of spacial frequencies. It can be seen from the phase spectrum that the phases of the low frequency spacial frequencies (the peak information) are essentially the same-Le., these spacial frequencies are in phase. A small slope in the phase spectrum in this region indicates that the reference point chosen is not the exact peak maximum or that the peak is slightly asymmetric. After a certain point, the phase spectrum begins to fluctuate essentially at random. This is an indication that the corresponding spacial frequencies are essentially due to noise in the original optical spectrum. In other words, it is unlikely that spacial frequencies resulting from noise in the original spectrum would happen, by chance, to have the same phase as the spacial frequencies resulting from the signal ( i e . , the peak) a t a specific point in space along the optical spectrum axis (i.e., the peak maximum). Thus the phase spectrum provides additional information about the distribution of signal and noise spacial frequencies. Fourier transformation is a cyclic operation. The original optical spectrum can be regenerated using the real and imaginary outputs. The specific method will depend on the particular FFT program that is being used. For the operations illustrated in the next section, the optical spectrum was regenerated using the real output as a set of real input data and the resulting real output was the desired optical spectrum. A second approach was to use the real output as the real input and the negative of the imaginary output as the imaginary input to the FFT program (12). Again the resulting real output was the desired optical spectrum. It is important to note that the amplitude spectrum of the spacial frequencies does not contain any information about the position of the spectral peaks in the original optical spectrum. Only the real and imaginary outputs d o in the frequency of their oscillations. However, the real and imaginary outputs can be regenerated from the amplitudes of the spacial frequencies using the phase information. (12) R. Bracewell, “The Fourier Transform and Its Application,” McGraw-Hill Book Co., New York, N.Y., 1965.
I
FOURIER TRANSFORMATION
I
MULT x
REAL OUTPUT
FOURIEK TRANSFORMATION
c
Figure 2. Smoothing function applied to a spectrum using Fourier transformations
n
SMOOTHING FUNCTION
Several data handling operations can be readily carried out on spectra using the information obtained upon Fourier transformation of the spectra. In general, the amplitude spectrum of the spacial frequencies or the real and imaginary outputs are modified before reconstruction of the optical spectrum. The modification typically involves multiplication of the real output by a relatively simple function. This is analogous to convolving the original optical spectrum with the Fourier transformation of the multiplication function. Smoothing, differentiation, and resolution enhancement of spectra using Fourier transformations are discussed and illustrated in the next section. These operations are only representative of the large number of data handling operations that can be performed on spectra using this approach.
"I
I
SPECIFIC OPERATIONS PERFORMED ON SPECTRA USING FOURIER TRANSFORMATIONS Smoothing. It is often desirable to smooth a spectrum in order to improve the signal-to-noise ratio. Several approaches to this problem have been discussed in the literature (I, 6, 7). One of the most common is to convolve the spectrum with an appropriate weighting function. This convolution can be carried out in a versatile and effective way using Fourier transformations. It was noted in the last section that some of the noise information in the original optical spectrum appears in a different region of the spacial frequency spectrum than does the signal information. This provides a means of discriminating against the noise with respect to the signal before reconstruction of the optical spectrum and, hence, some of the noise can be filtered out. This is illustrated in Figure 2. The original spectrum is shown in Figure 2A. This is the spectrum of the emission of 0.05 ppm Ca at 4226.7 A in an O2-H2 flame. The real part [X(J)] of its Fourier transformation is shown in Figure 2B. The high frequency spacial frequency region (primarily noise information) can be truncated by multiplying X ( J ) by the simple function shown in Figure 2C to generate the modified real output shown in Figure 2 0 . The smoothed optical spectrum that results upon regeneration from the modified real output shows a marked reduction in the noise level (Figure 2E). In addition, since the signal information was not truncated, the peak shows little or no broadening. This type of operation is approximately analogous to analog low pass filtering. The smoothing function illustrated in Figure 2C is a low pass digital filter
L SMOOTHED SPECTRUM
Figure 3. Smoothing functions and the resulting smoothed spectra. The original spectrum is shown in Figure 2A (See text for discussion)
for spacial frequencies. The digital filter approach allows for complete control of the cutoff frequency and control or elimination of phase shifts. These are often difficult to control with analog filtering. In addition the peak shifting property of analog RC filters is eliminated as both future and prior information is utilized (8,13). It is difficult to present a hard and fast rule for determining the extent of the truncation function illustrated in Figure 2C. This choice is highly dependent on the specific experimental conditions, the amount of filtering desired, and the degree of signal distortion that can be tolerated. The simplest approach to this problem is an empirical one. The optical spectrum of interest is first measured under condi(13) K. S. Seshadri and R. N. Jones, Spectrochim. Acta, 19, 1013 (1963). ANALYTICAL CHEMISTRY, VOL. 44, NO. 6, MAY 1972
0
945
A /
il
I 11 h
A
A
‘J GlFfERENTlATiNG FILTER
GIFFERENTIATED SPECTRUM
Figure 4. Differentiation of a spectrum. The differentiating Rlter is applied in a manner analogous to that depicted in Figure 2 tions that yield a high signal-to-noise ratio. Transformation of this signal will result in a well defined spacial frequency spectrum and the point beyond which little or no signal information is present can be easily established. The abrupt filter illustrated in Figure 2 may not be the most desirable for certain measurements. If some of the signal information is truncated abruptly, spurious side lobes will result. This is shown in Figure 3A. However, with very noisy spectra it may be desirable to have a low cutoff frequency. This will, in generaI, necessitate the truncation of some of the higher spacial frequencies that contribute to the peak information and as such the peak will be broadened. In this case, a smoothing function can be used that minimizes side lobes such as a linear truncation (see Figure 3B). The broadening of the peak is often quite acceptable and is simply the standard trade-off between the signal-to-noise ratio and resolution. Several other smoothing functions can be used such as Gaussians, exponentials, etc., and at this point the question might well be asked, “Is there a smoothing function that will result in an optimum value for the signal-to-noise ratio?” Suffice it to say that a considerable amount of work has been reported in this general area and the characteristics of such a matched filter have been rigorously established (14). When the noise is white, a filter that takes the form of the spacial frequency spectrum of the instrumental line shape function is a close approximation to a matched filter for these signals. This can be determined by calculating the spacial frequency spectrum of a line measured with a very high signal-to-noise ratio. The smoothing function determined in this way is shown in Figure 3C along with the resulting smoothed spectrum. This operation essentially amounts to convolving a noisy line with an essentially noise-free version of itself. Note that with this filter the line is considerably widened. This is simply a result of the goal that was set for the filter, i.e., maximizing the signal-to-noise ratio. If it is also desirable to preserve, as well as possible, the observed line shape, then a different filter must be designed. Ernst has discussed this problem (8). (14) G. L. Turin, IRE Trans. Information Theory, IT-6, 311 (1960).
946
ANALYTICAL CHEMISTRY, VOL. 44, NO. 6, MAY 1972
The improvement in the signal-to-noise ratio achieved by these smoothing operations can be used as an indication of their effectiveness. Calculated on the basis of peak maximum divided by the root mean square value of the base-line noise over sixteen points of the base line, the improvement in the signal-to-noise ratio achieved by the filter shown in Figure 2 is about 8. It must be emphasized that this number is only valid for this specific spectrum because the improvement is highly dependent on the relative extents and magnitudes of the spacial frequency spectra of the signal and noise information. Thus a statement about the relative bandwidths of the signal and noise information should be made when a signal-to-noise ratio is reported. This problem can be looked at in a very qualitative way. For the spectrum illustrated in Figure 2A, it could be said that the signal-to-noise ratio is not very good. But it could perhaps more accurately be said that the signal-to-noise ratio is not very good for narrow peaks but that it is not too bad for broad peaks, Le., peaks composed of low frequency spacial frequencies. The improvement achievable using the smoothing functions attests to this fact. Thus attempts should always be made to limit the noise bandwidth to that just necessary for the accurate transmission of the signal information. This bandwidth control can be accurately achieved with the digital smoothing functions described above. Differentiation. Differentiation of spectra has often been used to modify the spectral information. The first derivative of a spectral peak is used as an aid in exact peak location (3, 15) and higher derivatives are used for peak sharpening (2, 16). The derivative theorem of Fourier transforms (12) states that if the imaginary part of the Fourier transformation of a function is multiplied by a linear ramp (starting at the origin) the result is the real part of the Fourier transformation of the derivative of the original function. Multiplication by this linear ramp function in the Fourier domain eliminates the dc level and attenuates low frequency spacial frequencies with respect to high frequency spacial frequencies. It simply amounts to a high pass digital filter for spacial frequencies. The accentuation of high frequency spacial frequencies is a well known characteristic of a differentiation step (17). The accentuation should not be carried out beyond the point at which the signal spacial frequencies disappear or else the resulting derivative will be very noisy. Thus some low pass digital filtering of the spacial frequencies should always be used in conjunction with differentiation. This is directly analogous to techniques used in the design of analog differentiating circuits using active filters. In addition, as with analog differentiation, the quality of the differential is dependent on the high and low frequency cutoffs relative to the signal frequencies. The effect of such a digital differentiating filter on a spectrum is illustrated in Figure 4. The original spectrum is the sodium doublet as emitted by a sodium hollow cathode lamp and is shown in Figure 4A. The differentiating filter is shown in Figure 4B and it was applied in a manner analogous to that depicted in Figure 2. Note that both high and low pass filtering are readily carried out in one simple multiplication step. The resulting first derivative spectrum is shown in Figure 4C. Higher derivatives can easily be obtained by successiveapplication of this filter. (15) J. P. Walters and H. V. Malmstadt, Appl. Spectrosc., 20, 193 (1966). (16) L. C . Allen, H. M. Gladney, and S.H. Glarurn,J. Chem. Phys., 40, 3135 (1964). (17) F. R. Stauffer and H. Sakai, Appl. Opt., 7,61(1968).
A
spectrum. See text for discussion of the
Resolution Enhancement. Many approaches to resolution enhancement have been discussed in the literature. Common methods have used pseudo-deconvolution ( 4 , 18, 19), special convolving filters (9), and differentiation (16). Resolution enhancement is desirable because the observed spectrum is often not an accurate representation of the real spectrum. The observed spectrum is the result of the convolution of the real spectrum by the resolution function of the spectrometer. It is often the slit width of the spectrometer that determines the width of the resolution function. This convolution distorts both the shape and the width of the real spectral lines and this can limit fundamental interpretation of line shapes and resolution. The effect that this convolution can have on a simple spectrum is shown in Figure 5. The sodium doublet (5895.92 A, 5889.95 A) was measured at two spectral bandwidths, -1 A (Figure 5A) and -4 A (Figure 5C). The respective spacial frequency spectra are shown in Figures 5B and 5D. A consideration of these spacial frequency spectra provides unique insight into resolution enhancement. It is obvious that observation of the sodium doublet with the wider spectral bandwidth has altered the spacial frequency spectrum. For the situation illustrated in Figure 5 in which the spectral bandwidth is wider than the spectral line width of the real spectrum, some of the upper spacial frequencies are completely lost. In addition, for all situations, the amplitudes of the lower spacial frequencies are reduced. Essentially all resolution enhancement procedures utilizing a convolution or pseudo-deconvolution approach attempt to restore the spacial frequency spectrum to that of the real spectrum or in this case to that of a spectrum observed with higher resolution. This general approach to resolution enhancement has been discussed in an excellent paper by Bracewell and Roberts (20). This paper deals with radio astronomy but there is no fundamental difference between scanning the sky with a radio antenna and scanning a spectrum with a slit. The effectiveness and limitations of this approach to resolution enhancement should now be obvious. The upper spacial frequencies that are lost cannot be recovered. This imposes a fundamental limit as to how far resolution enhancement can be carried out using the information available in the spacial frequency spectrum. Sometimes it is possible, us(18) W. F. Herget, W. E. Deeds, N. M. Gailar, R. J. Lovell, and A. H. Nielsen, J. Opt. SOC.Amer., 52,1113 (1962). (19) P. A. Jansson, R. H. Hunt, and E. K. Plyler, ibid., 58, 1665 (1968). (20) R. N. Bracewell and J. A. Roberts, Austr. J . Phys., 7, 616 (1954).
ing a priori information about the spacial frequency spectrum, to extrapolate into this region but this is risky at best as is effectively illustrated in Figure 8 of Bracewell and Roberts (20). Thus resolution enhancement, in general, must rely on restoration of the amplitudes of the lower spacial frequencies still present in the spacial frequency spectrum of the observed optical spectrum. Resolution enhancement may be necessary or desirable in many experimental situations. For example, it may be necessary in carrying out a series of spectral measurements at low concentrations, to use a wider slit width than is desirable with respect to resolution in order to have sufficient sensitivity. The spectrum shown in Figure 5C can serve as an example of such a measurement. In general, it is possible under more ideal experimental conditions (i.e., higher concentration) to carry out the same measurement with better resolution. The spectrum in Figure 5A can serve as an example of this measurement. Resolution enhancement of the series of measurements made with lower resolution may now be carried out using the information contained in the higher resolution spectrum. The spacial frequencies of the lower resolution spectra are simply multiplied by an appropriate function to restore their amplitudes to those of the spacial frequencies of the higher resolution spectrum. For the situation discussed above and illustrated in Figure 5, this function may be determined by dividing the spacial frequency spectrum illustrated in Figure 5B by that illustrated in Figure 5D. The spacial frequency spectra should be normalized and the division can only be carried out to the point where noise begins to dominate the result. Using this approach, the spacial frequency spectrum in Figure 5D was restored to that shown in Figure 5F. The resolution enhanced spectrum is shown in Figure 5E. Often side lobes are generated in the resolution-enhanced spectrum because of truncation effects similar to those illustrated in Figure 3A. Side lobe generation can be minimized by using the smoothing techniques discussed earlier. All the data handling operations have been discussed and illustrated using simple optical spectra. There is no limitation to their application to more complex spectra of any kind except that they be digitized. In addition the operations that can be performed on spectra using Fourier transformations are by no means limited to those discussed here. Other possible operations include cross correlation of spectra and several other approaches to resolution enhancement, RECEIVED for review September 13, 1971. Accepted November 30, 1971. Financial support by the National Research Council of Canada and the University of Alberta is gratefully acknowledged. ANALYTICAL CHEMISTRY, VOL. 44, NO. 6, MAY 1972
947
