Some Observations on Digital Smoothing of Electroanalytical Data Based on the Fourier Transformation John W. Hayes,’ Donald E. Glover,Z and Donald E. Smith3 Department of Chemistry, Northwestern University, Evanston, 111. 60201
Mac W. Overton Food and Drug Administration, Chicago District Laboratory, Room 1222, Post Ofice Building, Chicago, Ill. 60607 A digital data smoothing technique based on the Fourier Transformation is evaluated for various types of electroanalytical data. Spurious side lobes (ringing) frequently introduced when the rectangular filter function is imposed are found to be suppressed without attendant broadening of the response peaks, if one combines Fourier transformation with a translationrotation transformation on the original data. The procedure is successfully applied to dc polarograms, fundamental and second harmonic ac polarograms, cyclic voltammograms, and transient decays. Digital Fourier transform smoothing is compared with the well-known floating least squares routine. Advantages and pitfalls are discussed.
MOST SCIENTIFIC INSTRUMENTS invoke some form of signal filtering to enhance the signal-to-noise ratio associated with the readout ( 1 , 2 ) . The majority of these data smoothing procedures involve the use of analog filters ( I , 3). However, much attention also has been focused on digital filtering techniques which have certain significant advantages over the analog methods (4-12). The so-called floating least-squares smoothing routine is probably the most ubiquitous of the digital schemes (10, 11). Nevertheless, digital data smoothing based on the Fourier Transform is an appealing alternative, as has been pointed out recently (4-9). The development of the Fast Fourier Transform (FFT) algorithm (13) and minicomputers (14, 15) recently have combined to enhance dramatically the convenience, accessibility, and speed associated with signal processing schemes utilizing the Fourier 1 On leave from the School of Chemistry, University of Sydney, Sydney, N.S.W., Australia. 2 Present address, Department of Chemistry, California Institute of Technology, Pasadena, Calif. 91 109. 3 To whom correspondence Fhould be addressed.
( 1 ) E. B. Magreb and D. S. Blomquist, “The Measurement of
Time-Varying Phenomena: Fundamentals and Applications,” Wiley-Interscience,New York, N.Y., 1971. (2) G. W. Ewing, “Instrumental Methods of Chemical Analysis,” 3rd ed., McGraw.-Hill,New York, N.Y., 1969. (3) J. G. Graeme, G. E. Tobey, and L. P. Huelsman, Ed., “Operational Amplifiers: Design and Applications,” McGraw-Hill, New York, N.Y., 1971. (4) B. Gold and C. M. Rader, “Digital Processing of Signals,” McGraw-Hill, New York, N.Y.. 1969. 44,943 (1972). (5) G. Horlick. ANAL.CHEM., (6) M. Caprini, S. Cohn-Sfetcu, and A. M. Manof, ZEEE Trans. Audio Electroacoustics, 18,389 (1970). (7) L. R. Rabiner, B. Gold, and C. A. McGonegal, ibid., p 83. (8) J. D. Bruce, ibid., 16,495 (1968). (9) J. D. Morrison, J. Chem. Phys., 39,200 (1963). 36,1627 (1964). (10) A. Savitsky and M. J. E. Golay, ANAL.CHEM., (11) D. M. Mohilner and P. R. Mohilner, J. Electrockem. Soc., 115,261 (1968). (12) R. Merkel, Coiztr. Eiig., 17 ( l ) ,92(1970). (13) J. W. Cooley and J. W. Tukey, Math. Comp., 19,297 (1965). (14) S. P. Perone, ANAL.C H E M . ,1288(1971). ~~. (15) M. Margoshes, ihid.,(4), lOlA(1971).
Transform (Z5-18). The resulting upsurge of interest in this data manipulation category has reached a level of activity and success which suggests the inevitability of digital Fourier Transform equipment eventually becoming a standard feature in the chemical laboratory. Whereas most recent chemical applications of the digital Fourier Transform have focused on extraction of the frequency domain response (spectra) from a time domain data array (e.g., an interferogram) (15-Z8), the data smoothing feature inherent in the Fourier Transform is a “subsidiary bonus” which should not be overlooked. The availability of appropriate instrumentation, and an interest in techniques which might enhance the precision and accuracy of electrochemical data stimulated our investigation on the applicability of Fourier Transform data smoothing. Typical results are presented here. They include those obtained with an apparently novel procedure which combines the Fourier Transform and a translation-rotation operation on original raw data to significantly enhance data smoothing fidelity. Because the relevant basic principles and properties are described adequately in the literature (4-9), background material on digital Fourier Transform data smoothing will not be presented here, except to illustrate the basic procedures. ’
EXPERIMENTAL
All raw electrochemical data were acquired with the aid of analog instrumentation (potentiostats, signal sources, cells, etc.) and solution preparation procedures which have been described elsewhere (19, 20). On-line digital data acquisition and ail digital data processing were performed with the aid of a laboratory computer system, based on the Raytheon 704 and peripherals, which is interfaced to the analog electrochemical equipment. The Raytheon system is characterized by a 16-bit, 16K core memory (1 psec cycle time), two type 73491A 9-track magnetic tape drives, and an AID system (analog-digital interface) comprised of a 14-bit analog-to-digital (A/D) converter with a 40-kHz repetition rate, a 16-channel A/D input multiplexer, two 12-bit digital-to-analog converters, two sample-and-hold amplifiers with 50-nsec aperture times and a 1-MHz real-time clock. Details of system performance characteristics are available in the manufacturer’s literature (21). Specific aspects of data acquisition procedures utilized in these laboratories are given elsewhere (22, 23). Software for the data smoothing (16) M. J. D. Low, ANAL.CHEM., 41 (6), 97A(1969). (17) T. C. Farrar, ibid.,42(4), lOgA(1970). (18) P. C. Jurs, ibid.,43,1812(1971). (19) E. R. Brown, T. G. McCord, D. E. Smith, and D. D. DeFord, ibid.,38,1119(1966). (20) E. R. Brown, H. L. Hung, T. G. McCord, D. E. Smith, and G. L. Booman, ibid., 40,1424 (1968). (21) “Raytheon 704 User’s Manual,” Raytheon Data Systems, Norwood, Mass., 1970. (22) S. C. Creason, J. W. Hayes, and D. E. Smith, J . Electroarzal. Chem., submitted for publication. (23) D. E. Glover and D. E. Smith, ANAL.CHEM., submitted for
publication.
ANALYTICAL CHEMISTRY, VOL. 45, NO. 2, FEBRUARY 1973
277
N
m N
4
278
ANALYTICAL CHEMISTRY, VOL. 45, NO. 2, FEBRUARY 1973
routines was written primarily in the SPIEL Language (24). Program listings are available from the authors on request. Simulated data were calculated with the aid of standard computer programs (25), or were entered from data tables via teletype. Specific information regarding data origins, experimental conditions, smoothing functions, etc. are given in the figure captions.
1
A ORIGINAL DATA
RESULTS AND DISCUSSION
The standard operation sequence associated with Fourier Transform data smoothing is quite straightforward, and has been illustrated for the cases of spectral and ionization efficiency data (5, 9). To facilitate the present discussion, a similar example is given in Figure 1for the case of fundamental harmonic ac polarographic data. As shown, the standard smoothing operation involves three straightforward steps: (a) the data array to be smoothed (Figure 1A) is subjected to the Fourier Transformation which yields a “Fourier spectrum” composed of real (in-phase) and imaginary (quadrature) components (Figure 1B); (b) The Fourier spectrum is multiplied by a smoothing function, such as the rectangular function (Figure IC), to obtain a filtered spectrum (Figure 1 0 ) ; (c) The filtered spectrum is inverse Fourier transformed, yielding the smoothed data array (Figure 1E). Truncation of the Fourier spectrum as illustrated assumes the contributions to components above the cut-off point cfo,Figure 1 0 arise primarily from noise, while the predominant contributions from actual data fall below the cut-off point. When such conditions are met, the smoothing routine in question can be quite effective, particularly because it provides a filter action characterized by an extremely sharp roll-off without introducing either phase or amplitude distortion of spectral components below the cut-off point. From a pragmatic viewpoint, such an effect must be considered inaccessible to analog approaches. At the same time, one must recognize that in continuous frequency space, the filter function resulting from these manipulations is not rectangular (except in the limit of infinite data points), and has a much more complex characteristic which includes low-level, periodic lobes running far beyond the cut-off point ( 4 , 7, 8). The practical implication is that noise components with frequencies beyond the cut-off point which lie between those represented in the discrete Fourier spectrum (e.g., Figure 1B) may not be completely attenuated. The filter function is “rectangular” only with respect to frequencies exactly matching those represented in the discrete Fourier spectrum. The quantitative characterization and origin of this relationship between the discrete and continuous Fourier spectra are discussed at length in the literature ( 4 , 7, 8) and, consequently the subject will not be pursued here. A comparison of Figures 1A and 1Eprovides an illustration of the effect of Fourier Transform smoothing on typical fundamental harmonic ac polarographic data which, to begin with, are not particularly noisy. Results of applying the same procedure to a much noisier data set are shown in Figure 2 . The phase-selective second harmonic ac polarographic data (Figure ? A ) were obtained with a kinetically-controlled process (26, 27) which yielded a relatively small response, resulting in (24) M. P. Harvey and E. J. Kligman, “Signal Processing Interactive Engineering Language.” Raytheon Data Systems, Norwood, Mass., 1970. ( 2 5 ) T. G. McCord, Doctoral Dissertation, Northwestern University, Evanston, Ill., 1970. (26) K . R. Bullock and D. E. Smith, J. Electronnal. Chem., submitted for publication. (27) M. S . Shuman and I. Shain, ANAL.CHEM.,41,1818 (1969).
3
”5:
. g
g 8
-0,019
B. SMOOTHED DATA
-0.019
...-*., -. e.
’k.
-
G,
-
2 W +O 019 L
C
- --0 759
- -
-0 846
-0 934
(volts vs
s c e )
COMPARISON OF ORIGINAL
original data ---s m o o t h e d data
-0 019
----
___
L . _ .
-0 759
_ _ _ -~ -0 846
~
(volts v s
-0 934
s c e )
Figure 2. Fourier Transform smoothing of second harmonic ac polarographic data System: Same as Figure 1 Same as Figure 1, except frequency = 228 Hz Applied: Measured: Quadrature alternating current at 456 Hz
an unsatisfactory signal-to-noise ratio. Smoothing gives a substantial apparent improvement in data quality (Figure 2B). Comparison of the smoothed data to the original raw data (Figure 2C) suggests that the smoothed data set is as good a representation of the noise-free. polarogram as one could expect, given the initial data. While this example indicates that rather noisy data can be smoothed in a qualitatively satisfactory manner, it is not intended to prove anything regarding quantitative fidelity of the smoothed data. The latter point is addressed more directly by subsequent examples. However, it should be mentioned that the smoothed polarogram shown in Figure 2B agrees well with theoretically predicted polarograms (28) for the system in question using literature values (27) of the rate parameters (see Reference 26 for complete details). Figure 3 illustrates that Fourier Transform data smoothing can be beneficial, even when raw data precision appears adequate on casual inspection. When a cot - E d c profile (Figure 3C) is calculated from the apparently clean fundamental harmonic ac polarograms (Figure 3A, 3B), noise in the raw data becomes more apparent. However, when the polarograms are smoothed, a significant improvement is realized in the cot - E d c plot (Figure 30). The examples shown in Figures 1-3 represent near-ideal situations with regard to Fourier Transform smoothing with a rectangular smoothing function. The property primarily
+
(28) T. G. McCord and D. E. Smith, J . Electroanal. Chem., 26, 61 (1970).
ANALYTICAL CHEMISTRY, VOL. 45, NO. 2, FEBRUARY 1973
279
a
a
a
a m
i
a a
a
a
a
0
a
0
a
a m
a aa a a
a
a
a
a
a
a
a
a
a
m a
a' a
a
a
a a
0
a
a a a
a
a a
a
a a a
0
a
a a a
a a
a 0
a
a
a
a
a
a 0 0
a
a 0
a 0 0
a
d d
m
!3
e
CI
e
e
LO3
e
0 LO3
1
8
a 8 8
. a a
0 0'
0
a aa
a
a
a a
a
a
a
a
a a
a
- 8 e
a
a
a a
a
a
a
a
a
a
a
a a
a
0
a a a a
a a 0
.
a a
a
a
a
a a
a
a
a
a a
a
a 0
a a
a
a 0
.
a
rg
280
ANALYTICAL CHEMISTRY, VOL. 45, NO. 2, FEBRUARY 1973
i
F
ANALYTICAL CHEMISTRY, VOL. 45, NO. 2, FEBRUARY 1973
281
I
A.
ORIGINAL "CLEAN' DATA --.__ DATA AFTER ADDING NOISE AND SMOOTHING
A.
----- 0
ORIGINAL "CLEAN" SIMULATED DATA - - - - - e DATA AFTER ADDING NOISE AND SMOOTHING - - - - - o
,..' TIME
POTENTIAL
A
SMOOTH
ADD NOISE ADD NOISE
SMOOTH
E. "NOISY" DATA
1
B.
'.
"NOISY" DATA
.*
.
.....,.....' ....
.*
I
... *
:*
POTENTIAL
Figure 5. Application of modified Fourier Transform smoothing to dc polarographic data System: 3.0 X 10-3M U 0 2 +in 3.OM HCIO, at 25 "C Applied: Computer-controlled incremental dc scan Measured: Direct current at end of mechanically-controlled mercury drop life
responsible for this "ideality" is found in the fact that the data sets begin and terminate with zero or near zero values. In situations where the data begin and/or end with ordinate magnitudes which deviate significantly from zero, spurious peaks are introduced by the rectangular smoothing operation (5, 9). In effect, the non-zero data initiation and termination is "seen" as a discontinuous transient by the mathematical algorithm and, as with any sudden transient, a continuous broadband Fourier spectrum is required for accurate representation. If the Fourier spectrum is abruptly truncated, as in Figure 1, the resulting smoothed data will exhibit ringing (small, spurious peaks) in place of the sudden "transient." Often these false peaks are found only at the "wings" of the smoothed data and do no significant harm to the data features of interest. Unfortunately, this is not always the case, as shown in Figure 4 where a simulated clean exponential decay is shown to be badly distorted throughout the data record by the standard operation sequence (Figures 4A-D). Horlick has discussed this problem in further detail ( 5 ) and has pointed out that certain modifications of the filter function can minimize the foregoing problem. However, replacing the rectangular filter function with the commonly-suggested alternatives such as the linear truncation or a matched filter invariably is accompanied by some distortion (broadening) of the actual response ( 5 , 6). In electrochemical measurements, response shape is often an important quantitative observable, so that signal distortion associated with smoothing is undesirable. Consequently, an effort was made to find an alternative scheme which might alleviate this spurious response problem without significant distortion of the 282
TIME
Figure 6. Application of modified Fourier Transform smoothing to simulated cyclic voltammetric data
true response. A simple, satisfactory approach was found in which the original data are modified prior to Fourier Transformation. The modification amounts to rotating and translating the original data record so that initial and terminal points have zero values, thus eliminating the apparent abrupt transient from the original data set. The operation in question is illustrated in Figure 4. From Figures 4A and 4E, the rotation-translation operation can be seen to involve subtracting from each successive data point a quantity, A,, which varies linearly between the first point's magnitude, Ai, and All, the magnitude of the last point. Thus, the magnitude R, of the nth point in the rotated-translated array is calculated from the expression,
R,
=
A,
- An
(1)
where
,4,= Ai n
- 1) + (Ak -k Ai)@ - 1 =
1 , 2 , 3 . .. k
(3)
The profound and desirable effect of this translation-rotation transformation on the Fourier spectrum for the exponential decay is evident upon comparison of Figures 4 8 and 4F. While the normal exponential decay is characterized by a spectrum with significant amplitudes for all components (Figure 4B), the rotated-translated counterpart is represented by a relatively small number of Fourier components (Figure 4F). Applying a rectangular filter function to the !atter spectrum (Figure 4F -+ 4G), inverse Fourier Transforming (Figure 4G -,4H), and performing the inverse of the translation-rotation transformation (Figure 4H 41) gives back
ANALYTICAL CHEMISTRY, VOL. 45, NO. 2, FEBRUARY 1973
-
the original exponential decay without significant distortion (
