Improving the Diagnostic Accuracy of N-Terminal B-Type Natriuretic Peptide in Human Systolic Heart Failure by Plasma Profiling Using Mass Spectrometry Donald J. L. Jones,*,† Richard Willingale,‡ Paulene A. Quinn,§ John H. Lamb,† Peter B. Farmer,† Joan E. Davies,§ and Leong L. Ng§ Cancer Biomarkers and Prevention Group, Biocentre, University of Leicester, Leicester, LE1 7RH, Department of Physics and Astronomy, University of Leicester, LE1 7RH, and Department of Cardiovascular Sciences, Clinical Sciences Building, Leicester Royal Infirmary, LE2 7LX Received January 15, 2007
Abstract: We have combined the measurement of N-terminal pro-B type natriuretic peptide (NTproBNP) with plasma peptide profiling to evaluate the effect on sensitivity and specificity of systolic heart failure (SHF) diagnosis. Plasma NTproBNP levels were measured from 100 SHF patients and 100 age/gender matched controls and plasma protein profiles obtained using MALDI-MS. Sixty-seven m/z peaks were significantly different between SHF and normals, and following logistic regression analysis with NTproBNP values, 6 peaks retained independent predictive value. Receiver operating characteristic (ROC) curves for SHF diagnosis had areas of 0.91 for NTproBNP, improving to 0.99 with the model. In a separate validation test set (32 SHF, 20 normals), the model remained highly accurate (ROC area 0.995). An artificial neural network with these 6 peak intensities and NTproBNP produced ROC areas of 0.99 in both training and test sets. The sensitivity and specificity of SHF diagnosis using NTproBNP in training (85, 85%) and test (93, 75%) sets was improved in the model for both training (96, 96%) and test (100, 95%) sets. The accuracy of SHF diagnosis using NTproBNP is improved by the use of a plasma profile of 6 peptide peaks, reducing the uncertainty in the diagnostic gray zone of using NTproBNP alone. Keywords: systolic heart failure • natriuretic peptides • mass spectrometry • sensitivity • specificity
Introduction Systolic heart failure (SHF, left ventricular systolic dysfunction) is a leading cause of mortality and morbidity and recent improvements in its diagnosis have included use of the natriuretic peptides such as the B-type natriuretic peptides * To whom correspondence should be addressed. Dr. Don J. L. Jones, Cancer Biomarkers and Prevention Group, Biocentre, University of Leicester, Leicester, LE1 7RH, UK. Phone, +1162231827; fax, +1162231840; E-mail,
[email protected]. † Cancer Biomarkers and Prevention Group. ‡ Department of Physics and Astronomy. § Department of Cardiovascular Sciences. 10.1021/pr070023d CCC: $37.00
2007 American Chemical Society
(BNP and N-terminal proBNP or NTproBNP). BNP or NTproBNP is effective at excluding SHF, but use for positively identifying SHF patients is more limited.1-3 Combinations of biomarkers may be more effective in improving accuracy of diagnosis, as has been demonstrated in predicting prognosis following acute coronary syndromes.4 Matrix assisted laser desorption ionization mass spectrometry (MALDI-MS) and surface enhanced laser desorption ionization mass spectrometry (SELDI-MS) allow the simultaneous examination of multiple potential biomarkers in plasma. These approaches, using a combination of mass spectrometry and machine learning algorithms, have been primarily used in neoplastic disease.5-7 Petricoin and others originally used SELDI-MS to examine diagnostic peptides in ovarian cancer.5 Tempst’s group developed alternative strategies to Petrocoin’s utilizing magnetic beads to extract the peptides prior to MALDI analysis.7 The same group also established methods identifying key variables that aid the standardization of the data.8 Biomarker discovery in cardiac research has more recently used established proteomic workflows.9-12 Protein expression has been investigated, for example, using gel-based techniques combined with MALDI. Mateos-Ca´ceres et al. looked at changes to protein expression using 2DE gels linked with MALDI-MS.13 In this study the authors found a number of key proteins related to acute coronary syndrome that were differentially expressed in patients. Also, “shotgun” approaches using (2D) LC-MS/ MS have been investigated. Kislinger et al. attempted, with some success, to elucidate biochemical alterations (such as transcriptional regulators, signaling factors and proteins linked with cardiac disease) between normal and diseased heart tissue.14 Profiling methods such as SELDI or MALDI are unlikely to offer the comprehensive coverage that the proteomic workflow approaches offer, but potentially could offer greater speed, sample throughput, and flexibility in use for diagnosis and prognosis. One such profiling attempt used the SELDI approach to identify biomarkers associated with idiopathic pulmonary arterial hypertension.15 In this paper, candidate biomarkers were identified using conventional statistical methods. All these papers provided mechanistic information regarding aspects of certain heart conditions. The ability to examine a broad variety of peptide biomarkers concurrently, using MALDI-MS may have potential in improving the diagnostic accuracy of NTproBNP in SHF. In the current Journal of Proteome Research 2007, 6, 3329-3334
3329
Published on Web 07/12/2007
technical notes
MS Plasma Profiling in Systolic Heart Failure Table 1. Patient Characteristicsa training
test
normal controls
number (male) age (years) plasma creatinine (µmol/L) drugs -diuretics -β-blocker -ACE-inhibitor/angiotensin receptor blocker aetiology -ischemic cardiomyopathy -dilated cardiomyopathy plasma NTproBNP (fmol/mL)
a
heart failure
normal controls
heart failure
100 (80) 65.5 [40.0-80.6] 88 [25-130]
100 (77) 66.9 [40.0-89.0] 101 [58-274]
20 (9) 54.5 [40.0-77.3] 86 [78-93]
32 (23) 60.6 [30.0-79.0] 107 [60-225]
0 0
66 33
0 0
20 18
0
73
0
25
0 0 30.4 [0.3-213.1] {5.7-89.7}
76 24 638.5 [3.7-29367] {200.7-1553.2}
0 0 35.8 [0.3-492] {0.3-114.6}
23 9 1168.8 [1.0-23645] {626.3-3768.6}
Medians [ranges] are reported. Interquartile ranges { } are provided for the NTproBNP levels.
study, we have examined this process in a training set of SHF and normal controls, and validated the features extracted from the training set on a further test set of SHF and normal subjects. The final array of 6 peptide marker intensities and NTproBNP provided a highly sensitive and specific diagnostic tool for SHF and could be easily incorporated into an artificial neural network for clinical use.
Methods Plasma Samples. Patients with SHF were recruited from the hospital. All had echocardiographically confirmed ejection fractions 5. The individual spectra were initially normalized to the sum of all the flux from the peaks. The mean intensity S and fractional rms scatter of the intensity σs/S of each peak was then calculated across all individuals. Spectra were then re-normalized using only the peaks within the range 100 < S < 10 000 and with σs/S < 0.6. The iterative process
technical notes
Jones et al.
Figure 1. Representative data matrix (in the range m/z 1000-3200) for 15 individuals (8 HF, 7 control) plotted as a heatmap or image. Isolating this narrow range and a small number of patients allows comparisons to be visualized between groups. Table 2. Logistic Model of NTproBNP and Sensitivity, Specificity, and Predictive Values for NTproBNP and the Logistic Modela (a) Logistic Model of NTproBNP and biomarker peaks, with their respective coefficients and odds ratios variable
m/z 3280.0 m/z 3159.0 m/z 3248.0 m/z 2647.9 m/z 1274.3 m/z 2469.0 log NTproBNP constant
B coefficient
-5.691 -7.921 7.016 8.133 -4.258 -5.004 2.477 -5.293
SEM
2.081 2.163 2.210 2.455 1.889 1.680 0.629 1.230
P value
0.0062 0.0003 0.0015 0.0009 0.0241 0.0029 8.3E-05 1.69E-05
odds ratio
0.0033 0.0004 1114.214 3403.687 0.0142 0.0067 11.9030 0.00503
mean S ((SE) HF patients
mean S ((SE) control patients
331 ((25) 416((20) 738 ((54) 675 ((44) 211 ((8) 908 ((80) na na
806 ((74) 603 ((26) 388 ((21) 285 ((22) 317 ((11) 1309 ((98) na na
(b) Sensitivity, Specificity, and Predictive Values for NTproBNP and the Logistic Model for Both Training and Test Sets training set
sensitivity specificity positive predictive value negative predictive value a
test set
NTproBNP
model
NTproBNP
Model
85% 85% 85% 85%
96% 96% 96% 96%
93.7% 75% 85.7% 88.2%
100% 95% 97% 100%
Mean S is the mean peak count and the standard error on the mean is shown in parenthesis. na denotes not applicable.
eventually led to a stable set of 181 normalization peaks, with S > 100 and σs/S < 0.6. The Kolmogorov-Smirnov (K-S) statistic was used to rank the peaks. A K-S probability for the null hypothesis (i.e., that the frequency distributions of the two classes are identical) is calculated directly from the absolute difference between the cumulative distributions at a given intensity level s0. The Fisher score, F, was used as a measure of the contrast between the two distributions. The application of the intensity mapping φ(s) forces all peaks to occupy the same dynamic range (nominally -1 to +1). Using a K-S value of g0.34, corresponding to a null hypothesis probability of 1.18 × 10-5, confidence level cl ) 0.99 with Ni ) 850 peaks, there were potentially 67 non-redundant biomarker peaks. Statistical and Bioinformatic Analyses. Statistical analyses were performed on SPSS Version 14 (SPSS Inc., Chicago, IL). Binary logistic regression analysis was performed on the training set, with the 67 biomarker peaks and Log normalized NTproBNP values. Probabilities from the model were saved for construction of receiver operating characteristic (ROC) curves. The same model was applied to the validation test set. Comparisons between ROC areas under the curve were performed using the method of Hanley and McNeil.16 Biomarkers identified on logistic regression were used to construct an artificial neural network, with 5 hidden layers, for prediction of diagnosis of HF (using the multilayered perceptron program on the Waikato Environment for Knowledge Analysis (WEKA), available at http://www.cs.waikato.ac.nz/∼ml/weka/index.html).17
All authors had full access to the data and take responsibility for its integrity. All authors have read and agree to the manuscript as written.
Results Table 1 illustrates the characteristics and therapy of SHF patients and age and gender matched normal controls in both training and test sets, with the plasma NTproBNP values. Mass spectra (normalized) for individual subjects in the training set (m/z range 1000-3200, Figure 1), denote some potential differences between normal and SHF cases. Binary logistic regression analysis identified a group of 6 biomarker peaks that had independent predictive value in addition to NTproBNP, accounting for a Nagelkerke r2 of 0.899 (P < 0.001), suggesting that a large proportion of the variance was accounted for. The coefficients of the model are reported in Table 2a, with sensitivity, specificity, positive and negative predictive values (PPV, NPV) in both the training set used to derive the model and an independent validation test set (Table 2b). The logistic model provides improved accuracy in diagnosis, with increased sensitivity, specificity, PPV and NPV for both training and test validation sets. Two of these biomarker peaks (m/z 3248.0 and 2647.9) are elevated in SHF, and 4 of them (m/z 3280.0, 3159.0, 1274.3, and 2469.0) are lower in SHF. Table 2a also shows the mean peak counts and the standard error for each mean obtained for the individual biomarker peaks. The peaks are clearly differentiated for each category (i.e., control and HF). Journal of Proteome Research • Vol. 6, No. 8, 2007 3331
technical notes
MS Plasma Profiling in Systolic Heart Failure
Figure 2. Histograms depicting distributions of NTproBNP and the predicted probability from the logistic model of 6 biomarker peaks and NTproBNP, in both the training and test sets
Figure 2 illustrates the overlap in NTproBNP values between SHF and normal controls in both the training and test sets, indicating a gray zone where diagnosis is unclear. The predicted probabilities from the logistic model show a clear separation between the SHF and controls, in both training and test sets. The predicted probabilites from the logistic model were used to construct ROC curves for both training and test sets. Figure 3 illustrates the improvement in area under the ROC curve that the model achieves (ROC AUC 0.989 [SEM 0.006]), above that of NTproBNP alone (ROC AUC 0.919 [SEM 0.021]) in the training set (P < 0.003). The ROC AUC for NTproBNP alone in the validation test set was 0.94 [SEM 0.031], rising to 0.995 [SEM 0.005] for the logistic model. While NTproBNP was correlated to age (rs ) 0.347, P < 0.001) and was higher in females (P < 0.001), the model predicted probability was weakly correlated to age (rs ) 0.167, P < 0.02) and was gender independent. In the SHF cases, NTproBNP was dependent on NYHA class, increasing with severity of disease (mean ( SEM of Log NTproBNP in fmol/mL 2.13 ( 0.25 2.47 ( 0.10, 3.14 ( 0.11, and 3.49 ( 0.12 for NYHA classes 1 to 4, respectively. In contrast, no such trend was seen in the predicted probability of the logistic model (0.94 ( 0.05, 0.90 ( 0.03, 0.98 ( 0.01, and 0.97 ( 0.02). The mean ( SEM Log NTproBNP and predicted probability in the normal controls was 1.31 ( 0.07 and 0.078 ( 0.016, respectively, suggesting that the method may be most useful to detect early cases of SHF, before plasma NTproBNP had risen substantially. 3332
Journal of Proteome Research • Vol. 6, No. 8, 2007
An artificial neural network with back-propagation was constructed from these 6 biomarker peaks and NTproBNP in the training set (with 5 hidden layers), using the multilayered perceptron program available on WEKA. The learning rate (or the amount by which weights are updated) was set at 0.3 and the momentum applied to weights during updating set at 0.2. This yielded a confusion matrix as shown below, with a ROC area of 0.99. The test set had a ROC area of 0.987, with a corresponding confusion matrix.
Discussion Plasma biomarkers have increasingly been used to identify patients with disease. The natriuretic peptides, in particular BNP and NTproBNP have been useful for ruling out heart failure, due to their high negative predictive values.1-3 However, the lower specificity and positive predictive values of BNP or NTproBNP for SHF diagnosis has not permitted positive identification of SHF cases. Such cases will need echocardiography or other imaging to provide a definitive diagnosis of SHF. This does not maximize the use of limited imaging resources. The present study suggests a potential solution to improving the diagnostic accuracy of NTproBNP, using an examination of the peptidome with MALDI-MS. This high-resolution method permits high-throughput relatively low-cost detection of cases of SHF, using the peptidome profile of 6 peaks. The combination of a multi-marker approach with a conventional molecular
technical notes
Jones et al.
Figure 3. ROC curves for NTproBNP and the predicted probability from the logistic model of 6 biomarker peaks and NTproBNP, in both the training and test sets
training set
test set
HF
normal
rclassified as
HF
normal
rclassified as
98 1
2 99
HF normal
32 3
0 17
HF normal
marker of systolic heart failure is a novel approach to the diagnosis of disease. By further augmenting the power of using NTproBNP with a group of disease-related peptides, the diagnosis is shown to be considerably improved. The criticisms of machine learning techniques tend to focus on “over-fitting”, an argument with certain validity when related to a small number of samples, but the validity declines as sample number increases. As these approaches have matured, the role of data management in making valid conclusions about candidate biomarkers has become paramount. Incorporation of this understanding into data management chains will offer great advances in the use of machine-learning techniques for diagnosis of disease. The models constructed from either logistic regression analysis or the artificial neural network were more accurate than when NTproBNP was used alone, increasing the specificity and PPV sufficiently for the positive identification of HF. In addition, the predicted probabilities from the logistic model appear to be less dependent on age than NTproBNP and were gender independent. This would simplify their use in diagnostics. Furthermore, the predicted probability from the model was independent of NYHA class, with a large difference in the values between the normals and NYHA class I patients. This suggests that the model could have utility in the early detection of SHF, even in patients with few symptoms. The current method takes approximately 9.5 h to complete for each sample but samples can be done in batches and the method could be developed for automation to improve its diagnostic use. Alternatively, following identification of the relevant peptides, two strategies could be pursued. First, the identification of the individual peptides could allow the development of a multi-antibody assay, for example, for use in multiplex immunoassay platforms enabling simultaneous determination of several biomarkers in 1 specimen. Alterna-
tively, a LC-MS/MS assay could be developed which specifically recognizes the individual peptides and, in the presence of internal standards, would provide quantitative data. Both these assays could be carried out on a blood sample and potentially have an analysis time less than an hour. Limitations to the current investigation include the relatively small number of subjects examined, although the number we have recruited exceeds that of many other MALDI-MS or SELDI-MS studies. The model derived from this study needs to be validated prospectively in a larger study sample. Many of these biomarker peaks are of low intensity and will need to be enriched before further identification could be carried out. The patients with systolic heart failure and the normal controls were highly selected, with marked differences in their left ventricular ejection fractions and wall motion scores, to highlight the differences in mass spectra profiles. Lesser degrees of systolic heart failure and other heart failure presentations such as those with preserved systolic function (or diastolic heart failure) should be examined further. In conclusion, we have demonstrated the validity of a multimarker approach to improving the diagnostic accuracy of NTproBNP in HF, using MALDI-MS followed by feature selection. The model has advantages over NTproBNP alone, since accuracy is improved, specificity of diagnosis is enhanced, and the output probability is independent of disease severity and gender with a reduced dependence on age.
Acknowledgment. We are grateful for the generous support of the Medical Research Council, UK (grant reference G0100873). References (1) Latour-Perez, J.; Coves-Orts, F. J.; Abad-Terrado, C.; Abraira, V.; Zamora, J. Accuracy of B-type natriuretic peptide levels in the diagnosis of left ventricular dysfunction and heart failure: A systematic review. Eur. J. Heart Failure 2006, 8, 390-399. (2) Wang, C. S.; FitzGerald, J. M.; Schulzer, M.; Mak, E.; Ayas, N. T. Does this dyspneic patient in the emergency department have congestive heart failure? J. Am. Med. Assoc. 2005, 294, 1944-1956. (3) Silver, M. A.; Maisel, A.; Yancy, C. W.; McCullough, P. A.; Burnett, J. C. Jr.; Francis, G. S.; Mehra, M. R.; Peacock, W. F., 4th; Fonarow, G.; Gibler, W. B.; Morrow, D. A.; Hollander, J. BNP Consensus
Journal of Proteome Research • Vol. 6, No. 8, 2007 3333
technical notes
MS Plasma Profiling in Systolic Heart Failure
(4)
(5)
(6)
(7)
(8)
(9) (10)
3334
Panel 2004: A clinical approach for the diagnostic, prognostic, screening, treatment monitoring, and therapeutic roles of natriuretic peptides in cardiovascular diseases. Congestive Heart Failure 2004, 10(5 Suppl 3), 1-30. Sabatine, M. S.; Morrow, D. A.; de Lemos, J. A.; Gibson, C. M.; Murphy, S. A.; Rifai, N.; McCabe, C.; Antman, E. M.; Cannon, C. P.; Braunwald, E. Multimarker approach to risk stratification in non-ST elevation acute coronary syndromes: simultaneous assessment of troponin I, C-reactive protein, and B-type natriuretic peptide. Circulation 2002, 105, 1760-1763. Petricoin, E. F.; Ardekani, A. M.; Hitt, B. A.; Levine, P. J.; Fusaro, V. A.; Steinberg, S. M.; Mills, B. G.; Simone, C.; Fishman, D. A.; Kohn, E. C.; Liotta, L. A. Use of proteomic patterns in serum to identify ovarian cancer. Lancet 2002, 359, 572-577. Zhang, Y. F.; Wu, D. L.; Guan, M.; Liu, W. W.; Wu, Z.; Chen, Y. M.; Zhang, W. Z.; Lu, Y. Tree analysis of mass spectral urine profiles discriminates transitional cell carcinoma of the bladder from noncancer patient. Clin. Biochem. 2004, 37, 772-779. Villanueva, J.; Philip, J.; Entenberg, D.; Chaparro, C. A.; Tanwar, M. K.; Holland, E. C.; Tempst, P. Serum peptide profiling by magnetic particle-assisted, automated sample processing and MALDI-TOF mass spectrometry. Anal. Chem. 2004, 76(6), 15601570. Villanueva, J.; Philip, J.; Chaparro, C. A.; Li, Y. B.; Toledo-Crow, R.; DeNoyer, L.; Fleisher, M.; Robbins, R.; Tempst, P. Correcting common errors in identifying cancer-specific serum peptide signatures. J. Proteome Res. 2005, 4(4), 1060-1072. Jaffe, A. S.; Babuin, L.; Apple, F. S. Biomarkers in acute cardiac disease - The present and the future. J. Am. Coll. Cardiol. 2006, 48(1), 1-11. Arab, S.; Gramolini, A. O.; Ping, P.; Kislinger, T.; Stanley, B.; van Eyk, J.; Ouzounian, M.; MacLennan, D. H.; Emili, A.; Liu, P. P. Cardiovascular proteomics - Tools to develop novel biomarkers and potential applications. J. Am. Coll. Cardiol. 2006, 48(9), 17331741.
Journal of Proteome Research • Vol. 6, No. 8, 2007
(11) McGregor, E.; Dunn, M. J. Proteomics of the heart - Unraveling disease. Circ. Res. 2006, 98(3), 309-321. (12) Willingale, R.; Jones, D. J. L.; Lamb, J. H.; Quinn, P.; Farmer, P. B.; Ng, L. L. Searching for biomarkers of heart failure in the mass spectra of blood plasma. Proteomics 2006, 6(22), 5903-5914. (13) Mateos-Caceres, P. J.; Garcia-Mendez, A.; Farre, A. L.; Macaya, C.; Nunez, A.; Gomez, J.; Alonso-Orgaz, S.; Carrasco, C.; Burgos, M. E.; de Andres, R.; Granizo, J. J.; Farre, J.; Rico, L. A. Proteomic analysis of plasma from patients during an acute coronary syndrome. J. Am. Coll. Cardiol. 2004, 44, 1578-1583. (14) Kislinger, T.; Gramolini, A. O.; MacLennan, D. H.; Emili, A. Multidimensional protein identification technology (MudPIT): Technical overview of a profiling method optimized for the comprehensive proteomic investigation of normal and diseased heart tissue. J. Am. Soc. Mass Spectrom. 2005, 8, 1207-1220. (15) Abdul-Salam, V. B.; Paul, G. A.; Ali, J. O.; Gibbs, S. R.; Rahman, D.; Taylor, G. W.; Wilkins, M. R.; Edwards, R. J. Identification of plasma protein biomarkers associated with idiopathic pulmonary arterial hypertension. Proteomics 2006, 6, 2286-2294. (16) Ng, L. L.; Loke, I.; Davies, J. E.; Khunti, K.; Stone, M.; Abrams, K. R.; Chin, D. T.; Squire, I. B. Identification of previously undiagnosed left ventricular systolic dysfunction: community screening using natriuretic peptides and electrocardiography. Eur. J. Heart Failure 2003, 5, 775-782. (17) Hanley, J. A.; McNeil, B. J. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983, 148, 839-843. (18) Witten, I. H.; Frank, E. Data Mining: Practical machine learning tools and techniques, 2nd ed; Morgan Kaufmann: San Francisco, 2005.
PR070023D
