LETTER: Metabonomics in Ulcerative Colitis† Horace R. T. Williams,*,‡,§ David G. Walker,‡,§ Bernard V. North,| Simon D. Taylor-Robinson,‡ and Timothy R. Orchard‡ Department of Gastroenterology and Hepatology, Imaging Sciences Department, and Statistical Advisory Service, Imperial College London, London, United Kingdom Received January 27, 2010
A recent publication in the Journal of Proteome Research applied metabolic profiling (“metabonomics”) to ulcerative colitis; several different biosamples were investigated (J. Proteome Res. 2010, 9, 954-962). Comparing urinary profiles, no differences were found between cases and controls; in a previously published study (Am. J. Gastroenterol. 2009, 104, 1435-1444), we had found significant differences. In this correspondence, the reasons for our experimental and analytical approach are explained, and the negative results of the Journal of Proteome Research study are discussed. Dear Sir, Metabolic profiling (“metabonomics”) is proving to be an effective means of investigating the pathophysiology of disease and identifying potential biomarkers using human biofluids, tissues, and other biological extracts.1 We would like to congratulate Bjerrum et al.2 on the comprehensive metabonomic approach employed in their recent study of ulcerative colitis (UC), published in the Journal of Proteome Research. Using nuclear magnetic resonance (NMR) spectroscopy, they acquired metabolic profiles from the biopsies, colonocytes, lymphocytes, and urine of 35 patients with active UC, 33 with quiescent UC, and 25 control individuals. Significant metabolic differences were found between the biopsies and colonocytes of controls and patients with active but not quiescent UC; they found no differences between the urinary or lymphocytic metabolic profiles studied. In the discussion, they made reference to our recent paper,3 in which significant differences were found between the urinary metabolic profiles of 86 Crohn’s disease (CD) patients, 60 UC patients, and 60 controls, with longitudinal sampling from 75 individuals. Bjerrum et al. suggested that in four areas our study could have been improved. In fact, these points had been carefully considered and the reasons for our approach are explained below: 1. They commented that in our paper “unfortunately Q2 is not described”. The use of such parameters in assessing a multivariate model has long been a topic for debate, mainly due to difficulty in interpretation.4 The documentation of sensitivity and specific* Correspondence- Corresponding author: Dr. Horace RT Williams, Imperial College London, St Mary’s Campus, GI Unit, 3rd Floor Clarence Wing, St Mary’s Hospital, Praed St, London W2 1NY. Email: h.williams@ imperial.ac.uk. Tel: +442078861072. Fax: +442078866871. † (Letter regarding: Bjerrum, J. T.; Nielsen, O. H.; Hao, F.; Tang, H.; Nicholson, J. K.; Wang, Y.; Olsen, J. Metabonomics in Ulcerative Colitis: Diagnostics, Biomarker Identification, And Insight into the Pathophysiology. J. Proteome Res. 2010, 9, 954-962.) ‡ Department of Gastroenterology and Hepatology. § Imaging Sciences Department. | Statistical Advisory Service.
2794 Journal of Proteome Research 2010, 9, 2794–2795 Published on Web 03/19/2010
Table 1. Q2 Values for the Models Examined in Our Study All Individuals
Q2
Individuals on No Medication
CD: Controls
UC: Controls
CD: UC
Colonic CD: UC
CD: Controls
UC: Controls
CD: UC
0.52
0.47
0.66
0.47
0.75
0.68
0.50
ity, positive and negative predictive values, in assessing the predictive ability of a PLS-DA model has been advocated,4 and this was our preferred method, particularly as our study was published in a clinical journal. The Q2 values for the models examined in our study are recorded in Table 1. 2. Bjerrum et al. then stated that our cross-validation was “based on excluding just one sample at the time (sic), whereas (their) study employed a one-seventh cross-validation”. We would contend that our analyses were conducted with rigor. Leave-one-out cross-validation (LOOCV) is equivalent to k-fold cross-validation where k is the total number of samples; each sample in turn is excluded from the analysis, a model is created from the remaining samples, and the class membership of the excluded sample is predicted. While LOOCV can be prone to high variance, the possibility of bias is low; k-fold crossvalidation (such as 7-fold employed by Bjerrum et al.), has a lower potential variance but may be prone to bias, overestimating the true prediction error.5 Using LOOCV, there is no possibility of sampling variation in the prediction estimates; k-fold cross-validation involves the formation of random sampling sets. Results of LOOCV are therefore consistently reproducible. In fact, the classification results of the two methods are very similar.6 Importantly, however, in addition to LOOCV, we undertook a second form of model validation to ensure the robustness of our results.3 As described in detail, we performed LOOCV on training sets distinct from test sets of held-back data. Crossvalidated models were therefore built in the absolute absence of the independent test set, to ensure meticulous and optimal external assessment of their predictive ability.4 Furthermore, our results were also validated using longitudinal sampling. 10.1021/pr100085a
2010 American Chemical Society
letters
LETTER: Metabonomics in Ulcerative Colitis 3. Bjerrum et al. correctly stated that we created homogeneous cohorts by excluding vegetarians, patients on a therapeutic diet, and subjects with an intercurrent illness, suggesting that this was in some way detrimental to the study. In fact, we would contend that failure to do this would have been a significant error. Ours was the first report of urinary metabolic profiling in the inflammatory bowel diseases, and as such we applied the utmost rigor in avoiding the potential bias that ethnic, dietary, and comorbid differences could have introduced to the study. Reasons for this were exhaustively discussed and referenced.3 In their study, Bjerrum et al. did not comment on the ethnicity of the subjects, nor on the individuals’ diet. The participants are likely to have undergone purgative bowel preparation and fasting (no mention is made of this), which will undoubtedly have had profound influences on their urinary results, making interpretation of their findings difficult. The effects on the metabolites in the other biosamples studied are less certain and merit further investigation. 4. We excluded 18 CD patients and 17 UC patients from multivariate analysis because of the presence of significant acetaminophen and 5-aminosalicylate (5-ASA) resonances in NMR spectra, due to the obvious bias that these may introduce into multivariate models, and their potential to mask biologically relevant metabolites. For these reasons, we excluded the patients’ samples, rather than excluding the regions of the spectra which included the “xenometabolites” from all samples. Bjerrum et al. implied that this made our study “clinically irrelevant”. We were in fact taking great care to ensure that medications were not influencing the findings, as they may also have a systemic influence on other endogenous metabolites. Significantly, in our study, the results of patients taking medications were also corroborated in cohorts of individuals who were not taking any. It would clearly be possible to ask patients to omit their medication before sampling, if the technique is to be employed in a clinical setting, thus making it potentially highly clinically relevant. The scores plot for the UC:control comparison from our study is shown in Figure 1. In the study of Bjerrum et al., eight outliers were excluded from the colonic biopsy data set prior to multivariate analysis. Unfortunately, there was no discussion of why these individuals’ samples were outlying, nor of the implications of these exclusions to their study’s clinical relevance. Bjerrum et al. are to be commended for the “systemic” approach used, investigating four types of biological sample. However, there are several further issues arising from the study. They made no mention of why the regions δ 2.12-2.22 and δ 4.70-9.50 were removed from their urinary NMR spectra. This was presumably due to the presence of 5-ASA (plus/minus other exogenous) metabolites. This actually means that a substantial proportion of the urinary NMR spectrum has been excluded, representing manifold biologically relevant metabolites, including the aromatic region. So it is perhaps not surprising that, in contrast to our study, no differences were found between their cohorts.
Figure 1. OSC-PLS-DA scores plot from our previous study (one component model). This shows the separation achieved between urine samples of 60 controls and 43 ulcerative colitis (UC) patients. Outliers excluded as discussed in the text.
No explanation was given for the exclusion of regions of the spectra acquired from the colonocyte and lymphocyte samples, nor was there a discussion of the contamination of their biopsy samples with propylene glycol and ethanol and the potential influence of this on the presence of other biological metabolites. The collection of biopsy samples without contamination is eminently feasible, and analysis of such samples will be necessary to reproduce and verify the results. While it is interesting that quiescent could be distinguished from active UC, it is of note that inactive UC biopsy and colonocyte samples could not be distinguished from controls, raising the question of whether the changes may relate to gut inflammation per se rather than UC. The inclusion of patients with Crohn’s colitis, and/or other gastrointestinal inflammation, is clearly mandatory for future studies. Both our study and that of Bjerrum et al. highlight the considerable potential of metabolic profiling in this field and the need for careful phenotypic correlation.
References (1) Nicholson, J. K.; Lindon, J. C. Systems biology: Metabonomics. Nature 2008, 455, 1054–1056. (2) Bjerrum, J. T.; Nielsen, O. H.; Hao, F.; Tang, H.; Nicholson, J. K.; Wang, Y.; Olsen, J. Metabonomics in Ulcerative Colitis: Diagnostics, Biomarker Identification, and Insight into the Pathophysiology. J. Proteome Res. 2010, 9, 454–462. (3) Williams, H. R.; Cox, I. J.; Walker, D. G.; North, B. V.; Patel, V. M.; Marshall, S. E.; Jewell, D. P.; Ghosh, S.; Thomas, H. J.; Teare, J. P.; Jakobovits, S.; Zeki, S.; Welsh, K. I.; Taylor-Robinson, S. D.; Orchard, T. R. Characterization of inflammatory bowel disease with urinary metabolic profiling. Am. J. Gastroenterol. 2009, 104, 1435–1444. (4) Westerhuis, J. A.; Hoefsloot, H. C.; Smit, S.; Vis, D. J.; Smilde, A. K.; van Velzen, E. J.; van Duijnhoven, J. P.; van Dorsten, F. A. Assessment of PLSDA cross validation. Metabolomics 2008, 4, 81–89. (5) Hastie, T.; Tibshirani, R.; Friedman, J. The Elements of Statistical Learning, 2nd ed.; Springer: New York, 2009. (6) Mahadevan, S.; Shah, S. L.; Marrie, T. J.; Slupsky, C. M. Analysis of metabolomic data using support vector machines. Anal. Chem. 2008, 80, 7562–7570.
PR100085A
Journal of Proteome Research • Vol. 9, No. 5, 2010 2795
