Environ. Sci. Technol. 2009, 43, 5545–5546
Response to Comment on “Household Water Treatment in Poor Populations: Is There Enough Evidence for Scaling up Now?” We thank Clasen and colleagues (1)for their response to our article on scaling up household water treatment (2). This is the sort of debate we were hoping to encourage, and we are sure will help policy makers in making the right decisions on this issue. We would like to emphasize that nowhere in the article do we reject household water treatment altogether. There may well be situations in which it could potentially deliver health benefits (as we emphasize repeatedly). However, it seems appropriate to revisit briefly the basic problem of the currently available evidence. Three blinded randomized controlled trials have been conducted in developing countries and none of them showed ANY disease reduction. We found no evidence that these trials were in other aspects of inferior quality compared to the unblinded trials reporting very large disease reductions. In many other fields of public health, this puzzling contradiction between blinded and unblinded studies would raise considerable alarm. In this light, our suggestion that further blinded trials are needed to come to a definite conclusion is modest. Perhaps the household water treatment protagonists may want to pool their resources and carry out such a trial. Apart from the letter by Clasen and colleagues, we received many, at times emotional responses to our article, ranging from strong support to outright rejection (see http:// www.irc.nl/page/47994). To suggest that any further debate on scaling up household water treatment is unwarranted seems premature. Public health has a long and unfortunate history of scaling up well-intended interventions that after rigorous evaluation at a later stage had to be abandoned. For researchers to deny the need for a scientific debate perhaps for fear of “confusing” matters for policy makers is setting the scene for a wrong health policy decision. Our response to the specific points raised by Clasen and colleagues are as follows: (1) The statement by Clasen and colleagues that 850 million people in the world already use household water treatment is at odds with Clasen’s recent statement that “current HWTS (household water treatment and safe storage) efforts represent only 1.7% of the 1.1 billion without access to improved water supplies and even a smaller percentage of those whose water is microbiologically unsafe. At the current pace, HWTS will not cover 100 million users until 2015” (3). (2) It is in our view not straightforward to apply the bias estimate found in unblinded clinical trials directly to community-based HWT trials where because of the high social desirability of most interventions bias may be much larger. It makes more sense to use the comparison between blinded and unblinded household water treatment studies as the best available estimate for bias in this context. (3) Objective outcomes: The Mahfouz study did not collect microbiological samples from the control group due to “lack of compliance” (another strong indicator of bias), so no comparison is possible (4). Quick and colleagues report a reduction in Campylobacter positive rectal swabs, but the 10.1021/es901311c CCC: $40.75
Published on Web 06/12/2009
2009 American Chemical Society
referral process for obtaining rectal swabs was not unbiased itself because it relied on reported diarrhea (5). Austin reported a reduction in malnutrition ascribed to household water treatment (in the absence of a diarrhea reduction!), but the statistical support for this statement was very low (p ) 0.44) (6). Crump reported a reduction in mortality, which is noteworthy (7). However, as the authors noted this was a surprise finding as mortality was not the primary study outcome. The problem is that the vast majority of HWT trials did not report on mortality but are likely to have done so had they found an impact. Nevertheless, the Crump trial can be seen as an invitation to conduct a large mortality trial, for example, along the lines of the large trials for bed nets. (4) The study by Jain and colleagues (8) does add to the evidence. First, it certainly does not make it more likely that further blinded trials will show an effect. Second, on the basis of the findings, one may hypothesize that safe storage is sufficient, and that no further treatment is required. (5) We included the studies from North America and Australia because they have not all “shown no health benefits”. A comparison between blinded and unblinded trials from these settings confirms the large potential for bias in diarrhea studies. (6) We are surprised that Clasen and colleagues are not more concerned about unblinded studies showing a large effect on diarrhea despite low use. Bias is the most parsimonious explanation for this discrepancy. This was confirmed by a recent trial in Ethiopia testing the effect of a portable filter device on diarrhea. The study found a 25% reduction in diarrhea despite only a very small minority of participants reporting use of the device with any consistency (9). (7) The statement that the only trial on safe storage “showed no statistical effectiveness” is in our view incorrect. In fact the trial showed a -21% reduction in diarrhea with a confidence interval of +2% to -39%, i.e., just including 0% (9). Public health research has long moved away from rejecting an intervention purely on the basis of an arbitrary cutoff point for significance. The results may of course also be due to bias. However, as we outline in the article, safe storage (and perhaps also boiling) requires less evidence as to its effectiveness than other methods of household water treatment. Also, if safe storage is not effective, then why do Clasen and colleagues believe that safe storage can partly explain the lack of effect of chlorination in the trial by Jain et al. (8)? (8) We do not contest that the private sector can have a very useful role in public health, particularly in the development and dissemination of interventions. But the private sector can also have a role in biasing evidence toward finding an effect and in influencing public health policy in the wrong direction (10). In fact the most obvious result of private sector involvement in household water treatment is that there are very few trials on boiling and safe storage, which do not necessarily require commercial products, repeat purchases, and a supply chain. Clasen and colleagues can be commended for registering all their trials at a trial register but to suggest that this is standard practice in the field is in our view overoptimistic.
Literature Cited (1) Clasen, T.; Bartram, J.; Colford, J.; Luby, S.; Quick, R.; Sobsey, M. Comment on “Household Water Treatment in Poor Populations: Is There Enough Evidence for Scaling up Now?” Environ. Sci. Technol. 2009, DOI: 10.1021/es9008147. VOL. 43, NO. 14, 2009 / ENVIRONMENTAL SCIENCE & TECHNOLOGY
9
5545
(2) Schmidt W.; Cairncross, S. Household water treatment in poor populations: Is there enough evidence for scaling up now? Environ. Sci. Technol. 2009, DOI: 10.1021/es802232w. (3) Clasen, T. Scaling Up Household Water Treatment: Looking Back, Seeing Forward; Public Health and the Environment, World Health Organization: Geneva, 2008. (4) Mahfouz, A. A.; Abdel-Moneim, M.; Al Erian, R. A.; al Amari, O. M. Impact of chlorination of water in domestic storage tanks on childhood diarrhoea: a community trial in the rural areas of Saudi Arabia. J. Trop. Med. Hyg. 1995, 98, 126–130. (5) Quick, R. E.; Venczel, L. V.; Mintz, E. D.; Soleto, L; Aparicio, J.; Gironaz, M.; Hutwagner, L.; Greene, K.; Bopp, C.; Maloney, K.; Chavez, D.; Sobsey, M.; Tauxe, R. V. Diarrhoea prevention in Bolivia through point-of-use water treatment and safe storage: A promising new strategy. Epidemiol. Infect. 1999, 122, 83–90. (6) Austin, C. J. Investigation of In-House Water Chlorination and Its Effectiveness for Rural Areas of the Gambia. Ph.D. Dissertation. Tulane University School of Public Health and Tropical Medicine: New Orleans, LA, 1993. (7) Crump, J. A.; Okoth, G. O.; Slutsker, L.; Ogaja, D. O.; Keswick, B. H.; Luby, S. P. Effect of point-of-use disinfection, flocculation and combined flocculation-disinfection on drinking water quality in western Kenya. J. Appl. Microbiol. 2004, 97, 225–231. (8) Jain, S.; Sahanoon, O.; Blanton, E.; Schmitz, A.; Imoro, T.; Hoekstra, M.; Quick, R. The Impact of Sodium Dichlorisocyanurate Treatment on Household Drinking Water Quality
5546
9
ENVIRONMENTAL SCIENCE & TECHNOLOGY / VOL. 43, NO. 14, 2009
and Health in Peri-Urban Ghana: A Randomized PlaceboControlled, Double-Blinded Trial. International Symposium and 4th Annual Network Meeting, Accra, Ghana, June 2-5, 2008. (9) Boisson, S.; Schmidt, W. P.; Berhanu, T.; Gezahegn, H.; Clasen, T. Randomized Controlled Trial in Rural Ethiopia to Assess a Portable Water Treatment Device. Environ. Sci. Technol. 2009, in press. (10) Roberts, L.; Chartier, Y.; Chartier, O.; Malenga, G.; Toole, M.; Rodka, H. Keeping clean water clean in a Malawi refugee camp: A randomized intervention trial. Bull. World Health Organ. 2001, 79, 280–287. (11) Bhandari, M.; Busse, J. W.; Jackowski, D.; Montori, V. M.; Schunemann, H.; Sprague, S.; Mears, D.; Schemitsch, E. H.; Heels-Ansdell, D.; Devereaux, P. J. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. Can. Med. Assoc. J. 2004, 170, 477–480.
Wolf-Peter Schmidt* and Sandy Cairncross London School of Hygiene and Tropical Medicine, Keppel Street, London WC1E 7HT, United Kingdom * Corresponding author phone: ++44-(0)20-7927-2461; fax: ++44(0)20-7636-7843; e-mail:
[email protected].
ES901311C
