government &
policy
NIH Peer Review System Under Scrutiny Impenetrable report proposing overhaul of system raises chemists' alarm; institute backpedals furiously to contain damage William Schulz C&EN Washington
T
o keep pace with the "stunning successes of the biomedical research enterprise," a National Institutes of Health advisory panel is recommending fundamental changes to the agency's peer review system for extramural grants. The first set of proposed changes in the panel's two-phase study of the NIH peer review system is outlined in a draft report, and comments on that report will be accepted until Oct 15. § The examination of NIH peer § review is being conducted by "g the Panel on Scientific Bound- Q aries for Review, a working | group of the NIH Center for Sci- | entific Review (CSR) Advisory Committee. The panel is chaired by National Academy of Sciences President Bruce M. Alberts. The phase-one activity of the boundaries panel has been to create a framework for organizing reviewing committees by proposing a set of 21 integrated review groups (IRGs), which are clusters of scientifically related study sections. It is a reorganization of the current structure of 19 IRGs. In that reorganization, three current IRGs—including a biochemistry IRG— are not continued, and their research applications will be "widely dispersed." Other work in phase one of the examination is a statement concerning cultural norms that ought to govern IRGs and review panels, as well as procedures and principles that will be followed in phase two of the process. "It may be that some chemistry will shift to less chemically focused IRGs," says Stephen J. Lippard, professor of chemistry at Massachusetts Institute of Technology. Lippard and several other chemists have expressed deep concern about changing the NIH peer review system. By Lippard's assessment, 72
OCTOBER 11,1999 C&EN
"there is a potential for lots of fundamental research to get lost." Kenneth N. Raymond, vice chairman of the department of chemistry at the University of California, Berkeley, sent a heated e-mail around to colleagues when he first got wind of the changes. He says a concern he shares with many in the chemistry community is that there appears to be an absence of molecular sciences in what the boundaries panel
Alberts (left) and Ehrenfeld
has proposed, just when this discipline is increasing in importance. "I was trying to sound a note of caution," he says. "The NIH review system has worked extraordinarily well." "I'm going to be looking for a crisp, concise statement about the importance of medicinal chemistry" in health-related research, says Columbia University professor of chemistry Ronald Breslow. He says he is not aware of any chemist who believes that what the boundaries panel has proposed so far will lead to increased support for chemistry. The draft report and comment section—electronic submissions are encouraged—can be found at http://www.csr. nih.gov/bioopp/select.htm. Unfortunately, the report is written in a less-than-clear style with a reliance on confusing terminology. Not surprisingly, NIH says chemists have misread what has been pro-
posed so far, confused terms that have similar names but very different meanings, and drawn unwarranted conclusions based on the facts at hand. But the response from chemists has prompted NIH and the boundaries panel to clarify exactly what has been recommended and why. Panel members have stressed that they seek additional input from the entire biomedical research community—chemists, physicists, biologists, clinical researchers, engineers, social scientists, and people from many other disciplines who can be included under the broad rubric of "biomedical sciences." "Everybody knew it was going to be hard; everybody knew it was going to be controversial," says NIH Deputy Director Ruth L Kirschstein, referring to the overall examination of the agency's peer review system. But she also promises that the current proposed changes, as well as future proposals for change, will "be talked about in several iterations. Nothing is etched in stone." "The critical issue is active management of the peer review process at a time of rapid change in science," Alberts says. "What we're trying to do is change the structure of the system," he adds, in such a way that it will "improve the range and strength of [NIH-funded] biomedical or health-related research. We're really worried that we're missing many opportunities by not having this active management." Perhaps the primary issue Alberts and other panel members have had to confront is the conservative sentiment, "If it ain't broke, don't fix it." By the panel's own reckoning, the more than 50-year-old NIH peer review system "may be the most important single reason for the remarkable success of our federally funded biomedical research enterprise" [Science, 285, 666 (1999)]. But the foundation of success in the NIH peer review system, many other researchers say, has been recognition of the need for continuous improvement. A strategic plan for NIH, written in 1993, states: 'To ensure that the best science is funded, NIH must continually reevaluate and strengthen its peer review system." In 1995, a Report of the Working Group on the [NIH] Division of Research Grants—renamed the Center for Scientific Review—recommended that "study sections should be broadened so that they become more diversified, with
Challenge yourself. Completely. Your efforts are extreme. Expectations high. And your goal: the top.
Likewise in the lab. You want a solvent to face all the challenges at hand. Versatile UltimAR® high purity solvents. Solvents that scale all laboratory applications. Liquid chromatography. Extraction/concentration. IJV spectrophotometry. General purpose. Multiple uses, single solvent. Because UltimAR solvents satisfy every major lab purpose, they reduce inventoiy costs. Cut storage space needs. And minimize the risk of using the wrong grade solvent. Specify UltimAR universal solvents. From Mallinckrodt, an ISO certified, cGMP manufacturer with a 125-year record of outstanding performance. Plus, a rock-solid, high-speed distribution team. Call 1-800-354-2050. Versatile UltimAR solvents.
Mallinckrodt lab chemicals. Fiercely competitive™ http://www.mallchem.com UltimAR® and Fiercely competitive7" are trademarks of Mallinckrodt Baker, Inc. ©1999 Mallinckrodt Baker, Inc. All rights reserved.
CIRCLE 53 ON READER SERVICE CARD
g o v e r n m e n t & policy less attention paid to methodology or to a specific organ or disease. This increased breadth of focus must maintain pace with scientific development through periodic evaluations that include the involvement of respected scientists from outside NIH." Indeed, as the boundaries panel report states: "Through extensive outreach to the extramural community, as well as through her own assessment, CSR's director, Ellie Ehrenfeld, has identified a number of issues regarding study-section organization and composition. While these are subjective impressions that are hard to document, they are worthy of concern and sufficiently common to ask whether there are better ways to organize the review process." Ehrenfeld adds that her discussions with NIH institute and CSR staff indicated a need to accommodate newly emerging areas of science and new approaches to problems that were not being met in the current review structure. Ehrenfeld agrees that "data-driven actions are the ideal, [but] perceptions become reality, and it is incumbent upon
me as CSR director to respond to consistently voiced concerns of the community. I understand the need to verify that any proposed changes are, in fact, endorsed by a majority of researchers before those changes are implemented." "It is generally acknowledged that the way we are doing science has changed significantly in recent years as multidisciplinary approaches to increasingly complex problems are being explored and developed," Ehrenfeld adds. "It is reasonable to anticipate that the way we review that science may also have to change in some ways to accommodate the new pace and approaches." Chemists who have read the report say they fear that the proposal to drop the biochemistry IRG as well as repeated use of the term "biomedical" research indicates a reduced interest in chemistry on the part of NIH. But NIH officials insist that "biomedical" denotes all health-related science conducted by researchers as diverse as behavioral scientists and members of the allied disciplines—for example, chemistry, physics, engineering, and computer science.
'The boundaries panel had extensive deliberations on how best to incorporate chemical science applications into the review process in light of its increasing importance," Ehrenfeld continues. She says she hopes the panel will receive comments on the web from chemists that reflect their thoughtful consideration of the boundaries panel report. Not every constituent research group is represented on the boundaries panel. Membership was deliberately limited to 15 researchers, and members were selected "for their broad perspective gained by participating in the development of diverse fields of science and for their vision," comments CSR Associate Director Linda W. Engel. Care was taken to avoid a constituency-based group of scientists, she says. Rather, panel members were chosen "for their leadership and their abilities to transcend personal interests." Engel reiterates that the boundaries panel is relying heavily on input from members of all constituent communities in order to develop its final phaseone recommendations.
Complex synthesis puzzles? Lancaster offers answers - fast, practical solutions to complex commercial synthesis scale-up problems. With 90 fume hoods in the USA and Europe, plus 27 years of experience, Lancaster has the resources for fast, effective product development. And as part of Archimica - a global top 5 fine chemical company - we can see the process through to the finish - at any scale.
Lancaster!
We offer the direct solution 74
OCTOBER 11,1999 C&EN
Lancaster Synthesis lnc PO Box 1000, Windham, NH 03087-9977, USA Tel: 603-889-3306, toll-free 800-238-2324 Fax: 603-889-3326, toll-free 800-542-3939
CIRCLE 5 1 ON READER SERVICE CARD
www.lancastersynthesis.com
"Medicinal chemists could respond as a group if they wanted to," for exam ple, says panel member Peter H. Von Hippel, who is a professor of chemistry at the University of Oregon, Eugene. "Plans call for NIH staff and mem bers of all involved communities to work together in phase two to populate the IRGs with study sections," Engel says. She notes that the term IRG in the boundaries panel report has been a source of confusion, but it refers to the cluster of scientifically related study sections proposed in phase one. The re port gives examples of topics that might fall under each grouping. There has been no attempt to be in clusive, says the boundaries panel re port. 'The fact that medicinal chemis try, for example, is not included as a top ic is not a statement about its value," Engel says. 'The panel in phase one is only creat ing a skeleton," Engel continues. Any recommendations regarding actual study sections will be the work of phase two, she adds, "and it will be conducted by a combination of NIH staff and mem
bers of each of the involved research communities." Yet another incorrect conclusion drawn by some researchers is that only one or two experts in a given field will populate study sections, NIH maintains. Engel reiterates that no one has even begun to talk about placing actual re viewers on study sections because the study sections have yet to be created. In fact, one of the principles laid out in the boundaries panel phase-one re port states: 'When it is necessary or de sirable to review very different types of research in the same study section, at least 30% of the applications reviewed should be of each type. In addition, re viewers with expertise in each broad category of research should comprise no less than 30% of the total member ship of the study section." The principle of clustering similar sci ence together for review is a clear princi ple of study-section design, Ehrenfeld says. "Achieving 'critical mass' of applica tions and reviewers is essential for good review." "This is the concept paper; it's step
one," Engel says. "People tend to think at the level of the study section—that's the heartbeat." But she emphasizes that all the details will be worked out in phase two in partnership with the extramural community. "Groups of NIH staff and ex tramural scientists will work together, guided by the principles outlined in the phase-one report, to design the study sec tions that will populate the IRGs." Early in the panel's deliberations, Ehrenfeld notes, a group of chemists was consulted about the breadth and kinds of chemistry applications likely to come to NIH for review and the best way to divvy up chemical research for review. Some preliminary conclusions were reached, Ehrenfeld adds, and their de liberations will continue. The goal, she says, is to recommend committees ap propriate for review of chemistry appli cations that "integrate chemistry into the fabric of NIH, foster innovation, pro vide for dynamic evolution of study sec tions, and encourage new science that may not fit into the current review structure."**
Is there a better method for process control? Flow Injection Polymer Analysis There is now. provides fast, precise, accurate, and reliable determination of Φ Molecular Weight Φ Molecular Size Φ Intrinsic Viscosity Φ % Polymer without the need for elaborate sample preparation.
£^VV /IC^ZI—I—Hhj—I^T ™ Introduces the Model 300 TDA Ph: 8 0 0 - 3 7 5 - 5 9 6 6 or 281-445-5966 · Fax (281) 931-4336 · www.viscotek.com CIRCLE 91 ON READER SERVICE CARD
OCTOBER 11,1999 C&EN
75
