GOVERNMENT AND SOCIETY
Potential misunderstandings Every situation is different, but according to many researchers who have participated in collaborations that include basic and clinical participants, initial expectations on both sides are often unrealistic. “I think that many clinicians have expectations that the basic scientists cannot deliver,” says Denis Hochstrasser, a clinician and a basic proteomics scientist at Geneva University and University Hospital. He explains that clinicians who don’t understand what is involved in a proteomics study will be disappointed with how long a discovery study takes or with the paucity of biomarker candidates that pan out.
Funding Patient’s perspective
Patient’s perspective Collaboration
Patient’s perspective
Collaboration
Accurate biomarkers
Redundancy
Redundancy
Instrumentation Redundancy
Instrumentation
Instrumentation
Validation
Validation Data context
Data analysis
Quick fix
Quick fix
Quick fix
Funding
Funding
Data analysis
Time
Inappropriate testing
Data analysis
Time
Inappropriate testing
Sample limitations
Time
play in the project. “The clinicians have their specific aims, but they have to understand that we are not just technicians who are going to run samples for them either. So, there is a give-and-take here,” says David Lubman, an analytical chemist who recently has joined the surgery department at the University of Michigan. Basic scientists also have some unrealistic or impractical notions. Because they were trained as developers and users of analytical instrumentation to solve basic biological and chemical problems, they typically do not know what types of tests will be appropriate in a clinical situation. Clinicians “bring
Inappropriate testing
Sample limitations
Expectations Expectations
The experience was enough to give someone whiplash. At a disease-oriented national conference a few years ago, leaders in the proteomics field organized sessions to showcase the potential of the science. During one session, which was heavily attended by clinical researchers, speakers and members of the audience had nothing but praise for a certain proteomics method. Listening to the exchanges, you’d think that finding protein biomarkers was a simple task. Just a few hours later, however, analytical chemists and basic proteomics scientists held a session in which they pointed out the numerous limitations of this technique and presented methods that they contended were more accurate, albeit more difficult to perform and with lower throughput. Basic researchers Are basic and clinical Basic researchers researchers really so Basic researchers Clinicians Clinicians far apart on the issues? Are clinicians naive Clinicians about the capabilities of current methods and instrumentation? Similarly, are there some aspects of biomarker research that basic scientists don’t understand? Do the two groups ever talk to each other?
Sample limitations
Several scientists on both sides of the equation point out, however, that clinicians are not the only ones who think that proteomics projects will have quick payoffs. Gil Omenn at the University of Michigan, who trained as a clinician and protein chemist, remembers that only days after the mapping of the human genome sequence was celebrated in 2001, the media already were shifting gears and touting the promise of proteomics. It’s easy for the public and scientists who are not involved in proteomics research to become too excited when newspapers and magazines oversell the capabilities of the technology, he says.
Expectations
Basic and clinical proteomics researchers: the great divide?
Validation
Data context
Data context
Accurate biomarkers
Collaboration
Accurate biomarkers
Clinicians might expect quick fixes, but Robert Cotter, an analytical chemist at Johns Hopkins University School of Medicine, points out, “You have to remember that their goal or focus is different from ours. [Clinicians] are always hoping that they can get something that, even if they don’t understand the mechanism, will actually alleviate pain or reverse a condition.” Although collaborations between the two types of researchers can work very well, basic scientists find that clinical teams may misunderstand the roles that analytical chemists or biochemists
840 Journal of Proteome Research • Vol. 7, No. 3, 2008
reality into the game because they are the ones that know exactly what is needed in the clinic, so you don’t go and try to identify markers or set up diagnostics that perhaps are never going to be used,” says Julio Celis, a basic scientist at the Institute of Cancer Biology and the Danish Cancer Society. Unfortunately, many basic scientists working on their own waste time and money pursuing biomarker candidates that will be worthless to physicians, say researchers. For example, some recent proteomics studies report possible protein biomarkers for myocardial
infarction, but Hochstrasser points out that good diagnostics for this condition already exist, so insurance companies are not likely to pay for yet another diagnostic technique. In addition, basic scientists can easily overlook the patient perspective when planning the sample collection protocol. Lubman says that he and his team want as much sample as they can get, but the clinicians they work with remind them that these samples are coming from real people, so only limited amounts of sample can be obtained. Louise Alldridge, a basic scientist at the Helen Rollason Heal Cancer Charity Research Laboratory of Anglia Ruskin University (U.K.), adds that working in the breast clinic with clinicians has made her realize what the patients have to endure. Currently, her team is analyzing blood samples from chemotherapy patients. “We are tracking the progress of these patients, and sometimes you find out that they can’t have their chemotherapy because they’re not very well. You get a lot closer to it, and you also get a feel for what’s doable,” she says. You learn that “these people are having so many nasty things being done to them, and you eventually tend to start thinking of things that are going to be less disruptive to what they’re already going through.” Clinicians note that their colleagues in basic science sometimes get prematurely excited about their research results. However, these studies often compare only healthy controls with patients who have a particular disease. Hochstrasser points out that biomarkers identified in this type of study are not yet ready for clinical use and should be validated further. “If you compare someone who has pulmonary cancer with a healthy person, you will immediately find biomarkers, but they are biomarkers showing differences between sick and healthy,” he explains. That doesn’t help, he says, because the doctor already knows the patient is sick. The question is, What does the patient have? Many diseases have common elements, such as inflammation, so specificity is important. “If you want to find a specific biomarker for small-cell carcinoma of the lung, then you should compare it versus adenocarcinoma, versus mesothelioma,” he says.
Working it out For a collaboration of clinical and basic researchers to be a success, scientists say that communication is key and that the participants should talk to one another regularly throughout the study. Serhiy Souchelnytskyi, a basic researcher at Karolinska University Hospital (Sweden), says that differences between clinicians and basic researchers quickly melt away after they discuss the issues. Barry Karger, an analytical chemist at the Barnett Institute at Northeastern University, has noticed that these interactions are becoming even easier over time. He says that ~30 years ago, clinical and basic researchers mostly kept to themselves. Now, the situation has changed and more scientists from diverse fields are coming together to share ideas. An important way to get everyone on the same page is for all of the collaborators to understand everybody’s role in the project from the start, says Karger. “The clinician brings obvious expertise, and the analytical people bring obvious expertise, but they really have to blend in with each other,” he adds. In his opinion, all of the participants should know why the project is being done, where the samples originated, and how the data will be interpreted. “If we understand why we’re doing it and what it means, then we will generate better data, and it will be more interesting and more exciting,” he points out. It is important for both parties to work out the procedures for obtaining and handling samples before the project begins, say veterans of such collaborations. Sometimes, initial misunderstandings can result in comical discussions. For example, Lubman says
the clinicians have had to remind his team that the samples from patients are available in limited quantities: obtaining a 100 mL blood sample from each patient is out of the question. “The clinicians have to explain to students and postdocs that only ~0.5 mL is available for the discovery,” he says. “Then they may come back to us and ask if we can run 450 of these samples, and we have to explain to them that we are not ready to run 450 samples at this time. We each have to understand where the other is coming from, but if you communicate, that seems to be resolvable.” On some occasions, basic and clinical scientists even work together to figure out details, such as the best buffers to use or the best way to get the samples from the operating room (OR) to the laboratory. Fresh samples are best for proteomics experiments, so Celis and his team actually wait in the pathology lab, which is downstairs from the OR at a nearby hospital. The surgeon cuts out the sample from the patient, puts it in a tube, and sends it through a chute to the pathology lab. Similarly, at the beginning of Alldridge’s collaboration, her team would stand beside the surgeons and nurses with an ice bucket. Arranging the sample collection required negotiation, because the long-established practices in the OR and in pathology had to change. For example, she notes that the surgery team was accustomed to putting the samples directly into preservative for pathology analysis, but this step is not ideal for proteomics analysis. (Currently, samples are collected directly from the OR and managed by a medical laboratory assistant in the pathology department who is funded by the same charity that funds Alldridge’s
‘No’ can be better than ‘yes’
Most people think that the purpose of a diagnostic test is to identify a person’s illness, and commercial tests are designed to yield such positive-predictive results. But when a patient comes into the emergency room with symptoms, a physician often wants to rule out certain conditions with tests that have negative-predictive value. For example, to rule out that a patient has a pulmonary embolism, a doctor performs a blood test to detect the levels of fibrin dimers, which are released from a dissolving blood clot. If the levels are low, a patient probably does not have a pulmonary embolism. However, if levels are high, the test is inconclusive. “As clinicians, we really like a test with negative-predictive value because it helps us to exclude something and [possibly allows us to] send the patient back home,” says Hochstrasser.
Journal of Proteome Research • Vol. 7, No. 3, 2008 841
GOVERNMENT AND SOCIETY group.) Likewise, Cotter’s team worked closely with clinicians to figure out the best way to handle samples of cerebrospinal fluid (CSF). “The clinicians knew better how to do this, because it turns out that trying to get CSF without getting blood contamination is really important,” he says. Data analysis is another key step where input from both types of researchers is vital, says Karger. Clinicians won’t necessarily be able to interpret mass spectra or protein microarray results, but they need to understand the technical challenges (such as dynamic-range issues) that are involved with obtaining the data. On the other hand, clinicians have access to detailed patient histories, which typically are off-limits to basic scientists, and this information helps put proteomics data into context. Without such close collaboration in the data analysis and validation steps, investigators run the risk of rushing biomarker-based assays to the clinic too soon. “We have a tremendous obligation to the public to not push out clinical tests that could potentially [do] harm,” says Ian Thompson, a clinical researcher at the University of Texas Health Science Center at San Antonio. He points out that false-positive and false-negative tests can have enormous ramifications
for patients—either a disease is not detected or a patient who really doesn’t have a disease is given unnecessary treatment or surgery. “The probability of false positives rises dramatically when such a test is used to screen healthy or even high-risk populations, rather than patients,” says Omenn. “The familiar PSA test for prostate cancer is a startling example. It is very useful to monitor treated patients for recurrence of tumor, but its predictive value for screening is negligible.”
already has begun to shift some of its funding strategies to cover larger collaborations, according to Karger. An example of NIH’s progress on this front is the creation of the Early Detection Research Network (EDRN), led by Sudhir Srivastava of the National Cancer Institute. To achieve EDRN’s mission of translating biomarkers from the bench to the bedside, the project brings together basic scientists and clinicians to rigorously validate potential biomarkers. “It is a monumental task to bring together disparate groups of peoMaking a match ple with varied agendas and cultural Additional challenges lie ahead for values,” notes Srivastava. Thompson, basic and clinical researchers. Several who conducts research on prostate, scientists say that promotion and tenure bladder, and kidney cancer, says, “The committees at various institutions still fascinating thing is that when you’re do not value collaborative research. sitting day in and day out with a breast “There continues to be this focus at the cancer researcher, for example, you national level on an investigator in a realize the issues and potential opporlaboratory cranking out identifications tunities are incredibly similar.” of markers,” says Thompson. To find clinical collaborators, SouCelis notes that funding mechanisms chelnytskyi pounded the pavement. “I must change to reward large collaborawas taking time to go to different places tive projects, which may take longer than and talk to as many people as possible,” a few years to produce concrete results. he says. When he spoke to potential col“So, you need some sort of security,” he laborators, he told them what his group says. A team cannot set up a venture and could do and asked the clinical investhen tell the patients after 2 years that tigators whether they would be a good the project has stopped because we “just fit. Lubman finds collaborators closer to didn’t get the grant,” he says. In the U.S., home because his department encourthe National Institutes of Health (NIH) ages such interactions. He often meets with his clinician colleagues at seminars to discuss possible Biobank warning opportunities. Sometimes, a The selection of the most appropriate samples for a retrospective clinical study can colleague in his department will be a herculean task and can take a lot of time, so clinicians and basic researchers refer another clinician to him for sometimes resort to samples of convenience that are stored in large repositories, a project. says Thompson. However, such samples may have biases that could lead a scientist Celis has found that clinicians down the wrong path. are a busy group, so he does some For example, lung cancer samples might have been taken from patients while they of the initial pathology analysis were under anesthesia during surgery, whereas the control samples might have been before getting others involved in taken from people who visited a hypertension clinic. A proteomics study comparing a project. “I’m a basic scientist, these samples might uncover a marker that seems to indicate the presence or absence but through my collaborations of lung cancer. “Then when you go out to a group of smokers and use these markers, with clinicians, I had to transthey may do a wonderful job telling you who’s under anesthesia and who has hyperform myself into a molecular tension,” cautions Thompson. pathologist,” he explains. He says In other cases, the clinical annotation for samples in biobanks is not complete, says that pathologists have to make Celis. Therefore, even if a researcher analyzes the descriptions that are included, una certain number of diagnoses known biases may be present. Omenn notes that some scientists do not even specify every day, so adding even five whether a blood-derived sample is actually serum or plasma or record which anticonew things to their schedules for agulant was used. In addition, samples collected in the past and stored in large bioa proteomics discovery study is banks weren’t necessarily collected with standard procedures that meet current rea burden. Therefore, he and his quirements. To address these problems, the U.S. National Cancer Institute (NCI) held team narrow down the candidate town meetings on the topic and, in June 2007, officials developed the NCI Best Pracbiomarker list, generate antibodtices for Biospecimen Resources document, which is available at http://biospecimens. ies, and then perform additional cancer.gov/practices. immunohistochemistry validation steps themselves. To avoid
842 Journal of Proteome Research • Vol. 7, No. 3, 2008
wasting pathologists’ time, only the best candidates are given to them for assessment. Once a pathologist is interested in some promising candidates, he or she will likely bring more clinical colleagues to the study. Many clinicians who conduct research were formally trained in medicine and basic science, so they are interested in developing research collaborations. Dennis Sgroi, who is at Massachusetts General Hospital, earned M.D. and Ph.D. degrees and performed postdoctoral research in a
genomics laboratory. As a translational scientist who works with patients, he says that he can identify clinical problems that could be solved with basic-science techniques. Translational science is a growing area, he adds. According to Omenn, NIH is revamping the clinical research training scheme for the U.S. “It’s a time of high expectations and what NIH Director [Elias] Zerhouni calls ‘transformation’ for clinical translational research,” he says. “NIH has invested billions and billions in basic science for medicine, and people
want to see results.” Omenn predicts that in the future, even more collaborations between clinicians and basic researchers will develop. “The emergence of validated biomarkers will probably be what will bring clinicians, basic scientists, and informaticians closer together,” he says. “Talking about things in the abstract is much less capable of attracting people’s detailed attention than having a useful application that can actually make a difference.” —Katie Cottingham
Proteomic profiling method not suitable for detecting prostate cancer
cohort. However, this new algorithm could not distinguish healthy from cancerous samples nor could it discriminate between samples from more aggressive versus less aggressive cancers. Semmes and colleagues note that a major challenge for biomarker research is minimizing the incidence of false discovery from biased samples. For that, researchers must spend the time and effort to put together enough ideal specimens. The authors say that their results do not indicate whether a particular method works and do not imply that previous studies were wrong. However, they do conclude that the particular SELDI TOFMS method used in their study has no diagnostic value for prostate cancer detection. Finally, they recommend that all other biomarker assays be rigorously validated.
to PubMed Central when they are accepted for publication. These articles will be made public no later than 12 months after the official publication date. In addition, as of May 25, authors must include the PubMed Central reference number when citing their NIHfunded articles in NIH applications, proposals, or progress reports. NIH joins public funding agencies in Belgium, Canada, France, Switzerland, and the U.K. that already have mandated open-access archiving. In addition, policies that strongly encourage open-access archiving have been implemented in Australia, Austria, and Germany. Most recently, the European Research Council mandated open-access archiving of publications and data from projects it has funded.
New law mandates open access for NIHfunded research The National Institutes of Health (NIH) has become the first public funding agency in the U.S. to mandate that publications resulting from its research grants be deposited in an open-access archive. The provision was included in the Consolidated Appropriations Act of 2008, which was signed into law on December 26, 2007. One requirement of the new policy (see http://public access.nih.gov) is that, starting April 7, electronic versions of final, peer-reviewed manu scripts that originated from NIH funds must be submitted
JupiterImages
The use of SELDI TOFMS for the early identification of cancers has been controversial. As a result of this and other questions about diagnostic approaches, the National Cancer Institute’s Early Detection Research Network organized a validation study of serum proteomic profiling to diagnose prostate cancer. Two recent papers published by a collaboration involving 29 researchers at 10 institutions present the results from the second stage of the validation study. (Clin. Chem. 2008, 54, 44–52; 53–60) In a previous publication, O. John Semmes from Eastern Virginia Medical School and colleagues determined in the first stage of the validation study that with frequent instrument calibration and automated sample preparation, the SELDI TOFMS method was sufficiently reproducible across laboratories. In addition, the decision algorithm could identify samples from cancer patients correctly when the samples originated from the patient cohort that was used to derive the algorithm. In the first of two new papers published by Semmes and co-workers, the decision algorithm could not distinguish between healthy and cancer patients correctly when applied to geographically diverse samples that were not from the cohort used to derive the algorithm. This result prompted the group to look for evidence of sample bias, which they found. In the companion paper, the authors compiled a new cohort of specimens that eliminated the sample bias, and they derived another decision algorithm from the new sample
For the public. Publications resulting from NIHfunded projects must be placed into an open-access repository starting April 7.
Journal of Proteome Research • Vol. 7, No. 3, 2008 843
