THE FBI LABORATORY Wilder D. Smith takes a look at
L
ast spring, the Federal Bureau of Investigation’s (FBI) laboratory was uprooted from the J. Edgar Hoover building in downtown Washington, D.C., and replanted in the heavily guarded, lush green sanctuary of the U.S. Marine Corps Base in Quantico, Va. The move represented more than a breath of fresh air for the lab; it symbolized the turning over of a new leaf and the chance to rebuild the lab’s image following a controversial 1997 Office of the Inspector General (OIG) report. Before the new FBI lab’s roots could even settle, however, the lab was faced with some weeds in its new Garden of Eden— concerns about its bullet lead analysis and peer review procedures. A subsequent report on these issues by the National Academies’ National Research Council (NRC) prompted many in the forensic community to ask the million-dollar question, or rather, the $130 million questions: Has the FBI lab truly made a fresh start with its new move to Quantico, and what improvements has it implemented since the infamous 1997 OIG report?
its new location, and the latest
round of controversy surrounding it.
The makeover Before discussing the changes within the new FBI lab, one has to briefly revisit the 517-page report, which many deem as the catalyst for the lab’s makeover. In summary, three units— Explosives, Materials Analysis, and Chemistry Toxicology— and certain employees of the FBI lab were cited for violations and misconduct in forensic practices. Subsequently, the FBI lab complied with 40 recommendations for improvement in 12 areas. Dwight E. Adams, who became director of the lab in 2002, says that his lab is better and stronger today. “We have become © 2004 AMERICAN CHEMICAL SOCIETY
accredited, we have periodical external reviews, and we have strengthened our quality assurance programs,” says Adams. He reminds the public that the OIG report cited only 3 of his 25 lab units and that the findings shouldn’t be exaggerated. “I don’t think that investigation should diminish the quality of people and results this lab has put out since our inception in 1932,” he says. From the outside, the improvements to the lab are quite noticeable. The 463,000square-foot lab is completely covered with glass windows (rather than concrete like its old residence). However, inside it quickly becomes apparent that the lab’s attention to subtle details is what distinguishes it from the old lab. “For starters, the laboratories are separate from the offices of the technicians, a luxury not found in the old lab configuration,” says Marc LeBeau, chief of the chemistry unit in the lab. The cream-colored labs are brightly lit with seemingly endless counter space, and large bay windows run the length of each lab. Sealed rooms, controlled environments, and separate entrances in and out of the lab prevent evidence contamination. In addition, there are new labs specifically designed for trace evidence, including paint analysis. Perhaps the biggest difference between the new and old labs is the amount of space. Unlike the cramped conditions in the chemistry unit in the old FBI lab, where technicians routinely shared a workbench, each technician now has his/her own space. The lab is currently armed with about 30 analytical chemists, but LeBeau estimates that his unit can accommodate a 50% increase in staff if needed. Ironically, says LeBeau, hiring personnel to fill the space has been a chal-
the new FBI lab,
M A Y 1 , 2 0 0 4 / A N A LY T I C A L C H E M I S T R Y
175 A
FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORAT
lenge. “It’s often easier to add instruments than it is to add people in the government,” he jokes. The lab has made significant improvements in its analytical techniques, particularly in the area of MS. “What we have in this lab with MS is the ability to have sensitivity up to a picogram if needed,” says LeBeau. Improvements include new applications for LC/MS, inductively coupled plasma MS (ICPMS) for metal analysis, and better analytical equipment such as upgraded auto-
In discussing the issue of adequate peer review, both the FBI lab and other scientific organizations have to adhere to the guidelines of Daubert v. Merrill Dow Pharmaceuticals, Inc. This landmark case essentially redefined the standard for admitting scientific evidence in all federal and many state court cases. In that decision, courtroom judges were designated as “gatekeepers” to determine what is acceptable expert testimony by a witness and whether a theory or technique used by scientists can be generally
“You just can’t be in the forest or myopic world of your own discipline and think there is no need to ask anybody else if your practices are valid.” —William Tobin
samplers for pyrolysis examinations. “The latest and greatest area of MS for our unit has to be the acquisition and application of TOFMS,” says LeBeau. The lab is currently validating its TOFMS applications with LC/MS and GC/TOFMS and expects to have those results later this year.
Peer review In the scientific community, the definition of peer review usually encompasses a review of one’s work or procedures by both internal and external peers. The 1997 OIG report found the old FBI lab’s peer review process to be problematic by not allowing adequate internal peer review. Many members of the forensic community have concerns about whether the FBI lab has adhered to the OIG report’s internal peer review suggestions and about the lab’s external peer review process. “In today’s legal and scientific environment, you just can’t be in the forest or myopic world of your own discipline and think there is no need to ask anyone else if your practices are valid,” says William Tobin, a metallurgist and former examiner in the material analysis unit of the FBI lab. Adams refutes such concerns about his lab’s peer review process. “To imply that our internal peer review process is not adequate science diminishes what has happened prior to [the OIG report],” he says. “The lab routinely publishes large amounts of articles on its techniques in peer-reviewed scientific journals.” Adams spotlights as examples the lab’s three years of R&D on DNA technology and the dozens of articles on the subject the lab has published in scientific journals. “Peer review doesn’t stop there,” says Adams. He points out that in keeping with the accreditation process, DNA applications require external audits every other year. In addition, as part of the lab’s quality assurance program, proficiency testing is conducted every 180 days. “We’ve even commissioned studies to have a technique, technology, [or] application critiqued to make sure it does what we say it does,” he states. 176 A
A N A LY T I C A L C H E M I S T R Y / M A Y 1 , 2 0 0 4
accepted as “scientific knowledge”. Scientific organizations and expert witnesses use the Daubert rule as a measuring stick for reliability of theory or technique and for meeting the requirements for peer review. However, LeBeau believes meeting the Daubert standard isn’t as cut and dry for his lab as for other forensic labs. “We are in a catch-22 in that we have to struggle with having our work peerreviewed by the scientific community while at the same time not revealing too much about the case,” he says. “We can’t release information about a case without penalties.” Another problem the FBI has with the Daubert standard involves allowing judges to decide which techniques are acceptable, says LeBeau. “If you are the first or have only a couple of labs out there using a new application or technique such as TOFMS, you risk the judge not understanding it or not allowing it because nobody else has done it,” he says.
Bullet lead: Analytically indistinguishable? The FBI lab also faced controversy surrounding its bullet lead analysis. Vincent P. Guinn pioneered bullet lead analysis in 1960. In that year, Guinn and colleagues published a study called Forensic Neutron Activation Analysis of Bullet-Lead Specimens, which looked at using neutron activation analysis to determine whether two bullet lead specimens have common or different sources. The researchers looked at the analysis of three trace elements— antimony, arsenic, and copper—in bullet lead. They concluded that three points are not usually sufficient to definitively identify the bullets as having a common source. The report went on to suggest that no fewer than six elements should be measured to achieve definitive comparisons. Subsequently, four more elements—tin, silver, cadmium, and bismuth—were added to help better distinguish bullets from one another. FBI examiners have testified that they can circumstantially link a suspect to a crime when they compare their analysis of trace elements found in the lead of a crime scene bullet to an analysis of
ABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY
FBI LABORATORY FBI LABORATORY FBI LABORATORY
examine >800 samples of bullet bullets seized from a suspect. In lead provided by the FBI lab. the past, the lab used a controCarriquiry and her colleagues versial statistical analysis method were responsible for determining known as data chaining. Under the probability of error in the this technique, analysts sequentialmethod used to link two differly compared bullets from a crime ent bullets from a crime scene to scene to those recovered from a those found on a suspect. The suspect. The data from these results? “We concluded that the analyses are then put into groups probability can be non-negligiwithin which bullets are considble,” she says. ered compositionally indistinguishCarriquiry adds, “One of the able (CI). The chaining method things that chemists don’t think potentially allows bullets that were about when looking at bullet lead originally deemed distinguishable, is that there are two sources of but whose error bars overlapped error: The first results from the in range to those of a CI group, to analytical technique, or how you become part of that group, and come up with the composition therefore a “match”. or measurement of the trace eleTobin compares data chaining ments in bullet lead.” Carriquiry to an almost sleight-of-hand illunotes that her group did not sion. “It’s a case of the very same “We will be the world’s foremost forensic specifically critique the analytical bullets being exculpatory or nontechniques that the FBI lab uses incriminating one minute [withlab, and we will provide correct, in its elemental analysis of bulout data chaining], and then being lets, but she believes the methincriminating the next [with data unassailable results in a timely fashion ods were irrefutable. “But an anchaining], with no change in any alytical match doesn’t mean the of the substantive case-specific facts two bullets have a common orior issues,” he says. However, Tobin and objective testimony.” gin nor does it mean that the is careful not to imply any intensuspect was the one that left the tional deceit by the FBI examiners. —Dwight E. Adams evidence at the crime scene,” she “Their use of data chaining was says. The second source of error, not intended to deceive, but rather she says, is that the bullets being was a result of over-zealous efforts and/or law-enforcement bias to ‘please the customer’ [prosecu- analyzed are just a tiny sample of all the bullets in the world, so the statistical error can be significant. tors], or just plain not knowing any better.” Carriquiry says she is unsure of what became of their findLeBeau explains the intent of the lab’s bullet lead analysis procedures: “When you are doing a chemical or any other com- ings after submitting them to the FBI. She adds that several parison, the strongest conclusion that we can make when you months later, she was called to court to dispute the testimony have a “known” and something from a crime scene is that they of an FBI lab expert who testified that an analytical match sugare not alike or not from the same source. . . that is the goal of gested that the suspect had been at the crime scene. “I don’t our chemical comparisons,” he says. “When we say ‘analytically know if they didn’t believe our report or if we were unclear, but indistinguishable’, we mean that we simply couldn’t tell them I do know that the testimony was at odds with our conclusions,” states Carriquiry. apart,” he adds. LeBeau believes there was some confusion surrounding the A greater problem for many in the forensic community is the court testimony on the controversial bullet lead analysis by FBI FBI’s request of the Ames lab. “My understanding of the study examiners called as experts. Tobin explains that FBI experts tes- was that the Ames lab looked at something totally different than tify to the analysis of seven analytes in bullet lead. However, what we asked and made a judgment based on that informawhat lay jurors aren’t told is that three of the seven elements an- tion,” says LeBeau. The source of confusion he says stems from alyzed are almost never there. “You can go to court and claim essentially a “reference” database of bullet lead results that the you analyzed for all 100 or so elements in the periodic table . . . lab stores but never meant for use as an investigative tool. but what if I don’t tell you that 95 are not expected to be found in bullets?” he says. “This type of testimony could be deceptive NRC fires its shots and cloud the perception of lay jurors if we aren’t careful.” The FBI asked the NRC to examine its bullet lead analysis proceThis is not the first time the FBI lab’s statistical method for dure to determine whether there were ways to improve the science. bullet analysis has come under the gun. In May 2000, the FBI The NRC report was published on February 11, 2004; although commissioned a group of scientists—Alicia Carriquiry, Hal Stern, not as scathing as the OIG report, it did fire holes in the FBI’s and Michael Daniels—at the Ames lab at Iowa State University to bullet lead statistical analysis techniques. In addition, it made some M A Y 1 , 2 0 0 4 / A N A LY T I C A L C H E M I S T R Y
177 A
FB
FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORATORY FBI LABORAT
strong recommendations for the lab’s peer review process and future courtroom testimony by its technicians called as experts. The NRC report put to rest some key concerns of the forensic community over the lab’s statistical method of analysis. The report recommended that the FBI cease using the controversial data chaining method of analysis. The NRC also found room for praise, such as the lab’s use of ICP-optical emission spectroscopy (OES) in its bullet lead analysis. The analytical technique was found to be accurate and reliable. However, the NRC did recommend that the FBI consider alternative techniques to allow better differentiation among samples, specifically a new high-performance ICP-OES method. The NRC report concluded that examiners can continue to testify that two bullets from the same compositionally indistinguishable volume of lead (CIVL), defined as only one day’s production of similar composition lead, can be analytically indistinguishable. However, the report recommended that technicians also convey to jurors that one CIVL can produce as few as 12,000 and as many as 35 million 22-caliber bullets, and that an unknown number of CIVLs can be produced by the same or different manufacturers, with identical compositions. The report also recommended that to reduce the number of false positives in future bullet lead analysis, the FBI should evaluate the use of one of two “t-test” methods for statistical analysis and implement stricter quality control. A summary of the report can be found at www.national-academies.org/morenews.
178 A
A N A LY T I C A L C H E M I S T R Y / M A Y 1 , 2 0 0 4
The future The new FBI lab, despite its change of location and state-of-theart upgrades, still remains a lab under the microscope of the media and its peers. Adams says that the continuing scrutiny doesn’t surprise him and that his lab is up to the task. “Clearly, any organization this size in the government is going to have situations or employees that will fall under scrutiny. . . . That is to be expected,” he says. Adams wagers that the public’s perception of the FBI lab isn’t focused on the constant references to the OIG report, concerns surrounding the lab’s peer review process, and the bullet analysis controversy. “I think if you were to question the typical American citizen about what they think of the FBI laboratory, I don’t think they will point to the findings of the OIG investigation,” says Adams. “I think they will have the image of a recent case in Houston [in] which our lab was able to identify a child molester in 24 hours. . . . That is the image I think they will have of this laboratory.” Adams reiterates that the focus of the lab today remains the same as the day of its inception. “We will be the world’s foremost forensic lab, and we will provide correct, unassailable results in a timely fashion and objective testimony,” he states. “We are not interested in who it benefits; we are interested in the truth.” Wilder D. Smith is an assistant editor of Analytical Chemistry.
