Capturing and Reporting Electronic Data - ACS Publications

Downloaded by NORTH CAROLINA STATE UNIV on December 31, 2017 | http://pubs.acs.org Publication Date: August 1, 2002 | doi: 10.1021/bk-2002-0824.ch024

Chapter 24

Electronic Data Submission: Pilot Efforts in the Office of Pesticide Programs, U.S. Environmental Protection Agency Kathryn S. Bouvé Office of Pesticide Programs, U.S. Environmental Protection Agency, Mail Stop 7502C, 1200 Pennsylvania Avenue, N. W., Washington, DC 20460

The Office of Pesticide Programs (OPP) has embarked on a series of pilots with pesticide registrants to develop a standard and process for accepting and reviewing electronic data submissions. OPP is seeking an electronic submission standard that strikes a good balance between the needs of registrants and data reviewers. The standard must be inexpensive and easy for the wide range of U.S. registrants to implement. It must also provide OPP reviewers with easily learned functionality that makes their work more efficient and effective. Other interests to be served include data integrity, protection of confidential business information, and international harmonization. While Adobe Acrobat 4.0 has been selected as the tool for pilot efforts, keeping pace with emerging technologies will present additional challenges. The results of early pilots as well as discussion of the many issues and concerns related to electronic submission and review will be presented.

Background The Office ofPesticide Programs (OPP) ofthe U.S. Environmental Protection Agency is responsible for regulating pesticides. Its mission is to protect public health and the environmentfromthe risks posed by pesticides and to promote safer

188

U.S. government work. Published 2002 American Chemical Society

Garner et al.; Capturing and Reporting Electronic Data ACS Symposium Series; American Chemical Society: Washington, DC, 2002.


189

means of pest control. OPP is essentially a licensing operation that registers pesticides for sale and use. There are approximately 20,000 U.S. registered products that include insecticides, herbicides, fungicides, rodenticides, disinfectants, sanitizers, and repellents. There are approximately2,000registrants and the number of product registrations they hold rangesfrom1 to 541. The median is 2. Six percent of registrants hold 70% of active products. Extensive information on the risks and benefits of using a pesticide must be submitted by applicants before it can be registered. This information includes reports of research in the scientific disciplines appropriate to the intended use of the pesticide. Applications for new uses for registered pesticides must also be presented with appropriate data. A pesticide can be registered only if it is determined that it will perform its intended function without unreasonable adverse effects on human health and the environment. That determination is made by scientists in the Office of Pesticide Programs upon review of the studies and assessment ofinformation about hazards and exposure. Further, as new risk issues emerge, the Agency can require that additional studies be conducted and submitted throughout the life of a pesticide. OPP determines the data requirements for various types and uses ofpesticides and designs the protocols for testing the pesticides. Registrants bear the cost of conducting the studies according to the standard protocols. Full reports of the studies must be submitted to OPP along with the application for registration. In the case of pesticides intended for food uses, a petition to set a tolerance, or maximum residue level in food or feed, must also be submitted with required data. The data requirements for pesticides vary widely depending on the proposed use of the product. At one extreme, a recent application for registration of a new food-use chemical used in an outdoor setting was accompanied by 230 different studies. Basic chemistry, acute and chronic toxicity, worker and dietary exposure, as well as environmentalfeteand ecological effects must be analyzed by OPP. At the other extreme, an application for a new product containing an alreadyregistered - and well studied - active ingredient requires only chemistry data and a standard battery of six acute toxicity studies. OPP scientists review and assess each study and prepare written documentation of their findings. During this process, basic questions are answered: Was the study conducted according to the protocol? Does the reviewer agree with the results and their interpretation presented by the study author? Does the study satisfy the data requirement it addresses? The reviewer will re-analyze the statistics and re-calculate values in tables. OPP's review document must include details about how the study was conducted and key data tables presented by the study author as well as the reviewer's conclusions. Review documents serve as the building blocks for higher tier science decision documents including hazard assessments, exposure assessments, and risk assessments. The reviews must be


190


archived by the program to document the bases for registration decisions and they may be re-used or re-analyzed in the future. Performance and submission ofresearch reports by registrants and the review and assessment of those data by OPP staff comprise the essential interaction between the regulated and the regulators in the pesticide program. Finding more efficient and effective ways of conducting that 'conversation' is an important goal of the program.

Significance of Study Data in Pesticide Regulatory Program Study data represent a major level of activity in OPP. Over the years, the program has developed guidelines for nearly 200 different types of studies for conventional, biochemical, and microbial pesticides. OPP receives approximately 8,500 studies per year. The work of over 5,500 laboratories is represented in OPP's inventory of280,000 studies. Because of the high volume of studies reviewed by OPP and the essential nature of the work, a significant proportion of OPP's resources are dedicated to it. Approximately 40% of staff in OPP - 325 scientists - are involved in the primary or secondary review of studies. Primary study review work is supplemented by contractor resources as well. Registrants also face challenges as they develop and submit data to OPP. They put significant expense and effort into conducting research, preparing reports, and assembling them into application submissions that meet OPP's data requirements and formatting standards. A registrant may spend $10 to $15 Million to conduct 100 or more OPP-required studies for a food-use pesticide. Registrants also need to preserve and archive the valuable data they develop. A well-conducted study can satisfy a data requirement for many years and other registrants may cite those data in their applications provided they compensate the owner of the data. Studies may be performed over many years in many labs in different countries. Resulting reports may be developed using different software packages for word processing, spreadsheet presentation, and statistical analysis. In some cases the only available medium of a still-valid study will be hard copy. During Fall 1999, significant factors converged that gave impetus to OPP's pilot efforts. OPP management needed efficiency improvements to address the high volume and high value of studies and their review as well as to meet growing demand on the program for regulatory decisions under statutory deadlines. OPP's technical infrastructure and staff capabilities had matured to the point where technology could be put to more sophisticated use. The program was aware of viable technology in use in industry, other EPA offices, and other government agencies. Finally, in October, 1998, Congress passed the Government Paperwork



191

Elimination Act (GPEA). This law mandates that by the year 2003, all federal government agencies must be prepared to accept electronic submission of whatever information they requirefromthe public, the states, orfromindustry. This must be achieved using open standards and non-proprietary software and hardware to the extent possible. As a result of these factors, OPP initiated a series of pilots to test the use of Adobe Acrobat Portable Document Format and related tools as the standard for electronic submission and review of study data. It is expected to be a two-year effort because it will take time to involve a good cross section of registrants, studies, and reviewers in the pilots.

Adobe Acrobat and PDF Before proceeding with a description of the pilots, it is important that the terminology about technology used in this papa* is well understood, especially PDF, Adobe Acrobat 4.0, and Acrobat Reader. PDF is an acronym for Portable Document Format. PDF is a file format created by Adobe which enables a user to view and print a file exactly as the author designed it without needing the application or fonts used to create the file. Fonts, colors, images, and layouts are preserved. PDFfilesare compressed thus reducing file size, transfer time, and storage space needs. It was introduced in 1993 and Adobe describes it as an open, de facto standard. A number of software packages are designed to save documents as PDF files, and it is likely that more will emerge in the future. Examples are the Corel WordPerfect version 9 and Adobe products such as PageMaker. At the present time, however, to create a PDF file with the enhanced features desired by OPP, the user needs the software package Adobe Acrobat. It converts word processing and spreadsheetfilesas well as scanned images into PDF and preserves the look and feel of the original. The user can create bookmarks and links as navigation aids and can import electronic versions of tables and spreadsheets that retain their native format. OPP specifies Adobe Acrobat 4.0 for its pilots. The software must be purchased and the price is about $240 for a single user license. Adobe Acrobat Reader is software that permits users to view, navigate, search, and print Adobe PDFfileson major computer platforms. The software isfreeand availablefromAdobe's web site. While the Reader is useful, it does not support all the functionality OPP's reviewers require. For the pilots, OPP is purchasing Adobe Acrobat version 4.0 to support review of studies submitted as PDF files. Adobe Acrobat permits review, mark up, annotation, extraction of text and tables for editing or other manipulation in addition to basic viewing, navigation, and printing capabilities.


192


Design and Implementation of Pilots Adobe Acrobat and PDF emerged as the technology tools for OPP's pilots for a number of reasons. It is inexpensive. Itis widely used in areas well beyond the pesticide program and can be considered a defactostandard. Adobe supports the product, seeks user feedback, and continues to increase its functionality. PDF has been selected as a standard for electronic submission of new drug applications by the Food and Drug Administration (FDA). FDA's functions are similar to OPP's and there is some overlap between the regulated communities of both agencies. OPP's sister office, the Office of Pollution Prevention and Toxics (OPPT), has selected PDF for electronic submission of data. OPP was also persuaded by the feet that PDF was suggested by several major registrants that submit large data packages. PDF lends itself to structured formats through use of a feature called "bookmarks." OPP can build on this in the future if more highly structured formats are desired such as XML. Finally, Adobe PDF is consistent with the basic requirements under GPEA. Please note that OPP also identified the need for electronic submission of other types ofprogram information. One isfilesand data that supplement studies. Another is the text of current and proposed product labels. These are topics of other chapters presented in this book. OPP established operating principles for the pilots related to electronic submission and review of studies. First and foremost, the electronic formatting standard must strike a good balance between registrants' and reviewers' needs. It must be inexpensive and easy for registrants to adopt. It must provide reviewers easily learned functions that make their work more efficient and effective. Another principle is to build on the lessons learned by others. OPP has benefitted from the experiences ofthe Food and Drug Administration, the Office ofPollution Prevention and Toxics, registrants, Canada's Pest Management Regulatory Agency pilot efforts, and early electronic submission efforts in the European Union. As OPP embarked on thefirstpilots, it specified what it hoped to learn from them. From the registrants' perspective, can PDF be readily producedfromthe variety of sources available (electronic, scanned images, etc.)? Are OPP's technical specifications clear and achievable? Can OPP ensure the integrity ofthe electronic version of the subm ission? From the reviewers' perspective, OPP hoped to learn if Adobe Acrobat and PDF would meet a variety of requirements. Is Acrobat easy to learn? Are OPP's work stations economically suitable for this type of work? How useful are the tools to review the data and to prepare review documents? Specifically, do the tools allow reviewers to navigate among studies and within a study, perform full text searches, annotate, view and print, manipulate data including exporting it to



193

other software for further analysis, and excerpt and edit text or tables? The bottom-line question for the program is this: Is the electronically-assisted review process more efficient and more effective? OPP established criteria for selecting candidates for early pilots. The studies should support an application for a new use for a registered pesticide. The data review must be scheduled on the current priority work plan. A small number of studies in one discipline that had already been submitted on paper seemed most manageable. The program would avoid the complication of piloting with studies that contained confidential business information (CBI). And most importantly, OPP needed a registrant ready, willing, and able to create PDF documents from the original electronic source. In order to initiate even a simple pilot, OPP had to prepare a number of documents that would guide registrants and OPP staff on preparation and inprocessing of electronic versions of studies. With the assistance of stafffromthe registrant Rhone-Poulenc (now part of Aventis), OPP developed technical specifications on how to prepare the PDF files using Adobe Acrobat 4.0. This document was based on specifications developed by FDA. OPP also prepared a document that specifies the process by which registrants would organize and name the PDF version of studies on compact disk and how OPP would in-process them. The in-processing includes virus scans, verifying compliance with formatting requirements, and posting the studies on the OPP LAN. It was also essential for registrants to certify that the information in the electronic version of the study was the same as the paper version of the study, so a Certification with Respect to Data Integrity was developed. Finally, OPP developed a Reviewer Assessment form that would capture the reviewer's experiences during the pilot. Topics addressed include the learning curve, the performance ofthe tools, ergonomics, LAN and PC performance, and the efficiency and effectiveness of the tools compared to the 'paper' process. To undertake the first pilot, OPP staff contacted registrants that had applications and studies in house and on the priority list for review and asked if they were willing to participate in the pilot. From among them Rohm and Haas volunteered and submitted an electronic version of 7 residue chemistry studies submitted in support ofa tolerance petition. OPP's Health Effects Division (HED) selected a reviewer for the data, a staff person comfortable with technology and experienced with this type of data. The results of thefirstpilot were very encouraging. The reviewer found the software easy to learn in all categories of functionality that were used. The annotation feature was not used. Tools supporting navigation, text searches, viewing and printing were scored 'excellent.' Tools to support excerpting and editing text were scored 'good'. Tools to support excerpting and editing tables and the manipulation of data for export to Excel were scored 'average', It was noted by the reviewer that a good knowledge of Excel was needed to get the data into a



194

usable form. For this reviewer, the size of the monitor at home (17-inch) and office (21-inch) was adequate. The desk, chair and mouse posed no problems. The OPP LAN system performance and response times were acceptable. The critical requirements of improved effectiveness and efficiency were supported by the PDF and Acrobat tools. The reviewer could better analyze the data because it was easier and faster to check data against conclusions and statistics. This was accomplished by switching back and forth between linked tables of raw data and the summaries. Regarding efficiency, the reviewer found improvement in nearly all aspects of the work. Data requiring re-analysis could be dropped into the analysis software via drag and drop rather than by retyping and error checking the data. Selected textfromthe study could be excerpted and dropped into the review document, with proper attribution, without loss of accuracy that may occur during the usual paraphrasing and summarizing of the information. Suggestions for future pilots were also identified in thereviewer's assessment. For example, it would be more helpful ifPDF tables were derivedfromtrue tables created by the 'table' function of the original word processing files. That suggestion was added to the technical specifications document. Buoyed by the encouraging results ofthefirsteffort, more pilots are underway or planned. This includes entire submission s presented in PDF in early2001. The goal is to involve more registrants, different types of data, and more reviewers both OPP staff and contractors. It is also assumed that PDF submissions will be a mix of studies derivedfromelectronic sources, scanned images of paper with text captured via optical character reader (OCR), and scanned images (photographs, chromatograms, etc.). OPP is eager to gain experience in all these areas.

Issues and Concerns While pilot efforts continue, OPP must address a number of other issues and concerns in order to make electronic submission and review routine. Communication and Coordination - Internal and External There are other opportunities to use electronic submission tools and processes to make pesticide registration work more efficient and effective. OPP has formed the Electronic Data Submission Workgroup with representatives from all Divisions to explore those opportunities. The goal of the workgroup is to develop an electronic data submission approach that achieves operating efficiencies



195

through the promotion and facilitation of the electronic submission process including the delivery, review, data interchange capability and archiving of data supporting national pesticide registration. This approach will be implemented using current technology, consider the needs of reviewers and stakeholders, and address legal, archival and other requirements. Sub-groups were formed which are developing and refining guidance on the overall formatting of electronic submissions and typical studies as well as supplemental files. A web site has been established to make available in one place information about OPP's pilots efforts, other workgroup activities, and guidance documents and technical specifications. The URL for the site is www.epa.gov/oppfeadl/esr_pilots.htm. A close working relationship with pesticide registrants is essential in all these efforts. Much ofthe progress to date on pilots and related guidance documents has been the direct result of consultation, brain-storming, and information-sharing with individuals in several registrant companies. In March 2000, the American Crop Protection Association, a trade association ofmajor pesticide registrants, and EPA conducted a joint workshop on Electronic Submissions far Pesticide Registration. The agenda included presentations on electronic submission from industry and government perspectives, a demonstration of a PDF submission, and breakout sessions on creating and submitting overall submissions and supplemental files. The break out sessions were excellent opportunities for representatives of industry, laboratories, and regulatory agencies to critique early versions ofguidance documents and specifications. The ideas and suggestions that emerged were used to revise - and greatly improve - interim guidance documents for the pilots. Another close and fruitful working relationship is that between OPP and Canada's Pest Management Regulatory Agency (PMRA). PMRA is also hard at work on developing a standard for electronic submission and OPP has learned a great dealfromtheir early pilot efforts. As NAFTA partners, OPP and PMRA are working to harmonize the electronic data submission and review standard and processes to the greatest extent possible. Infeet,this work is being conducted under the auspices of the NAFTA Technical Working Group on Pesticides. Both OPP and PMRA are active in international harmonization efforts through the European Commission and the Organization for Economic Cooperation and Development (OECD). OPP continues communication and coordination on electronic submission standards and processes with its sister office, the Office of Pollution Prevention and Toxics (OPPT), and EPA's Office of Environmental Information (OEI). In addition, FDA is an especially useful source of information. FDA staff have worked hard on these issues for the last several years and they generously share their knowledge and experience with OPP.


196


Technological Challenges Without a doubt, the fast-changing technical environment presents great challenges to OPP and industry as they work to develop and implement an electronic data submission and review standard. While speed and change define technology today, OPP, by contrast, is a large and somewhat conservative organization. The many individuals involved in data review vary considerably by age, experience with technology, and work habits. There are approximately 25 organizational units with data reviewers in OPP. Managers of those units must implement electronic review tools consistently in terms of training and adherence to processes and procedures. The same is true for staff overseeing contract work. OPP's information technology infrastructure must be continuously upgraded to ensure adequate work stations, sufficient LAN storage for electronic versions of studies, effective virus detection capabilities, and data security and integrity. The infrastructure must serve OPP's needs and, at the same time, remain within Agency and Federal government standards. Costs must be within the program's budget. The registrant communityfecessignificant challenges related to technology on several dimensions. Companies that develop and submit data are generally large, complex organizations that mayfecework force issues similar to OPP's. Data submitted in support of a single registration may have been developed over many years in different labs in different countries using different data collection and analysis software and word processing packages. In addition, the pesticide industry has been quite volatile in recent years with companies merging, splitting, buying and selling large blocks of products and related data. It is likely that these factors complicate efforts to establish and maintain standards for electronic archiving and submission of study reports. All this variability within the regulator and regulated communities argues for inexpensive, simple,flexibletechnology solutions. At present, Adobe Acrobat 4.0 best meets those needs. While Adobe Acrobat and PDF are a logical choice for today, this technology may be completely obsolete in five years. No where is this dilemma better illustrated than in the issues faced by the National Archives and Records Administration (NARA). Studies submitted to the pesticide program are considered permanent records according to the record disposition schedule approved by NARA. A copy of every study received by OPP is sent to NARA's Federal Record Center where it is retained for 20 years. After that, the document is transferred to the National Archives for permanent storage. At the present time the only media NARA accepts for storage of permanent records are paper and microfiche. Digitized media are not allowed. Many government agencies are acquiring and generating electronic versions of records scheduled for permanent


197

or long term retention. NARA is tasked withfindingsolutions to this problem in ways that are technologically feasible and cost effective.


Conclusion

OPP will continue efforts to establish a standard for electronic submission and review of pesticide prog-am data through a series ofpilots. OPP is encouraged by the success ofdie early pilot efforts and by the interest and cooperation of industry and government agencies to date. The pilots will allow industry and OPP to determine ifAdobe Acrobat and PDFfilesstrike the needed balance between ease and low cost for the former and improved efficiency and effectiveness for the latter.


Capturing and Reporting Electronic Data - ACS Publications

Recommend Documents