Auditing Electronically Captured Analytical Chemistry Data - American

Chapter 12

Downloaded by NORTH CAROLINA STATE UNIV on January 2, 2018 | http://pubs.acs.org Publication Date: August 1, 2002 | doi: 10.1021/bk-2002-0824.ch012

Auditing Electronically Captured Analytical Chemistry Data Mary E. Lynn th

The Windward Company, 4821105 Avenue,N.W., Gig Harbor, WA 98335

Auditing data captured electronically has produced new challenges for Quality Assurance (QA) personnel. In general, auditing electronically captured data during studies conducted in compliance with Good Laboratory Practices (GLP) should be approached no differently than performing any other data audit. However, there are additional considerations when auditing data generated electronically. This chapter discusses these additional considerations for QA personnel. Additionally, suggestions are provided for the QA auditor when conducting in-process inspections, data/report audits, and facility inspections and monitoring records retention.

Auditing data captured electronically has produced new challenges for Quality Assurance (QA) personnel. In general, auditing electronically captured data during studies conducted in compliance with Good Laboratory Practices (GLP) should be approached no differently than performing any other data audit. However, there are additional considerations when auditing data generated electronically. The current Environmental Protection Agency (EPA) FIFRA GLPs (/) discuss data captured electronically in the context of data recording. Additionally, computers and other electronic equipment that is used for the "generation, measurement or assessment of data" should be considered

© 2002 American Chemical Society

Garner et al.; Capturing and Reporting Electronic Data ACS Symposium Series; American Chemical Society: Washington, DC, 2002.

79


80 "equipment". The proposed EPA GLPs (2) also provide additional language for assuring the integrity of data from computers and other automated laboratory systems. The proposed regulation indicates that "the integrity of data from computers, data processors, and automated laboratory procedures involved in the collection, generation or measurement of date shall be ensured through appropriate validation processes, maintenance procedures, disaster recovery, and security measures." There are a number of areas to consider when auditing data generated electronically including system validation, security, data audit trail, and retention of records. Documents are available that provide suggestions and guidance in this area and with these concepts. The following documents can be useful for QA and technical personnel working in this area. • •

• •

EPA's Good Automated Laboratory Practices (GALP) with implementation guidance (3) Organisation for Economic Cooperation and Development (OECD) Consensus Document on the Application of the GLP's to Computerised Systems (4) Computerized Data Systems for Nonclinical Safety Assessment: Current Concepts and Quality Assurance (5) Food and Drug Administration (FDA) 21 CFR, Part 11; Electronic Records/Electronic Signature Rule with preamble (6)

The documents listed above are just a few of those available and come from several different sources, not all EPA related. It is interesting to note that many of the available guidance documents and resources for validation and use of computerized systems in a GLP environment are comparable in the information they provide. Therefore, this chapter does not limit its use of references to EPA related materials.

Quality Assurance Considerations QA personnel required to monitor the GLP compliance of computerized systems should be familiar with and/or receive training on each system that is utilized in electronic data capture for the studies they are to audit. This electronic data capture training could include hands-on training, attending an outside training course and reading available documents about the system (e.g., the validation report, applicable Standard Operating Procedures (SOPs), system user manuals). Additionally, it is desirable for the auditor to have training in the regulatory areas related to computer systems. In some companies, there are



81 specific QA personnel with specialized training within the Quality Assurance Unit (QAU) assigned to audit computerized systems. In smaller companies with limited resources, it may not be practical for this specialization. QA personnel are required to inspect/audit each study conducted in compliance with GLPs, but the extent to which QA personnel are involved in software development and the validation/verification process varies from company to company. In some companies, there is little or no QA involvement in these processes, while in others QA personnel are involved by performing inspections and audits just as they would during a GLP study. It is generally a more successful process if QA personnel are involved, QA personnel can provide assistance in the area of vendor audits for purchased software or can conduct inspections of in-house software development to assure internal procedures are being followed. QA personnel can also provide inspection support during the validation and verification process by conducting in-process inspections and reviewing the resulting data and validation report for accuracy. During system development and validation, QA personnel may also be utilized for regulatory advice to assure the system will meet government standards. Being involved early in the validation process QA personnel become more familiar with the system(s) that will be used. The QAU should have written procedures (i.e., SOPs) for the conduct of inspections and audits. These procedures should incorporate any considerations for the QA review of electronic data systems. For example, the QAU SOPs should address the role and responsibilities of the QAU in software development, purchase, and validation activities; QAU in-process audit procedures for data collected on-line; the procedure for on-line review of data (e.g., what will be verified, how much data will be reviewed); and the procedure for report audits using on-line data.

In-process Inspections During the conduct of chemistry analysis in-process inspections, the data collection practices, including the data capture system, should be inspected in addition to observing the procedure being performed. The QA auditor should review the protocol and applicable SOPs for data collection practices as well as the procedure prior to conducting the in-process inspection. The auditor can observe several items related to data collection practices during in-process inspections. The auditor should observe if the protocol and applicable SOPs are being followed with respect to the procedure and data collection practices; if the appropriate security procedures are being utilized; and if changes are noted appropriately (e.g., the original entry is available, the



82 reasonforthe change is documented, etc.). Additional items to look for include computers left unattended without the user logging out, user name and password posted in the laboratory, and group use of user name and password. Additional items that can be reviewed during the in-process inspection include personnel training records and equipment records. The personnel training records should be checked to verify not only that the individual(s) observed during the inspection has the appropriate training in the technique, but also to verify that training on the computer system has been documented. Equipment records should be reviewed for the equipment used during the in-process inspection. These records generally include documentation of maintenance and, if applicable, calibration. Each computerized data acquisition system should have records documenting maintenance. Maintenance for a computer system should include hardware as well as software. If a controlled change was made (e.g., software upgrade), the records should indicate what was changed, why it was changed, who made the change, when it was made, and was the system revalidated after the change. An authorization procedure should be in place for approving controlled changes and this authorization should be documented. Computer system maintenance records should also record unanticipated events, what corrective action was taken, who performed the corrective action and when. These events might include system crashes, date/time changes after a power failure, etc.

Data/Report Audits QA's responsibility for the report is to "review the final study report to assure that such report accurately describes the methods and standard operating procedures, and that the reported results accurately reflect the raw data of the study."(i) To adequately perform a data/report audit, QA personnel need direct access to the on-line data. Access for QA personnel should be in the form of read-only access. The amount of data audited and how the data points are chosen for audit should be specified in the QAU SOPs. An auditor may choose to perform more thorough and more frequent audits on a recently validated system. The validation report can be used to assist in determining what and how much to audit. For example, if data summary printouts from the chromatographic computer system are used in the report, it is important to review the validation report to verify that this function was tested during validation. If this portion of the computer software was successfully validated, it may be sufficient to verify a few values on each table in the report.



83 QA personnel should also review data on-line. Data changes should be reviewed to ensure the audit trail was appropriately maintained (e.g., the original and changed data are both available, the date of the change was recorded, the reason for the change was recorded, and the person responsible for the change was identified). No data should be overwritten. For example, reintegrated or recalculated data should not overwrite the original data. The on-line review could also include items such as tracking several samples through the system and a check of the calibration and integration parameters (meta data). If data were reintegrated, all integration parameters should be saved. If spreadsheets or statistical packages are used, these should be included in the audit. Items such as input values and equations or routines used should be verified during the data audit. This type of software should also be included in the validation program to assure it is providing the correct output values.

Records Retention QA's responsibility during GLP studies should not be limited to in-process inspections and data and report audits. The QAU should also review the procedures for storing and archiving electronic data. This review should include assuring back-up and archiving procedures were performed as specified in the SOPs; reviewing the records for any instances of equipment failure and assuring contingency plans were followed; and reviewing to assure long-term storage procedures are followed. Archived electronic data should be treated no differently than archived paper data. Data should be archived at the completion of the study, an archivist should be assigned, access should be limited to authorized personnel, and the material should be indexed to permit expedient retrieval. Depending on the media used for storage, it may be necessary to provide an area within the facility with specific environmental controls needed to maintain the integrity of electronic data. This should be specified in the data storage SOPs.

Facility Inspections Computerized systems should be included during facility inspections of analytical laboratories. Items discussed previously, such as, computer maintenance records and personnel training records, can be reviewed more thoroughly during the facility inspection.



84 In addition to the personnel training records for laboratory technical staff, training records should be reviewed for staff involved in the software development, validation and computer maintenance. These personnel may reside in a separate computer functional group or may be part of the analytical laboratory group. In either case, their training records should contain documentation of education, experience and training to support the duties they perform. Depending on the function performed, this training should include training in GLPs and other regulations or guidance as applicable. The facility inspection should include a review to assure applicable computer systems and software have been validated or tested as needed. There may be differences between how network systems are validated in contrast to stand-alone systems. The records and procedures for the different systems should be reviewed as part of the facility inspection. If validation reports have not previously been reviewed by the QAU, the facility inspection may provide an opportunity to review these reports and data. As discussed earlier, each computer system used to capture data during GLP studies should have records of maintenance. The records should be reviewed to assure they are compliant with the applicable SOPs and GLPs. Consistency of record keeping between systems should also be reviewed. Items such as system to system variation in time/date settings, passwords being changed as required, completeness of documentation of software and/or hardware upgrades for each system, and availability of maintenance records, user manuals and other system documentation in the laboratory can be evaluated during the facility inspection.

References (1) Environmental Protection Agency. FIFRA Good Laboratory Practice Standards; Final Rule. 40 CFR Part 160. 1989; Federal Register Vol. 54, No. 158. (2) Environment Protection Agency. Consolidation of Good Laboratory Practice Standards; Proposed Rule. 40 CFR Parts 160, 792, 806. 1999; Federal Register Vol. 64, No. 249. (3) Environmental Protection Agency. Good Automated Laboratory Practices: Principles and Guidance to Regulations for Ensuring Data Integrity in Automated Laboratory Operations with Implementation Guidance. Office of Information Resources Management. 1995. (4) Organisation for Economic Cooperation and Development (OECD). The Application of the Principles of GLP to Computerised Systems. 1995;


85


Environment Monograph No. 116. GLP Consensus Document, Number 10. (5) Drug Information Association. Computerized Data Systems for Nonclinical Safety Assessment: Current Concepts and Quality Assurance. 1998. Drug Information Association. (6) Food and Drug Administration. Electronic Records; Electronic Signatures; Final Rule. 21 CFR Part 11. 1997; Federal Register Vol. 62, No. 54.


Auditing Electronically Captured Analytical Chemistry Data - American

Recommend Documents