Chapter 14
Metrology: A Tool and Approach to Ensure Data Quality 1
2
2
Gail E. Schneiders , John C. Brown , and Joseph Manalo 1
DuPont Crop Protection, Newark, DE 19714 DuPont Pharmaceutical Company, Route141and Henry Clay Road, Wilmington, DE 19880-0353
2
The computer has become a major tool in the scientific laboratory for the capture, manipulation, transfer and storage of data. As a result, the main focus on data quality has shifted from the instruments that generate the data to these electronic systems, often neglecting the fact that the data are only as accurate as the instrument measurements. Metrology, the science of measurement and calibration, can provide a disciplined approach for ensuring the accuracy of scientific measurement, and a metrology program should be included as part of an organization's scientific best practices. To achieve this goal a metrology program needs to consist of four main functions. These functions include: 1) an inventory tracking system; 2) a process for qualifying new instruments or re-qualifyingthem periodically or when there are changes; 3) a calibration and maintenance program to assure proper function and performance; and 4) a point of control for all critical documents needed to demonstrate that the equipment or systems have been qualified and calibrated, and perform in a way that ensures the quality of the data. The following outlines a framework and components for a metrology program. This program provides the necessary controls to
98
© 2002 American Chemical Society
99
promote a disciplined approach for ensuring measurement accuracy and precision, from the point of instrument installation, throughout its life-cycle, to meet both scientific and regulatory goals.
Introduction Metrology is defined as: "The Science of Measurement for the determination of conformance to technical requirements including the development of standards and systems for absolute and relative measurements." (1) In other words, it is the science used to demonstrate that an instrument performs at a specific level of accuracy and conforms to known standards. Data generated on this system should be reproducible and consistent. A program based on metrology principles can provide an organization with a measure of assurance that the data generated are true and accurate as measured. This means the instrument meets performance standards and contains proper documentation of equipment qualification, calibration and maintenance. Including computer system validation in the program provides a means to ensure the data integrity throughout its life-cycle. A good metrology program with outstanding documentation practices and controls can meet the compliance needs of current Good Manufacturing Practices (cGMP), Good Laboratory Practices (GLP) or International Organization of Standardization (ISO) Guide 25 standards. There are many reasons an organization may consider setting up a metrology program to ensure data quality. Thefirstquestion an organization should ask is who are the stakeholders and how do they use die data provided. For instance, the equipment or system user wants to generate high quality work, with reduced equipment system failures and elimination of the need to conduct repeat analyses. The business organization wants confidence in the data, reduced costs, increased capacity, consistent practices across the organization, reduced product recall, while meeting customer needs. The customer wants the assurance that the product meets their specifications. For many organizations there are regulatory requirements regarding data validity and compliance with documented procedures. Non-compliance with these requirements can result in regulatory actions against an organization. To satisfy the Environmental Protection Agency (EPA) (2) and Food and Drug Agency (FDA) good laboratory and manufacturing practices regulatory requirements, as well as international regulatory requirements, instruments need to be adequately tested, calibrated and/or standardized according to documented procedures. Current EPA FIFRA regulations state: "Equipment
100
used in the generation, measurement, or assessment of data and equipment used for facility environmental control shall be of appropriate design and capacity to function according to the protocol and shall be suitably located for operation, inspection, cleaning, and maintenance." (3) The EPA 1999 draft GLP regulations, which consolidate the 1989 FIFRA and TSCA GLPs (4)> are more specific on the requirements for ensuring electronic data integrity: "The integrity of data from computers, data processors, and automated laboratory procedures involved in the collection, generation, or measurement of data shall be ensured through appropriate validation processes, maintenance procedures, disaster recovery and security measures" (5)
Elements of a Metrology Program A metrology program, capable of satisfying its various stake holders, is composed of multiple elements: an accurate inventory and tracking system (database) containing information on individual components of instruments or systems; a process for qualifying instruments when purchased or when a component is upgraded, a calibration and maintenance program, and an effective record keeping system (Figure 1). The program should be defined in a metrology or qualification program description, containing standard operating procedures (SOPs) for each step of the process, and have personnel with appropriate training for their responsibilities.
Accurate Inventory
Instrument Qualification
Calibration & Maintenance
Figure 1 Metrology Program - Ensuring Data Quality
101
Inventory Management Key to ensuring data quality is being able to verify that the instruments and systems were performing accurately at the time of data generation. In order to do this, an organization needs to know which instruments and systems they own, as well as their historical and current state of calibration and maintenance. The simplest way to do this is to purchase metrology database software. For organizations with only a small number of instruments and systems, the database could be as simple as an Excel spreadsheet. These metrology or Excel databases should contain appropriate data fields, such as, system and individual component identification, manufacturer, model, serial number, description, location, and custodian. In addition, critical performance parameters, such as, range, resolution and user requirements, can help identify which equipment should be used for specific measurements. The status of the instrument and components (qualified - active, out of tolerance - locked out, retired) and calibration dates/schedule can also be tracked. A sample instrument information form is shown in Figure 2. A good metrology database will provide automatic reminders when the equipment is ready for re-calibration or maintenance. A person responsible for taking inventories and maintaining the database is necessary for an inventory management system to be effective. Additional benefits of an instrument inventory include the ability to shift workloads among instruments, facilitate scheduling of calibration and preventive maintenance, and leverage calibration and preventive maintenance contracts. Developing data on the historical performance of an instrument and components facilitates equipment renewal assessments to either purchase additional equipment because of workload demand or replace older equipment that is showing repeat failures or out-of-tolerance responses. Information on the version of software being used by the equipment facilitates routine upgrades and software validation of the equipment operating systems.
Qualifying Instruments The life cycle of an instrument or system consists of many stages (Figure 3). It begins with the identification of a need to generate specific data with defined performance parameters; then, identification of vendors whose products meet those performance specifications, followed by purchase, installation, qualification, daily use, maintenance, failures and repairs, and eventually retirement and replacement. Thefirststep in an instrument life-cycle is identifying an instrument need that is currently not being met. A detailed list of specific design parameters
102
System ID:
Component ID: Custodian to fill out: Description:
Purchase order number:
Manufacturer:
.Acquisition date:
Model: Serial number:.
Location:
Custodian:
Manuals:
,
Metrologist to fill out: Usage: CRITICAL REFERENCE NON-CRITIC
Type: Status:
ACTIVE
inactive
Status date:
retired
Logbook:
Department:
_
CRITICAL INSTRUMENT CALIBRATION INFORMATION: Range & Resolution:
—
User Requirements: Calibrator
Calibration SOP:
Calibration Points & Tolerances: Interval type: MONTH
END OF MONTH
. SPECIFIC MONTH
Basis for Date Calculation: CURRENT DUE DATE Comments:
,
N/A
DATE OF EVENT
NumberN/A
.
Figure 2. Instrument User Information Form.
103
Evaluate Design + Spec.
I Purchase
8 PQ/Sys. Suit
OQ/PV
Calibrate Operate
Install
Maintain
Fail IQ
Repair/Requalify or Retire
Figure 3. Instrument Qualification.
104
and specifications needs to be created. The purchase criteria or design specifications for the instrument should include the intended use of the instrument, critical parameters, specific ranges, disposition of raw data, and the user needs. These criteria are then compared to the vendor specifications to ensure that the proposed instrument has the functional range to meet these identified needs. Once the instrument has arrived, it should be set up and its performance qualified against both the manufacturers^ specifications and the purchase criteria. Draft revisions to the ISO 25 guideline state: "Equipment shall be capable of achieving the accuracy required and shall comply with specifications relevant to the teste and calibrations concerned. New equipment shall be checked against the purchase order to establish that it meets the laboratory's specification requirements, complies with the relevant standard specifications, and is calibrated and/or verified before use." (6) The Installation Qualification (IQ) demonstrates the equipment/system has been installed correctly at the user site according to vendor standards. The vendor should install the equipment to demonstrate to the buyer that all the components are operating appropriately. The qualification process includes appropriate documentation of the system components, physical installation and hook-up, and a performance check to verify the individual components operate and can communicate with each other. System component information, serial numbers, type of use and user performance requirements should be captured in the metrology database for easy tracking and scheduling of maintenance and/or calibration. Operation Qualification (OQ) involves verifying the system operates according to the specifications as agreed on between the vendor and purchaser. This should include a test of each critical component and function according to the vendor specifications and user requirements (if different) using specific standards. The Operation Qualification is usually done by the vendor; however, in-house or a qualified third party contractor may be used. The qualification can be conducted on each component of the system or holistically on the entire system. If each component is qualified, then the system has been operationally qualified and is ready for Performance Qualification (PQ). Conversely, if one component does not qualify, the system cannot be qualified! If the instrument contains computer software, then its ability to accurately capture, store, transfer and manipulate the data should be validated at this time. Detailed test scripts and a complete qualification documentation package needs to be generated. Many vendors now supply OQ documentation packages. However, in-house procedures need to be written to address custom specifications. If a component is replaced or upgraded, a new IQ and OQ needs to be conducted on this component. A complete operation qualification is usually not required for the system.
105 to be conducted on this component. A complete operation qualification is usually not required for the system. Performance Qualification and System Suitability (SS) demonstrate that the performance specifications of the system meet the user's expectations and needs for a given use. Performance Qualification is often called method validation. Method validation can be a general validation of commonly used parameters or a specific method validation. At a minimum, it should include expected performance and limit/failure testing. Performance Qualification of a method can include demonstration of precision, resolution, separation, recovery, and signal-to-noise ratios. It should be done before putting the instrument into routine use. Performance Qualification may be repeated many times during the life-cycle of the instrument as new methods with different performance criteria are used. Additionally, it should be conducted after routine calibration and maintenance, relocation, repair and component upgrades. The user or a qualified third party contractor can do this qualification. System suitability is performed daily by the user, usually prior to and throughout sample analysis, using specified standards and performance criteria. Again, the performance of the system is documented so that the accuracy of data generated can be verified. Table 1 summarizes the roles and responsibilities of the vendor and purchaser during instrument qualification.
Table 1. Instrument Qualification Roles and Responsibilities Activities
Design Specifications Installation Qualification Inventory Management Operation Qualification Performance Qualification and System Suitability Calibration/Maintenance
Accountability
User Vendor Metrologist, User In-house Vendor User In-house User
Options
Third Warty contractor Third Party contractor Third Party contractor
Calibration and Maintenance To facilitate the maintenance of equipment with different performance criteria, written procedures are needed. These procedures serve as a record of the process used to evaluate the systems' performance. The maintenance technician can sign the procedure record on completion of the task, verifying
106 the equipment meets the performance criteria. Equipment may need calibration as part of the maintenance procedure. Maintenance by appropriately trained personnel should be performed at regular intervals, before equipment parts fail. The technician could be any of a number of involved personnel: the person responsible for the metrology program, laboratory personnel who are familiar with the instrumentation, specifically trained maintenance or calibration personnel, third party contractors, or the original equipment manufacturer. Each choice has its benefits, and different scenarios may be used within the same company. If an organization is relatively large, it may make sense to have in-house personnel whose primary job is to maintain and calibrate the equipment. However, for smaller organizations or for specialized equipment, using outside experts may be more cost effective. Defined maintenance procedures should include model or manufacturer specifics and a list of parts to be inspected, cleaned, lubricated, replaced and/or calibrated. The replacement part numbers, cleaning solutions and lubricants, and calibration standards, along with the manufacture's maintenance procedures to be followed should be specified. Documentation is easily managed by creating a one-page checklist of instructions or performance parameters that can be checked off by the technician as each task is completed. Any issues or comments can be captured directly on the checklist. Provisions for failure or out-of-tolerance notification need to be clearly defined, as the equipment cannot be put back into service until the performance has been verified by conducting a performance qualification. As part of maintenance, some equipment may also need to be calibrated. Written calibration methods and a report format need to be available at the time of calibration. A calibration technician is responsible for performing and tracking calibrations, and writing calibration reports. The technician's responsibilities may also include handling and re-certification of standards. For quality calibration standards use NIST (National Institute of Standards and Technology) standards or other intrinsic standards of known purity, quality and stability. These standards should have certificates attesting to their performance properties. When an instrument is calibrated, the accuracy ratio of the instrument to the standard is ideally 10:1; however, a 4:1 ratio is acceptable. For example, if a thermometer is accurate to ± 1.0°C and the temperature standard used to calibrate it is accurate to ± 0.25°C, then the accuracy ratio is 4:1. If the accuracy ratio drops below 4:1, an uncertainty analysis is required. An uncertainty analysis (7) is the probability of false calibration decisions based on the accuracy of the standard relative to the instrument under test. After calibration, the metrologist or other responsible person should review the calibration reports to identify any issues with the equipment that may need further attention.
107
Documentation An organized document filing system must be maintained. This could be a paper file, an electronic document file or a mixture of both. The equipment inventory system already contains key information on the components of each system, their performance criteria and maintenance and calibration status. Additionally the qualification process will generate installation and performance documentation. Other documents necessary to demonstrate the quality of the data include standard operating procedures for the qualification procedures, calibration, maintenance, personnel training, etc. If the manufacturer's operating, service or maintenance manuals are used or cited in the operating procedures, copies of these manuals should also be maintained. To facilitate retrieval, documentation should be stored in a central location and be indexed for easy retrieval.
Regular Process Audits In addition to incorporating the preceding elements into any metrology program, periodic audits of the instrument qualification, calibration and maintenance practices should also occur. This is particularly important for systems generating data that are subject to review by regulatory agencies or certifying organizations. Audits should check the thoroughness and completeness of documentation and procedures, as well as adherence to the procedures. Documentation auditing will include qualification records, calibration records, maintenance records, and training records of those responsible for the organization's metrology program. Additionally, adherence to written procedures and a check against current regulatory standards applying to the organization should be done. These process audits provide a system of checks and balances to help ensure data quality. Two of the most frequent citations by auditing organizations are failure to have SOPs in place and, if in place, failure to follow them.
Metrology Program - Ensuring Data Quality An organization interested in ensuring data quality from the time of generation of the data through its life-cycle should seriously consider including a metrology program as part of their scientific best practices. Many elements of such a program may already exist within the organization, such as standard operating procedures, certified reference standards, quality assurance/quality control verification, and calibration and instrument maintenance. A formalized
108
metrology program will provide a point of control and standardization of processes that could significantly reduce the cost of generating true and accurate data and result in more satisfied customers.
References 1.Military-Standard(MIL-STD)-11309C PAR 3.1.369 2. The EPA GLP's currently exist in two separate regulations: at 40 CFR Part 160, applicable to Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA): and at 40 CFR Part 792, applicable to the Toxic Substances Control Act (TSCA). 3.
USEPA, Good Laboratory Practice Standards, Code of Federal
Regulations, 40 CFR Part 160, Section 160.61,54, 158, August 17, 1989, 34052. 4. USEPA, Toxic Substances Control Act: Good Laboratory Practice Standards, 40 CFR Part 792, Federal Register Vol. 54, No. 58, August 17, 1989. 5.
USEPA, Consolidation of Good Laboratory Practice Standards (40 CFR
Parts 160 and 792), Federal Register, 64, 249, December 29, 1999, 72972. 6. ISO/IEC Guide 25, General Requirements for the competence of testing and calibration laboratories, draft 4 section 3.4.2, international Organization of Standardization, 1996. 7. The uncertainty analysis formula is:[a /(a +b )]*100% 2
2
2
