Tools, Technologies and Training for Healthcare Laboratories

ISO 10012:2003 Measurement management systems

Dr. Paulo Pereira continues his discussion of ISO standards, taking a look at ISO 10012:2003 "measurement management systems - requirements for measurement processes and measuring equipment"

ISO SERIES UPDATE
Part 2 - ISO 10012:2003 “Measurement management systems - Requirements for measurement processes and measuring equipment” in medical laboratory

Paulo Pereira, Ph.D.
March 2017

Purpose

The discussion of metrological issues in a medical laboratory is extensive. Metrological specifications can be cross-referenced with those intended for medical laboratories, such as specified in ISO 15189 as “examination processes” (5.5 of [1]) and “ensuring the quality of examination results” (5.6 of [1]) This lesson is related to those devices wich are not classified as “analyzers,” usually identified as “metrological devices” or later as “monitoring and measuring resources” (7.1.5 of [2]). ISO 10012:2003 specifies general measuring equipment specifications and guidance for demonstrating the metrological verification and the management of measurement processes [3]. A measurement process can include a single device or a complex set of devices and an LIMS. The same principles applied to verify an instrument are applied to verify a set of instruments. The goal of ISO 10012 is not to show compliance with the requirements of ISO 9001 or ISO 15189, but to be used as support to ensure that metrological specifications are met. Other metrological standards are available to specific testing and calibration activities. For instance, the most well-known  guidance is from ISO/IEC 17025 [4], which contains general requirements for the competence of testing and calibration laboratories. However, other ISO guidance has a more limited scope, such the ISO 8655 series for piston-operated volumetric devices (including pipettes) [5-11]. ISO 10012 is not intended as a substitute for the requirements of ISO/IEC 17025. This international norm was last reviewed and confirmed by the Technical Committee ISO/TC 176/SC 3 “Supporting technologies” in 2015. Consequently, the 2003 version continues to be the most up-to-date version of the standard.

Approach

Usually, specifications associated with “monitoring and measuring equipment” are sometimes misunderstood as a “single equipment,” instead of a pipette, or a set of equipment - e.g., a laboratory auto-analyser. ISO 9001 introduces the terminology “monitoring and measuring resources,” which seems to be more exact than the previous definitions. Typically, the test report and calibration certificates of monitoring and measuring resources are issued by external providers. Note that ISO also refers to these resources as monitoring and measuring equipment. These providers have the laboratory techniques accredited to the ISO/IEC 17025 standard assuring the metrological traceability of the results determined in good metrology conditions. The calibration of devices is costly when the number of pieces of equipment is small, and many medical laboratories choose to contract for these services. In the case when externally provided services are purchased, the laboratory must  assure that metrological specifications are verified. For instance, the services must check if the measurement uncertainty (5.5.1.4 of [1],7.3 of [3]) and the bias (5.5.1.3 of [1], 7.1.1 of [3]) of the accreditation certificate of the providers are within the limit of error claimed by the medical laboratory. In this case, the medical laboratory provides skilled laboratorians to verify if the reported measurement uncertainty and bias are within the claimed limits of error.

Larger medical laboratories are expected to have a significant number of metrological devices. In this circumstance, having internal personnel perform the calibrations usually has less of an impact on the operating budget. This calibration is metrological traceable, but it does not need to comply the ISO/IEC 17025 requirements for testing and calibration methods. For instance, a report does not have to fulfill all the clauses of 5.10.2 specifications. However, ISO/IEC 17025 is suggested to be used as a support document for the metrological actions. On both scenarios, ISO 10012 is proposed to sustain the metrological activities for an easier and successful practice.

The contribution of ISO 10012 to meet the metrological specifications in ISO 9001:2015 and ISO 15189:2012

ISO 9001 and ISO 15189 specifications do not differ significantly. ISO 9001 requires in clause 7.1.5 that the laboratory “shall determine and provide the resources needed to ensure valid and reliable results when monitoring or measuring is used to verify the conformity of products and services to requirements.” The medical laboratory selects the suitable resources for monitoring and measurement procedures. This is not only required for what is typically referred as “monitoring and measuring equipment,” e.g., thermometers, balances, pipettes, and hygrometer but required for the more complex equipment/resources, such as auto-analyzers (which are out of the scope of this essay). These resources are sustained according to a maintenance program “to ensure their continuing fitness for their purpose.”

On the other hand, ISO 15189 clause 5.3.1 includes the equipment specifications. All ISO 9001 requirement are included, adding some other stipulations. This standard defines laboratory equipment as not only the hardware but also the software of instruments and measuring systems, such as the laboratory information management systems (LIMS).

The ISO customer focus is fulfilled by ensuring the laboratory metrological specifications are in compliance with client specifications to both standards (5.2 of [3]). Accordingly, the limit of error, also recognized as maximum permissible measurement error or maximum permissible, should be defined by the medical laboratory according to the intended use of the measurements. For instance, to verify the limit of error in balance, the limit should be defined by the user. However, the laboratorian may not have the skills to enable this definition. In practice, this limit is defined by who is performing the metrological function (5.1 of [3]). Note that this error limit specification should be based on independent literature. For instance, global standards or manufacturer insert. The use of the manufacturer limits is acceptable in most situations. If the medical laboratory rejects the manufacturer measurement uncertainty and bias as unacceptable, this equipment should not be purchased!

The job functions (6.1 of [3]) related to the measurement system are documented in both ISO standards. The competence and training requirements guarantee that all the staff is successfully trained, i.e., they demonstrate their ability to successfully perform the metrological tasks. The equipment is only to be operated by laboratorians with successful training, including at the time of equipment acceptance testing. In this phase, the medical lab verifies if the equipment achieves the specifications according to its intended use. In ISO 15189 clause 5.1.6 the competence assessment of the laboratorian also assures the “direct observation of equipment maintenance and function checks.” The metrological function in a medical laboratory is suggested to be performed by an experienced laboratorian in a small laboratory or by a skilled team in a larger lab. The goal is to establish, document and maintain the measurement management system and constantly improve its usefulness.

ISO 15189 requires that the metrological procedures are documented (paper or electronically) to assure the harmonization of practices and guides to training (5.3.1.4 of [1]). All documents (including reports) must be controlled, as dictated by ISO 9001 (7.1.5 of [2]). In contrast to ISO 9001, however, ISO 15189 requires a “documented procedure for the selection, purchasing, and management of equipment.” Also, the documented practice is required for the calibration of equipment affecting the reported results - including the metrological traceability of the standard. There is another specification beyond what is stated in the general ISO 9001 standard: a mandatory “documented procedure for the selection, purchasing and management of equipment”. The documented practice is required for the calibration of equipment affecting the reported results on both standards - including the metrological traceability of the standard.

ISO 15189 clause 5.3.1.5 dictates a set of equipment maintenance and repair requirements, close to what is required in the quality management system norm. A rule of thumb is that after any device maintenance or repair, calibration activities must precede before any patient testing. After one of the actions, the previous reports are obsolete. Testing and calibrations are used to verify the conformity of the claimed limit of error post-maintenance or repair. Note that this verification does not always happen via calibrations. For instance, to check the success of a pipette maintenance, the ISO 8655 approach is used by the medical laboratory, since it verifies the measurement uncertainty and bias using a metrological limit of error. The verification should use the internal quality control results of the test or tests where it is used. If there are no out-of-control results, it is interpreted that the error is allowable, independently the contribution of the pipette.

Measurement traceability [12-15] is only required if it is a specification of the medical laboratory. So, it is not mandatory. However, when required to a specified resource, it is calibrated and verified according to a timetable using primarily measurement standards traceable to international or national measurement criteria (7.3 of [3]). See Figure 1 for an example of a metrological scheme. This figure shows the traceability of measurement uncertainty and bias for a set of materials with different levels of uncertainty and accuracy. The availability and the stability of the materials progress inversely to the uncertainty, inaccuracy, and cost. When standards are unavailable, the lab should use an alternative approach (see the reference material for more documentation of these approaches).

ISO10012 Figure 1 Measurement Uncertainties

All the conditions impacting the measurement equipment must be controlled, such as the environment (6.3 of [3]). Equipment that does not fulfill the specification cannot be used and should be identified accordingly. The environmental conditions are verified, such as the room temperature and humidity. Therefore, the resources are identified according to their metrological status. For instance, accepted/rejected, in service/out-of-service. Note that when the calibration measurement error and measurement uncertainty fall within the vendor claims, it assume that good laboratory practices are being followed, such as transport and laboratory room conditions, and technical skills of the laboratorian. Accordingly, the resource is “safeguarded from adjustments, damage or deterioration that would invalidate the calibration status and subsequent measurement results.” The measurand - e.g., the temperature of a freezer in suitable room temperature condition - is monitored by the medical lab to verify in routine practice that the resources are performing according to their intended use.

ISO10012 Figure 2 different error scenarios

Figure 2 illustrates the major operating scenarios that could occur. The green Gaussian curve is unbiased with a low measurement uncertainty interval. This is the best case. The yellow curve remains unbiased, but the measurement uncertainty is high. This case requires a repair (if possible and the cost is acceptable) since the correction is not feasible. The blue curve has a low measurement uncertainty but is biased. In this scenario, the results on this point should be corrected, and the outcome will be similar to the green curve. This is probably the typical case in medical laboratories. The red curve represents the worst example - high measurement uncertainty and a significant bias. In this condition the correction of the results outcome is equivalent to the yellow curve, still requiring repair. Figure 3 show a simplified scheme of the metrological confirmation methodology for a device.

ISO10012 Figure 3 an example flowchart

In the metrological confirmation, i.e., verification of the reported results, it is defined what measurements are required, such as the claimed limit of error (7.1 of [3]). These determinations are measurement uncertainty and bias or derived computations such as the range, stability, hysteresis, and drift. Others measurements could include the effects of influencing quantities, discrimination (threshold), and dead band. The intervals between metrological confirmation should be determined primarily in synchronization with the maintenance intervals. These periods are defined preferably considering the use per equipment, and secondary systematic, i.e., independently the use per device. For instance, it is recognized that the same pipette used 20 times per day or 20 times per year should not have the same period of maintenance. However, maintenance or other verification activities on these devices are usually scheduled during the same period. The impact of the utilization frequency on the reliability of the dispensing/reported results should be properly considered.

The metrological verification could be computed using different mathematical models. The following simple models are suggested:

a) Testing reports
High limit of error ≥ maximum result + measurement uncertainty
Low limit of error ≤ minimum result - measurement uncertainty
Note: It is suggested that the laboratory use the expanded measurement uncertainty (worst-case scenario). The measurement uncertainty and bias is on the weakest measured point of the testing report for the two limits of error. For instance, the weakest points in a testing report of a freezer. If one weakest point is rejected, the freezer should not be used, in this example to storage. The next weakest point is verified.
b) Calibration certificates
|Limit of error| ≥ measurement uncertainty + |bias|
Note: according to some limits of error it could be used the standard measurement uncertainty or the expanded measurement uncertainty.

In a future essay, I will present and discuss a set of metrological confirmations for medical laboratory devices.

When a set of metrological instruments is verified , it is suggested that the limit of error should be related to the process outcome. If the limit is nonconforming, a modeling approach should be considered to identify which instrument(s) is (are) contributing to the unacceptability of the measurement.

Outside suppliers should be evaluated and selected based on their ability to meet the documented criteria (6.4 of [3]). Testing and calibration methods accreditated by ISO/IEC 17025 is understood as a demonstration of good metrological practices, for what generally it is required on the agreements.

The medical laboratory defines the quality objectives associating key performance indicators (KPIs) to control the measurement system (5.3 of [3]). The goals are usually defined in a timetable, to able monitoring over time. For instance, all metrological verifications are to be completed as scheduled. The medical laboratory management should reviews the measurement system systematically through the control of the KPI to assure its continuous adequacy, effectiveness, and suitability (5.4 of [3]). The laboratory management should also assure the availability of the resources to permit the successful operation of the system.

When the operation is out-of-compliance, a correction (if applicable) and a corrective action/preventive action (CAPA) report is started (8.3 of [3]). The human samples, reagents, etc. results determined in out-of-compliance condition are reviewed. Auditing and monitoring practice (8.2 of [3]) is implemented to the control of nonconformities and to assure the improvement of the metrological system.

Summary

The pros of ISO 10012:2003 can be summarized as:

  • Focused on the measurement process
  • Focused on measurement equipment
  • Useful support for (part of ) the metrological specifications

Nevertheless, there are a few cons to ISO 10012:

  • Sometimes it is redundant to ISO 9001 and ISO 15189
  • It does not suggest mathematical models on the metrological confirmation
  • It does not suggest other ISO standards on specific testing and calibration
  • Some terminology is outdated

References

  1. International Organization for Standardization (2012). ISO 15189 Medical laboratories - Requirements for quality and competence. 3rd ed. Geneva: The Organization.
  2. International Organization for Standardization (2015). ISO 9001 Quality management systems - Requirements. 5th ed. Geneva: The Organization.
  3. International Organization for Standardization (2003). ISO 10012 Measurement management systems - Requirements for measurement processes and measuring equipment. Geneva: The Organization.
  4. International Organization for Standardization (2005). ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories. 2nd ed. Geneva: The Organization.
  5. International Organization for Standardization (2002). ISO 8655-1 Piston-operated volumetric apparatus - Part 1: Terminology, general requirements and user recommendations. Geneva: The Organization.
  6. International Organization for Standardization (2002). ISO 8655-1 Piston-operated volumetric apparatus - Part 2: Piston pipettes. Geneva: The Organization.
  7. International Organization for Standardization (2002). ISO 8655-1 Piston-operated volumetric apparatus - Part 3: Piston burettes. Geneva: The Organization.
  8. International Organization for Standardization (2002). ISO 8655-1 Piston-operated volumetric apparatus - Part 4: Dilutors. Geneva: The Organization.
  9. International Organization for Standardization (2002). ISO 8655-1 Piston-operated volumetric apparatus - Part 5: Dispensers. Geneva: The Organization.
  10. International Organization for Standardization (2002). ISO 8655-1 Piston-operated volumetric apparatus - Part 6: Gravimetric methods for the determination of measurement error. Geneva: The Organization.
  11. International Organization for Standardization (2005). ISO 8655-1 Piston-operated volumetric apparatus - Part 7: Non-gravimetric methods for the assessment of equipment performance. Geneva: The Organization.
  12. EURACHEM/CITAC. (2003). Traceability in chemical measurement. Europe: The Organizations. Retrieved from: http://www.eurachem.org/images/stories/Guides/pdf/EC_Trace_2003.pdf.
    Accessed: March 31, 2017.
  13. Clinical and Laboratory Standards Institute (2006). X-05R Metrological traceability and its implementation, A report. Wayne (PA): The Institute.
  14. Vesper H, Thienpont L (2009). Traceability in laboratory medicine. Clin Chem 55(6):1067-1075.
  15. Pereira P (2016). Uncertainty of measurement in medical laboratories. In:Cocco L (Editor). New trends and developments in metrology. Rijeka: InTech. Retrieved from: http://www.intechopen.com/download/pdf/50379. Accessed: March 31, 2017.