Tools, Technologies and Training for Healthcare Laboratories

Case studies of metrological verification of measuring equipment

Dr. Paulo Pereira demonstrates 6 case studies of metrological verification of measuring equipment in the medical laboratory, following the ISO 10012:2003 standard.

ISO 10012:2003
Part 4: Case studies in the Metrological verification of measuring equipment in the medical laboratory

Paulo Pereira, PhD
JULY 2017

ISO SERIES UPDATE
Part 4 - Metrological verification of measuring equipment in medical laboratory based on ISO 10012:2003: Case studies

Paulo Pereira, Ph.D.
June 2017

Purpose

This essay follows the ISO SERIES UPDATE Part 2 [1] featuring a brief discussion including pros and cons of the ISO 10012:2003 [2] as a support to fulfill the requirement for measurement processes and to measure equipment in the med lab. The control of measuring equipment is a good laboratory practice required by the technical lab standards. This text presents some examples of metrological verification discussing the benefits and limitations. A rule of thumb: the metrological verification based on ISO/IEC 17025 calibration certificates and testing reports (5.10 of [3]) is required uniquely for measuring equipment with significant impact on the trueness of the reported results in the medical laboratory when the result accuracy cannot be controlled by validation or internal quality control scheme. When this rule is not fulfilled, the metrological verification may be interpreted as the absence of good laboratory practice. The cases are based on data taken from calibration certificates and test reports according to the ISO/IEC standard. Therefore, the required data on the metrological verification is accessible.

Case 1: Metrological verification of a weight
Intended use: control of the weighing of human whole blood collection for transfusion.

Requirements: For instance, the European Directive [4] requires that each whole blood must be in the following interval: 450 +/- 50 mL (400 to 500 mL or 424 to 530 g). Therefore, the weight to be used for the control of the whole blood collection should be close to 477 g [(424 g - 530 g)/2]. Weights with a mass within the limit but close to the limit are not suggested.

Step 1: Computation of error and combined standard deviation
Source: Calibration Certificate
Error = average of measurement – reference quantity value
On this case, the weight has 450 g. The reference value is the average of measurements by the ISO/IEC 17025 accreditated supplier (measured on a standard balance), which is equal to 450 g. So, the error is 0 g.
Combined standard deviation u = expanded standard deviation U / constant k for an approximate level of confidence of 95%.
U is equal to 0.15 g for a k of 2. Accordingly, u is 0.075.

Step 2: Metrological verification
The second step could be more complex. Formulation of the question: what is the maximum permissible measurement error for a measuring device? An error that does not significantly affect the accuracy of the weight. “Significantly affect” is interpreted as an error which would cause a measurement to be nonconforming. If the weight mass was 477 g, the limit should be +/- 53 g. Like it the mass is 450 g, assume the limit as the worst condition, i.e., the difference between 450 g and 424 g = 26 g. The verification is determined as follows:
measurement uncertainty + |bias| ≤ |limit of error| <=> 0.075 g + 0 g ≤ 26 g <=> 0.075 g ≤ 26 g. Therefore, the sum of measurement uncertainty and bias is not significant for what the weight is allowable for its intended use.
If the weight should be used to weigh reagents the limit of error could be the one for its class (consult the appropriate literature). This error is assumed to be not significant to the accuracy of the weighings.

Case 2: Metrological verification of a thermometer

Intended use: control of the temperature (ºC) in a certain interval. On this example, the thermometer is intended to be used at temperatures between 0 and 40 ºC.

Requirements: The limit of error associated with the device is typically determined by the manufacturer. It is assumed to be not significant for the intended use: on this example, the thermometer is to be used to control the temperature of test reagents from 2 to 8 ºC. The limit of error for measurements within the interval from 0 to 40 ºC is +/- 0.75 ºC.

Step 1: Computation of error and combined standard deviation
Source: Calibration Certificate
At least two calibration points are verified. Let consider the minor (0 ºC) and major results (40 ºC). If these points are within the limit of error, the other points are presumed to be also.
For the point 0 ºC the average of the supplier standard thermometer is 0 ºC and average on the lab thermometer average is 0 ºC. So, the error is 0. For the point 40 ºC the standard thermometer average is 40 ºC and the lab thermometer average is 40.1 ºC. So, the error is +0.1 ºC.
U is equal to 0.24 ºC for a k of 2. Accordingly, u is 0.12 ºC.

Step 2: Metrological verification
The verification is determined as follows:
Point 0 ºC: measurement uncertainty + |bias| ≤ |limit of error| <=> 0.24 ºC + 0 ºC ≤ 0.75 ºC <=> 0.24 ºC g ≤ 0.75 º
Point 40 ºC: measurement uncertainty + |bias| ≤ |limit of error| <=> 0.24 ºC + 0.1 ºC ≤ 0.75 ºC <=> 0.25 ºC g ≤ 0.75 º
Therefore, the sum of measurement uncertainty and bias is not significant for what the is allowable for the thermomether‘s intended use.
The average of both points could also be used to determine the regression equation to be used for the correction of the lab thermometer results (y) to the standard thermometer results (x). On this condition, the results of two calibration points should be recomputed for a more accurate evaluation.

Case 3: Metrological verification of a watch

Intended use: control of the time (minutes) in a centrifuge. The purpose is to obtain serum or plasma for virological screening.

Requirements: The limit of error is typically unknown.

Step 1: Computation of error and combined standard deviation
Source: Calibration Certificate
On this case, the reference value is the selected centrifugation time: 15 minutes. The average of measured time on five determinations is 15 minutes 1 second. So, the error is +1 second.
U is equal to 2.65 seconds for a k of 2.65. Accordingly, u is 1 second.

Step 2: Metrological verification
The limit of error should be provided by the reagent manufacturer or in an internal standard operating procedure (SOP). However, it is usually not defined. So, it is suggested to determine this based on experimental data. For instance, for a time of 15 minutes, it could be tested at 15 minutes 0 seconds, 15 minutes 15 seconds and 14 minutes 45 seconds (replicate such as five measurements per point). Use a t-test to verify determine if two sets of results for a certain lab test (for instance, a screening immunoassay) are significantly different from each other. If the t-test p-value is below than the threshold chosen for statistical significance (for instance, 0.05), then the null hypothesis is rejected in favor of the alternative hypothesis - the error in centrifuging time is presumed to not significant for the interval of 15 minutes +/- 15 seconds could be assumed as the limit of error. The verification is determined as follows:
measurement uncertainty + |bias| ≤ |limit of error| <=> 1 second + 1 seconds ≤ 15 seconds <=> 2 seconds ≤ 15 seconds. Therefore, the sum of measurement uncertainty and bias is not significant for what the watch is allowable for its intended use.

Case 4: Metrological verification of a pipette

Intended use: Accurate dispensing of a fixed or variable volume. In this example, it is the use of a fixed volume pipette for 100 mL.

Requirements: The limit of error is typically based on metrological standards.

Step 1: Computation of error and combined standard deviation
Source: Calibration Certificate
In this case, the reference value is the selected dispensing point: 100 mL. The average value is 99.9 mL. So, the error is -0.1 mL.
U is equal to 0.25 mL for a k of 2. Accordingly, u is 0.125 mL.

Step 2: Metrological verification
The limit of error is usually based on metrological standards provided in the manufacturer’s literature. However, the limit of error should be based on the error contribution to the measurement uncertainty or total analytical error of the reported results. This could be performed using the internal quality control than the calibration of pipettes. It is simpler, less expensive and more appropriate. For instance, testing a pipette in replicate conditions (for example, five dispensations), and verify if the results are in control or out-of-control. If they are in control, the error arising from the pipette is acceptable. Anyway, if the lab must calibrate a pipette - for instance because it is required in a certain standard - it is suggested to be determined based on experimental data for variable volume pipettes. For instance, for a dispensation point of 100 mL, it could be dispensed a human sample on 100, 95 and 105 mL (replicate such as five measurements per point). Use a t-test to verify if there is a statistically significant difference between the averages results for a certain lab test. If the t-test p-value is below than the threshold chosen for statistical significance, the error is presumed as no significant for what the interval 100 +/- 5 mL could be assumed as a limit of error. The verification is determined as follows:
measurement uncertainty + |bias| ≤ |limit of error| <=> 0.125 mL + 0.1 mL ≤ 5 mL <=> 24.38 rpm ≤ 100 rpm. Therefore, the sum of measurement uncertainty with bias is not significant for what the pipette is allowable for its intended use.

Case 5: Metrological verification of a tachometer

Intended use: control of the rotations per minute (rpm) in a centrifuge. The purpose is to obtain serum or plasma for virological screening.

Requirements: The limit of error is typically unknown.

Step 1: Computation of error and combined standard deviation
Source: Calibration Certificate
On this case, the reference value is the selected centrifugation point: 1000 rpm. The maximum value of one minute in stable centrifugation conditions is 1020.4 rpm. So, the error is +20.4 rpm.
U is equal to 8.48 rpm for a k of 2.13. Accordingly, u is 3.98.

Step 2: Metrological verification
The limit of error should be in the literature provided by the reagent manufacturer or in an internal SOP. However, it is usually not referred. So, it is suggested to be determined based on experimental data. For instance, for a centrifugation point of 1000 rpm, it could be centrifuged a human sample at 1000 rpm, 900 and 1100 rpm (replicate such as five measurements per point). Use a t-test to verify determine if two sets of data for a certain lab test are significantly different from each other. If the t-test p-value is below than the threshold chosen for statistical significance it is supposed the error is no significant for what the interval 1000 +/- 100 rpm is assumed as a limit of error. The verification is determined as follows:
measurement uncertainty + |bias| ≤ |limit of error| <=> 3.98 rpm + 20.4 rpm ≤ 100 rpm <=> 24.38 rpm ≤ 100 rpm. Therefore, the sum of measurement uncertainty and bias is not significant for what the tachometer is allowable for its intended use.

Case 6: Metrological verification of a freezer

Intended use: Storage of reagents

Requirements: 2 to 8 ºC

Step 1: Computation of error and combined standard deviation
Source: Testing report
This case differs from the calibration since the evaluation is not focused on a standard measurement but the homogeneity of the temperature in different areas of the freezer. So, what the laboratorian wish to evaluate is if all areas have a temperature between 2 and 8 ºC. The minimum and maximum temperatures are taken from the averages per point of the area. 12 areas are tested/12 testing points. On this case, the minimum is 3.4 ºC (testing point no. 3), and the maximum temperature is 5.9 ºC (testing point no. point 9). However, this information is not sufficient to evaluate if these points (such as all the points between) are fulfilling the claimed interval since measurement uncertainty as a critical roles as it will be seen on the next step.
U is equal to 1.7 ºC for a k of 2. Accordingly, u is 0.85 ºC.

Step 2: Metrological verification
Let consider the measurement uncertainty distribution as quadratic in a worst scenarium. Differently than the Gaussin distribution, on a quadratic the results within the measurement uncertainty interval have the same chance to occur. Consequently, the verification is determined as follows:
Low limit of error ≤ minimum result - measurement uncertainty <=> 3.4 ºC - 0.85 ºC <=> 4.25 ºC ≥ 2 ºC
High limit of error ≥ maximum result + measurement uncertainty <=> 5.9 + 0.85 ºC <=> 6.75 ºC ≤ 8 ºC
Therefore, the freezer areas are accepted for its intended use. If one area is rejected, it must be verified the next weakest point. The rejected area cannot be used to storage and a maintance action should be taken.

References

  1. Pereira P (2017). ISO SERIES UPDATE Part 3 - ISO 10012:2003 “Measurement management systems - Requirements for measurement processes and measuring equipment” in medical laboratory. In: Westgard QC. Retrieved from: https://www.westgard.com/iso-10012-2003.htm. Accessed: March 31, 2017
  2. International Organization for Standardization (2003). ISO 10012 Measurement management systems - Requirements for measurement processes and measuring equipment. Geneva: The Organization.
  3. International Organization for Standardization (2005). ISO/IEC 17025 General requirements for the competence of testing and calibration laboratories. 2nd ed. Geneva: The Organization.
  4. Commission Directive 2004/33/EC Implementing Directive 2002/98/EC of the European Parliament and of the Council as regards certain technical requirements for blood and blood components. Official Journal of the European Union L91/25-39.