ISO

Funny thing about measurement uncertainty - it's not always clear how to calculate it. Particularly when clinically significant bias exists.

Uncertainty in How to Calculate Measurement Uncertainty: Different approaches for incorporating effects of clinically significant bias

James O. Westgard, PhD
April 2021

According to ISO 15189 [1], section 5.5.1.4, “the laboratory shall determine measurement uncertainty for each measurement procedure in the examination phases used to report measured quantity values on patients’ samples.” Although this requirement has been in place for many years now, there are continuing arguments about how to calculate measurement uncertainty. A new ISO document 20914:2019 [2] specifically addresses the issue, but there still is discussion in the literature about how to properly calculate measurement uncertainty [3-4], particularly how to incorporate the effects of uncorrected clinically significant bias.

Originally, the debate was about proper application of the bottom-up methodology recommended by GUM – Guide to the expression of uncertainty in measurement [5]. The bottom-up approach depended on identifying individual components of variation, estimating their size, then summing the variances and extracting the overall standard deviation, or standard uncertainty. After many attempts at implementation of this methodology, it was concluded that the bottom-up approach was too complicated and impractical for medical laboratories. The alternative was to employ a top-down methodology that made use of available data on measurement precision, specifically, internal quality control data obtained over a period of a few months, commonly referred to as intermediate precision data. By 2012 when the CLSI published guidance on “Expression of Measurement Uncertainty in Laboratory Medicine”, both bottom-up and top-down methodologies were included [6].

In all these discussions and approaches, the handling of bias was and still is problematic, with the ideal remedy being the elimination of bias, a secondary remedy being the correction for bias, and often a final remedy of ignoring bias. Today, even after publication of ISO 20914, there is uncertainty in how to calculate measurement uncertainty. Individual countries are publishing their own guidelines for calculating MU as part of their particular accreditation requirements, but there still is variation in the calculation procedures.

ISO 20914 nomenclature

It is important to review the standard terminology and abbreviations used in ISO 20914.

Measurement uncertainty – parameter characterizing the dispersion of the quantity values being attributed to a measurand, based on the information used.

Note 1. MU includes components arising from systematic effects, as in the case of corrections to the assigned quantity values of measurement standards. Sometimes estimated systematic effects are not corrected for, but instead, the associated MU components are incorporated.

Note 2. The parameter maybe, for example, a SD called standard MU (or a specified multiple of it), or the half-wide of an interval, having a stated coverage probability.

Note 3. MU comprises, in general, many components. Some of these may be evaluated by Type A evaluation of MU from the statistical distribution of the quantity values from series of measurements and can be characterized by SD. The other components, which may be evaluated by Type B evaluation of MU, can also be characterized by SD or evaluated from probability density functions based on experience or other information.

Note 4. In general, for a given set of information, it is understood that the MU is associated with a stated quantity value attributed to the measurand. A modification of this value results in a modification of the associated uncertainty.

Note 5. All measurements have bias and imprecision. For example, replicate measurements of a sample performed under repeatability conditions generally produce different values for the same measureand. Because the different values could all be reasonably attributed to the same amount of measureand, there is uncertainty as to which value should be reported as the value of the measureand.

Note 6. Based on available data about the analytical performance of a given measurement procedure, an estimation of MU provides an interval of values that is believed to include the actual value of the measurand, with a stated level of confidence.

Note 7. Available data about the analytical performance of a given measurement procedure typically comprise uncertainty of calibrator assigned values and long-term imprecision of IQC materials.

Note 8. In medical laboratories, most measurement are performed in singleton, and are taken to be an acceptable estimate of the value of the measurand, while the MU interval indicates other results that are also possible.

Abbreviations and symbols

• u(y) measurement uncertainty expressed as a standard deviation, also called standard uncertainty, for the measured value y of measurand Y
• k coverage factor applied to u to obtain an expanded confidence interval U
• U(y) expanded uncertainty of the measured value y of measurand Y
• u_Rw standard uncertainty for long-term imprecision of measured values obtained under defined conditions in same laboratory for a period sufficient to include all routine changes to measuring conditions, e.g., different lots of reagents, operators, and environmental conditions.
• u_cal standard uncertainty of the value assigned to an end-use calibrator
• u_ref standard uncertainty of the value assigned to a reference material
• u_bias standard uncertainty of a bias value
• b estimate of bias

Process for estimation of measurement uncertainty

ISO 20914 provides 3 options for calculating uncertainty, depending on the information available and whether bias has been corrected. See Figure 3, page 27, of ISO 20914 document.

 Option 1. Within laboratory imprecision or random error

u(y) = √u2_Rw

Use this equation if the lab has IQC data, but the manufacturer has not provided information on the uncertainty of calibration. When only considering random error with a single laboratory, the standard uncertainty is equal to the standard deviation determined from intermediate precision data.

Option 2. Within lab imprecision and calibration uncertainty

u(y) = √(u²_Rw + u²_cal)

Use this equation if the lab has IQC data to estimate uRw and the manufacturer has provided information on ucal.

Option 3. Within lab imprecision, calibration uncertainty, and bias correction

u(y) = √(u²_Rw + u²_cal + u²_bias)

Use this equation if the lab has corrected for bias and determined the uncertainty of the bias correction, ubias, has the IQC data to estimate uRw and the manufacturer has provided information on ucal. Note that laboratories may not be permitted to make corrections in reported results due to accreditation or regulatory guidelines, thus this option may not be available in many laboratories.

The Missing Option 4. Within lab imprecision, calibration uncertainty, and bias

There is one other obvious situation that is not addressed - what do you do when bias is clinically significant but cannot be eliminated or corrected. According to Note 1 under the definition for measurement uncertainty, the uncertainty components arising from systematic effects may be incorporated in the estimate of measurement uncertainty. However, the ISO 20914 document does not describe how to do this. Perhaps this omission occurs because metrological theory does not allow for uncorrected significant bias, even though that the existence of clinically significant biases is a common reality in measurements in laboratory medicine.

However, an explicit answer has been provided in a paper by a Spanish group [ 4].

u(y) = √(u²_Rw + u²_cal + b²)

where b is the bias that is observed for the measurement procedure.

These authors reference this equation to the ISO 20914 document, though in my reading of that document I cannot find this equation or an explicit statement for use of the bias-squared term to incorporate systematic effects in the calculation of measurement uncertainty. Another reference is to a paper by Frenkel et al [7] that reviews recommendations for incorporating uncorrected bias into measurement uncertainty.

“A general feature of all proposals for incorporating uncorrected bias is that a single bias, or its square, is summed with uncertainty components."

That statement confirms that a bias-squared term is one way of incorporating systematic effects in the calculation of measurement uncertainty, but it also suggests there are other ways that have been recommended earlier.

Possible approaches for incorporating uncorrected bias

Frenkel et al [7] reference earlier papers by Phillips et al [8] and Maroto et al [9] that summarize the different approaches for combining uncorrected bias. Historically, at least 4 different approaches have been considered.

Root Sum of Squares u (standard uncertainty), called U_RSSu

U_RSSu = k√(u²_Rw + bias²)

Note that bias² is added to the standard uncertainty of random error within lab, then the square root extracted before multiplying by coverage factor to provide the expanded uncertainty that includes systematic effects.

Root Sum of Squares U ( expanded uncertainty), called U_RSSU

U_RSSU = √(U² + bias²)

Note that bias² is added to the expanded uncertainty within lab (U²), then the square root extracted to provide the expanded uncertainty that includes systematic effects.

Sum of Expanded Uncertainty +/- Bias, called U_SUMU

U_SUMU+ = U - bias,    U_SUMU- = U + bias

Note that this approach provides the correct interval for a measured quantity because it is offset by the size of the bias.

Sum Expanded Uncertainty + Absolute Bias, originally called Ubias by Maroto [9] but termed USUMU|bias| here

U_SUMU|bias| = U + |bias|

Note that when U is estimated as a 95 percent interval, i.e., 1.96*SD, this is similar to the Total Error model and provides an estimate of the maximum error that may be expected. One difference is that the TE model typically considers a 95 percent one-sided interval and therefore uses a multiplier of 1.65. Another is that the TE model is used internally in the laboratory to manage the testing process when validating method performance and planning SQC procedures to achieve the intended quality of results, rather than for reporting the quality of results to users.

In comparing these equations, they are similar in that bias and u or U are added in different ways, starting with (1) RSS addition of u and bias, then multiplying by the coverage factor, (2) RSS addition of U and bias, coverage factor already included in U, (3) linear sum of U plus and minus bias to provide an asymmetric interval that corrects for bias, and (4) linear sum of U plus absolute value of bias to provide an enlarged interval that includes the complete interval in #3.

Comparison of uncertainty results when bias is included

Remember, the purpose of estimating measurement uncertainty is to be able to describe the interval or range of values that are expected to include the true value. Therefore, the best way to compare the calculations is to compare those intervals, not the values for U. Furthermore, according to ISO 15189, measurement uncertainty applies to the analytic part of the process. As the calculation models show, the uncertainty components are limited to the uncertainty due to random variation within the lab and the uncertainty due to calibration. Thus, we can compare the calculated results based on a limited number of random and systematic components.

Consider an example application where the target value (or true value) is 100 units and the random component varies from 0 to 10 and the systematic component (bias) varies from -20 to +20 units. [To keep things simple, we assume that u_cal is small and neglible.] The table below shows the upper and lower limits for the interval in which the true value is expected to be found.

• Example 1 is a check on the calculations. When uRw is 0.0 and bias is 0.0, there is no uncertainty and upper and lower limits by all 4 models are the same and represent the TV of 100 units.
• Example 2 also checks the calculations and shows that when uRw is 10 and bias is 0, all models provide the same uncertainty intervals of 80.4 to 119.6 units.
• Example 3 shows that when uRw is 0.0 and bias is 10 units, there is no random uncertainty and the SUMU model correctly identifies the TV as 100 units. The other models interpret the bias as a random variation that provides a range of values. Their uncertainty intervals include the TV, but their widths are different.
• Example 4 shows the effects of both random and systematic components. The SUMU model correctly describes the uncertainty interval corresponding to the correction for bias. Note that the reported test value of 110 units would not be corrected, only the uncertainty interval is corrected to show 80.4 to 119.6 units.
• Example 5 shows the effects of a larger bias. The SUMU model again correctly describes the uncertainty interval corrected for bias. The other models provide increasingly wider uncertainty intervals.
• Examples 6 and 7 show the effects of negative biases, where the SUMU model again correctly describes the uncertainty interval corrected for bias.

What’s the point?

It seems clear that the best way to handle uncorrected bias is to correct the uncertainty interval, rather than including bias as a component of variation that enlarges the expanded uncertainty. That is also the recommendation in the papers by Phillips et al [8] and Maroto et al [9]. It is difficult to understand why metrologists in laboratory medicine have focused on the value of U rather than providing the best estimate of the uncertainty interval. While it is often taboo to correct the result to be reported, there need not be a similar restriction on correcting an uncertainty interval that is provided to help users interpret the reported result.

References

1. ISO 15189:2012. Medical laboratories – Requirements for quality and competence. 3rd ed. International Organization for Standards, Geneva, Switzerland, 2012.
2. International Organization for Standardization. Medical laboratories – a practical guidance for the estimation of measurement uncertainty. ISO/TS 20914:2019.
3. Farrance I, Frenkel R, Badrick T. ISO/TS 20914 – a critical commentary. Clin Chem Lab Med 202;58-1192-90. https://doi.org/10.1515/cclm-2019-1209.
4. Rigo-Bonnin R, Diaz-Troyano N, Garcia-Tejada L, MArce-Galindo A, Valbuena-Asensio M, Canalias F. Estimation of the measurement uncertainty and practical suggestion for the description of metrological traceability in clinical laboratories. Biochem Med (Zagreb) 2021;31(1):010501.
5. Bureau International des Pois et Mesures. Evaluation of measurement data – guide to the expression of uncertainty in measurement (GUM). Joint Committee for Guides in Metrology 100:2008. http://www.bipm.org/en/publications/guides/gum.html.
6. . CLSI C51-A. Expression of measurement uncertainty in laboratory medicine; Approved guideline. Clinical and Laboratory Standards Institute, 950 West Valley Road, Wayne, PA 19087 USA.
7. Frenkel R, Farrance I, Backrick T. Bias in analytical chemistry: a review of selected procedures for incorporating uncorrected bias into expanded uncertainty of analytical measurements and a graphical method for evaluatiang the concordance of reference and test procedures. Clin Chim Acta 2019;495:129-138. https://doi.org/10.1016/j.cca.2019.03.1633.
8. Phillips SD, Eberhardt KR. Guidelines for expressing the uncertainty of measurement results containing uncorrected bias. J Res Natl Inst Stand Technol 1997;102:577-585.
9. Maroto A, Boque R, Riu J, Rius FX. Should not-significant bias be included in the uncertainty budget. Accred Qual Assusr 2002;7:90-94.