August 2009

Dr. Westgard recently visited Argentina and Chile, where he chaired a discussion on the measurement of uncertainty vs. the management of quality.

A little uncertainty goes a long way!

Many thanks to the laboratory scientists in Beunos Aires and Santiago for the kind hospitality during my recent visit. It is a long way to travel – so far that the seasons are opposite. August is the end of winter in South America, but the weather and hospitality were much warmer, at least spring-like. The uncertainty of winter did set the stage for a serious discussion of the “measurement of uncertainty vs the management of quality,” which was the title of my lecture. As the picture below shows, we were in a palatial setting, which set a pretty high standard for the presentation.

The subject of measurement uncertainty doesn’t get much attention here in the US, but because of my international travels and colleagues, I’ve gradually gotten interested in the subject and concerned about the difficulties in measuring uncertainty, “where relevant and possible.” Anyone who has read ISO 15189 [1] knows that quote “where relevant and possible” and those who are working for accreditation also know the difficulties involved in real applications in the laboratory. I sometimes take the position that measurement uncertainty is “irrelevant and impossible,” but I do so to make people stop and think whether it is actually useful for improving and assuring quality in the medical laboratory. Just measuring uncertainty to satisfy accreditation guidelines is not by itself a sufficient reason to spend the time and effort needed to do this right.
There are some discussions of this topic on this website, under the theme “A War of Words in Laboratory Medicine.”

• Part I. “Total Error vs Measurement Trueness and Uncertainty” provides an introduction to the current issue.
• Part II. “Concepts and Terminology in a Changing Uncertain World” reviews the history of accuracy, precision, and uncertainty and the terminology that has evolving over time.
• Part III. “Intended Applications and Customers” provides some thoughts about the differences between the theory of metrology and the applications in a medical laboratory.

I covered much of this material in the rather short time of an hour (though it might have seemed much longer to anyone in the audience who didn’t find this topic of interest), so hopefully the links to these discussions will provide a more thorough review and presentation of ideas.

Lecture at Santiago de Chile

As you will discover, I have a vested interest in the current argument about “measuring uncertainty vs managing quality.” I was involved in the introduction of the “total error” concept in medical laboratories over 30 years ago [2]. At that point in time, the metrologists did not have a practical way of evaluating the net or total effect of random and systematic errors on a test result made by a single measurement. They considered precision and accuracy separately because in their world it was common practice to make multiple measurements (and calculate the average) to reduce the effects of random error. In medical laboratories, such a practice was too costly, thus individual measurements were made and, as such, were subject to the total effect of imprecision and inaccuracy of a method.

A picture with a smaller workshop audience in Buenos Aires

Of course, the outcome of the current argument need not be either total error or measurement uncertainty because both concepts could be useful in appropriate ways. Practical applications of measurement uncertainty should evolve out of our traditional error framework and the existing concept of total error. The two concepts can co-exist, but many metrologists argue that the concept of total error must be vanquished to make way for measurement uncertainty. Instead, consider the following cases where each may be useful:

• In cases where quality requirements can be readily defined, then the priority should be to manage quality to guarantee that test results are good enough for their intended use. ISO 15189 makes the statement that “internal quality control shall be designed to verify the attainment of the intended quality of test results.” It doesn’t include the qualifier “where relevant and possible.” Therefore, the first priority of the medical laboratory is to manage quality.
• In cases where quality requirements are not so easy to define, the laboratory should at least characterize the performance or variability of the measurement procedure, i.e., the uncertainty of measurement. Even then, it is not simple to figure out how to do so, and there are many arguments to consider about the GUM (Guide for estimation of the Uncertainty of Measurements) bottom-up methodology, which requires detailed models to quantify the variation of individual steps and components of variation, vs top-down methodology that utilizes data from internal quality control, method validation, and sometimes external quality assessment.

With new guidelines for the calculation of measurement uncertainty emerging from both ISO and CLSI, the medical laboratory will have even more information about what to do, yet I suspect will still struggle with making measurement uncertainty “relevant and possible.” To accomplish the latter will mean identifying specific applications that provide additional value to our current error framework for managing quality, for example, by placing a greater emphasis on traceability and using measurement uncertainty to characterize the quality of calibration and the comparability of different measurement procedures. Traceability of methods and comparability of test results merit our full attention and increased efforts to improve the quality of laboratory testing.

References

1. ISO 15189:2007. Medical laboratories – Particular requirements for quality and competence. ISO, Geneva.
2. Westgard JO, Carey RN, Wold S. Criteria for judging precision and accuracy in method development and evaluation. Clin Chem 1974;20:825-833.