Tools, Technologies and Training for Healthcare Laboratories

Traceability, Harmonization and Standardization: Why It Matters

Traceability, Standardization, Harmonization Comparability. Even within the laboratory, these are not terms and concepts that excite most of us. But as the world gets smaller, as health records get more connected and as medical practices converge, reliable, comparable test results are going to matter more and more. Whether we want to get involved or not, we are already in the struggle for standardization.

 

 

Traceability, Harmonization and Standardization in Laboratory Medicine: Why It Matters - and Why Labs Need to Participate

Sten Westgard, MS
October 2011

In the course of my travels this year, I have come across some striking examples about a major problem with laboratory test results.

In the first case, I was in Brazil and speaking with a translator before one of my presentations. As I was trying to explain (and make her comfortable with) the concepts of laboratory test results and quality control, she suddenly interrupted, "You mean to tell me, all lab tests are not the same?" She was shocked.  Then she quickly related to me the story of her mother, who had just gotten three different laboratory results from three different labs for the same test. One of those lab reports indicated her mother needed to go on drug therapy; the other results said she didn't. The translator and her mother were perplexed about what to do.

In the second case, I was in Astana, the capital of Kazakhstan. As I was being introduced before a lecture, the moderator stated that Kazakhstan was a country where you could go to five different laboratories and get five different results for the same test. This was a case that had actually happened. The audience nodded in agreement with this problem.*

This is the retail end of the Traceability and Standardization problem. When patient results vary based not on the health status of the patient, but on the type of method used to measure the specimen. There is a common assumption that "a test is a test is a test," an unsubstantiated belief that grows stronger as one gets further away from the laboratory. Those of us in the laboratory know this is a myth. We know that differences between methods exist - and can be quite significant for some tests and/or methods - but we sometimes fail to grasp the extent of the differences.

For those of us in more "advanced" healthcare systems, we can't simply dismiss these anecdotes as something that only happens elsewhere. There are plenty of examples in even the most sophisticated healthcare system of test results that don't match up. Think about the differences between creatinine assays, or TSH assays, or troponin assays. Or even think about the difference between POC devices and core laboratory instruments.

Ultimately, the patients who navigate through our healthcare systems are the ones who feel the true impact of a lack of harmonization and standardization. When we have non-traceable methods, we inject mystery into our test results. Without a firm foundation, the numbers may be higher or lower or more uncertain. All of this can result in delay, confusion, and in the worst case, the wrong clinical action.

What Standardization, Harmonization and Traceability means

On a formal level, Traceability is the "property of a measurement result whereby the result can be related to a stated reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty." This is a definition from the International Vocabulary of Basic and General Terms in Metrology (VIM), which is something of a bible for metrologists. As for Standardization and Harmonization,

"The term "standardized" is used when results for a measurand are equivalent and, in addition, the results are tracable to the International System of Units (SE) through a higher-order primary (pure substance) reference material and/or reference measurement procedure. The term "harmonized" is generally used when results are equivalent either by being traceable to a reference material or based on a consensus approach, such as agreement to an all-methods mean, but neither a higher-order primary reference material nor a refererence measurement procedure exist."[Clin Chem 57:87:1108-1117(2011)]

[Please note: in this article, I won't be trying to quote all the formal definitions of Traceability and Standardization. There are many terms and definitions involved in Traceability, Harmonization and Standardization. In later articles, we intend to explore in more detail these terms and concepts. Right now we want to give a more plain-language introduction. My apologies to the metrological fundamentalists for any terminology uncertainty in the discussion that follows.]

Inevitably, every discussion of Standardization, Harmonization and Traceability uses a chart like this example:

traceability

Source: http://www.bipm.org/utils/en/pdf/Traceability_in_LabMed.pdf

This chart is called a "Traceability Chain" and it shows how methods and assays proceed from the reference measurement of the "true value" down to a routine measurement.

The essential concept of this chart is that we start with an agreed-upon standard for "Truth" for the test method in question. Sometimes, organizations like NIST or the WHO or a similar metrology institute can establish a primary reference method or material, which then can be used to create a primary calibrator. At a certain point, the manufacturer takes that primary calibrator and creates their own method and calibrators. Again, we will save all the definitions of the terms here (for example, the difference between working calibrator and product calibrator) for future articles. Suffice it to say that there is a path (or chain) from the primary reference standard down to the routine measurement procedure, and during this process, the measurement process gets a little bit less precise and more uncertain. So by the time the regular laboratory is using the method, there is some accumulated amount of error in each routine test. But the fact that there is an unbroken chain of Traceability means that we can determine how much error (or uncertainty) exists in the routine measurement. If we didn't have the Traceability chain, we wouldn't know how much error was present - we could only guess. The chain anchors us to the truth. Without it, we drift on a sea of uncertainty.

Where does Standardization Start? At the Top

The architecture of Traceability doesn't exist for many analytes. Furthermore, there is often a lack of agreement on what the "true value" of a particular test should be - there may even be a lack of any "true value" method or material at all. So there is a great deal of work to be done. Establishing Traceability requires that a system must be set up for reference measurements:

  • Measurand definition
  • Stated reference procedure and/or reference material
  • Knowledge of the measurement uncertainties
  • Unbroken chain of calibrations and value assignments.

In other words, we first must agree on what we intend to measure, then find a method or material that represents a known "true" value of that measurement, followed by as assessment of the error (uncertainty) that is involved in trying to measure that "true" value. Finally, once we've got one good reference measurement or material, then we have to walk down that Traceability chain from the truest reference method/material down to the routine laboratory measurement on a routine patient specimen. The unbroken chain allows us to estimate the how far our routine result might be dispersed from the true value.

Much of the Traceability work is being led by ISO. Here's a short list of the standards that govern Reference methods and Reference measurement Procedures:

  • ISO 17511 In vitro diagnostic medical devices - Measurement of quantities in biological samples - Metrological
    traceability of values assigned to calibrators and control materials
  • ISO 18153 Metrological traceability of values for catalytic concentration of enzymes assigned to calibrators and
    control materials
  • ISO 15193 Presentation of reference measurement procedures
  • ISO 15194 Description of reference materials
  • ISO 15195 Reference Measurement Laboratories

Further, committees like the JCTLM (Joint Committee on Traceability of Laboratory Medicine) are working on establishing databases of reference methods and materials, so that manufacturers have a resource to find the primary standards and reference methods for their own instruments and methods. JCTLM is also assembling a list of Reference Measurement Laboratories, laying the groundwork for the first steps of the Traceability chain.

What does progress look like?

A gross simplification of the Traceability, Harmonization and Standardization process is to say that it is the result of committee work. Expert groups form and work together to achieve agreement on the definition of measurands and the selection of reference methods.

A recent publication in Clinical Chemistry provides a Roadmap for Harmonization of Clinical Laboratory Measurement Procedures [W. Greg Miller, Gary L Myers, Mary Lou Gantzer, Stephen E Kahn, E Ralf Schonbrunner, Linda M Thienpont, David M Bunk, Robert H Christenson, John H Eckfeldt, Stanley F Lo, C Micha Nubling, and Catharine M Sturgeon. Clin Chem 57:8:1108-1117, 2011.] This spells out the steps necessary to move an analyte toward Standardization, Harmonization and Traceability.

One of the interesting reports of this Roadmap is a table which lists the state of progress in Harmonization. For instance, Electrolytes, glucose, and cortisol are part of a Category 1 Reference System, which means a Reference Measurement Procedures (RMP) exists, as well as a Primary (pure substance) reference material and Secondary (value-assigned) reference material. At the other end of the scale, methods for Epstein-Barr virus, varicella zoster virus are in the Category 5 Reference System, which is to say, no RMP and no Primary or Secondary reference materials exist (yet). Using the catgories for reference systems identified in ISO 17511, it's possible to determine how hard (or easy) it will be to establish Harmonization for a measurand.

Another interesting detail in this roadmap are some of the milestones. In the section on Establishing the clinical requirements for harmonization, the authors discuss a system for developing method harmonization criteria. This is directly related for the 1999 Stockholm consensus heirarchy on global quality specifications:

"Total allowable errror (TEa) based on clinical outcomes data is ideal for [determining allowable bias and imprecision for laboratory tests] but frequently is not available... Data on biological variability, ideally derived from studies fulfilling the STARD criteria, may enable a useful definition of TEa....Finally, when no other data are available, approaches based on recommendations from professional organizations, or an expert panel, are used to established TEa for a given measurand."

Thus, the concept of Total Error, long familiar to the laboratory, can help in the Harmonization and Standardization process. Total Error helps to set the criteria for how much error we can allow in methods.

To see a more concrete example of Harmonization, here's a list of the some of the papers generated by IFCC working groups on the topic of thyroid function testing:

  • Analytic bias of thyroid function tests: analysis of a College of American Pathologists fresh frozen serum pool by 3900 clinical laboratories. Steele BW, Wang E, Klee GG, Thienpont LM, Soldin SJ, Sokoll LJ, Winter WE, Fuhrman SA, Elin RJ. Arch Pathol Lab Med. 2005 Mar;129(3):310-7.
  • Feasibility study of the use of frozen human sera in split-sample comparison of immunoassays with candidate reference measurement procedures for total thyroxine and total triiodothyronine measurements. Thienpont LM, Van Uytfanghe K, Marriott J, Stokes P, Siekmann L, Kessler A, Bunk D, Tai S. Clin Chem. 2005 Dec;51(12):2303-11.
  • Measurement of free thyroxine in laboratory medicine--proposal of measurand definition. Thienpont LM, Beastall G, Christofides ND, Faix JD, Ieiri T, Miller WG, Miller R, Nelson JC, Ross HA, Ronin C, Rottmann M, Thijssen JH, Toussaint B; International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) IFCC Scientific Division Working Group for Standardization of Thyroid Function Tests.
    Clin Chem Lab Med. 2007;45(4):563-4.
  • Proposal of a candidate international conventional reference measurement procedure for free thyroxine in serum. International Federation of Clinical Chemistry and Laboratory Medicine IFCC, IFCC Scientific Division Working Group for Standardization of Thyroid Function Tests WG-STFT, Thienpont LM, Beastall G, Christofides ND, Faix JD, Ieiri T, Jarrige V, Miller WG, Miller R, Nelson JC, Ronin C, Ross HA, Rottmann M, Thijssen JH, Toussaint B. Clin Chem Lab Med. 2007;45(7):934-6.
  • Report of the IFCC Working Group for Standardization of Thyroid Function Tests; part 1: thyroid-stimulating hormone. Thienpont LM, Van Uytfanghe K, Beastall G, Faix JD, Ieiri T, Miller WG, Nelson JC, Ronin C, Ross HA, Thijssen JH, Toussaint B; IFCC Working Group on Standardization of Thyroid Function Tests. Clin Chem. 2010 Jun;56(6):902-11.
  • Report of the IFCC Working Group for Standardization of Thyroid Function Tests; part 2: free thyroxine and free triiodothyronine. Thienpont LM, Van Uytfanghe K, Beastall G, Faix JD, Ieiri T, Miller WG, Nelson JC, Ronin C, Ross HA, Thijssen JH, Toussaint B; IFCC Working Group on Standardization of Thyroid Function Tests. Clin Chem. 2010 Jun;56(6):912-20.
  • Report of the IFCC Working Group for Standardization of Thyroid Function Tests; part 3: total thyroxine and total triiodothyronine. Thienpont LM, Van Uytfanghe K, Beastall G, Faix JD, Ieiri T, Miller WG, Nelson JC, Ronin C, Ross HA, Thijssen JH, Toussaint B; IFCC Working Group on Standardization of Thyroid Function Tests. Clin Chem. 2010 Jun;56(6):921-9.
  • Standardization activities in the field of thyroid function tests: a status report. Thienpont LM, Van Uytfanghe K, Van Houcke S; IFCC Working Group for Standardization of Thyroid Function Tests. Clin Chem Lab Med. 2010 Nov;48(11):1577-83.
  • IFCC international conventional reference procedure for the measurement of free thyroxine in serum: International Federation of Clinical Chemistry and Laboratory Medicine (IFCC) Working Group for Standardization of Thyroid Function Tests. Van Houcke SK, Van Uytfanghe K, Shimizu E, Tani W, Umemoto M, Thienpont LM. Clin Chem Lab Med. 2011 Aug;49(8):1275-81.

As you can see, it can take years of effort to move toward Traceability and Standardization. This effort must be replicated across the entire spectrum of testing, for every analyte, when and where professionals can agree to make the first step and form the committee to start the Traceability, Harmonization and Standardization process.

Where does the momentum for Traceablity and Standardization need to come from? The Bottom

One of the paradoxes of Traceability, Harmonization and Standardization is that laboratories on the front line feel like they have little power to influence manufacturers and the Traceability process. When ISO committees are doing all the major work, how does one individual laboratory - or just one individual laboratory professional - make a contribution?

On the most basic level, laboratories need to move the market toward Traceable, Harmonized and Standardized methods. One of the major obstacles to Standardization is that manufacturers may find it more profitable to have a proprietary, non-standard method. Standardization might rank low on the list of priorities when a new instrument is being purchased (for example, cost, speed, size, and service might be larger concerns of the laboratory and the administration). So if it's cheaper to manufacture and/or purchase a non-standard method, the only way to counter that is by strongly expressing demand for Traceable methods. Manufacturers must be forced to realize that their non-tracable methods do not get rewarded in the marketplace. Instead, they need to see concrete evidence that an investment in shifting toward tracable methods means more business and profit.

Traceability, Harmonization and Standardization of methods is one of those "public goods" where the benefits do not exclusively accrue to an individual laboratory or an individual diagnostic manufacturer. The main benefit is to the patient population - their test results can be compared across instruments and health systems without errors, uncertainty or misunderstanding. In the ideal world, that means patients can move from hospital to hospital, have their tests at different labs, and the results are comparable. Better patient care is the result, something that can reduce costs in specific healthcare systems (particularly if duplicate testing can be reduced), but again those costs might not be realized as direct profits to the diagnostic manufacturers.

Here's a case where the mission of healthcare has to override the profit margin of manufacturers and health systems. Traceability means better care for our patients. It's worth the extra effort and expense to make that happen.

Some good introductory references on Traceability and Standardization:

  • Traceability in Laboratory Medicine, Hubert W Vesper and Linda M Thienpont, Clinical Chemistry 55:6:1067-1075(2009).
  • Roadmap for Harmonization of Clinical Laboratory Measurement Procedures, W Greg Miller, Gary L Myers, Mary Lou Gantzer, Stephen E Kahn, E Ralf Schonbrunner, Linda M Thienpont, David M Bunk, Robert H Christenson, John H Eckfeldt, Stanley F Lo, C Micha Nubling, Catherine M Sturgeon, Clinical Chemistry 57:8:1108-1117(2011).
  • The Joint Committee for Traceability in Laboratory Medicine (JCTLM): a global approach to promot the standardisation of clinical laboratory test results, Armbruster D, Miller RR, , Clin Biochem Rev 2007 Aug: 28(3):15-113.

[*Note: some of the details of these anecdotes have been changed to preserve the individual privacy. Also, it is entirely possible that the discrepancies between test results may not be caused by a lack of standardization, but simply by poor instrumentation and/or poor quality practices. Standardization alone will not solve all problems, but it is a key and essential step toward the solution. Standardization will expose poor instrumentation and poor practices, but excellent instrumentation without standardization may only lead to an argument over "who's right."]