Quality Requirements: The Debate heats up!

After years of neglect, people are beginning to get interested in the issue of what quality requirements are, where they come from, and which ones must be met by today's laboratory. Dr. Westgard gives an introduction to some of the issues and suggests a system of quality requirements is needed to accomodate different approaches and applications.

ISO TAG 212 WG3 draft document
Need for a system of quality standards
An example system
What's the point?
References

After simmering on the back burner for many years, the issue of how to define quality requirements for laboratory tests is beginning to heat up again! This was one of the topics of the program and the subject of much of the informal discussion at the European Conference on Quality [R]evolution in Clinical Laboratories that was held in Antwerp on October 29-30, 1998. Contributors included Callum Fraser (Scotland), Renee Dybkaer and Per Hyltoft Petersen (Denmark), Sverre Sandberg (Norway), Henk Goldschmidt (Holland), and Jean Clauder Libeer (Belgium), as well as David Kelly (USA, NCCLS), David Bruns (USA, University of Virginia, editor of Clinical Chemistry), and Sharon Ehrmeyer (USA, University of Wisconsin-Madison). In its 4th year, this conference emerged as one of the key venues for the serious discussion of international issues in laboratory quality management.

We are pleased to provide on this website an in-depth discussion by Dr. Callum Fraser on the use of "biological variation data for setting quality specifications", along with a database of figures for intra- and inter-individual biological variability from Drs. Maria Angeles Sebastian-Gambaro, Francisco Javier Liron-Hernandez, and Xavier Fuentes-Arderiu.

ISO TAG 212 WG3 draft document

The current discussion of quality requirements is being stimulated by a draft document on analytical goals that is being developed by Working Group 3 (WG3) of the International Standards Organization Technical Advisory Group number 212 (ISO TAG 212). The ISO TAG 212 activities are being administered via NCCLS, but the ISO process for developing a standard seems to be quite different from the NCCLS process. One difference is the composition of the committee and working groups, where ISO depends on official country representation whereas NCCLS traditionally depends on scientific experts in the specific area. Therefore, many of the laboratory scientists who are experts in the area of analytical goals are not members of the working group and have not been involved in the development of the standard. Their reaction to the ISO draft document has been so heated that a proposal has been made to the International Union of Pure and Applied Chemistry (IUPAC) to sponsor a conference to formulate a consensus document to aid ISO in their efforts to prepare a useful standard.

Need for a system of quality standards

Hopefully, the outcome of this debate will be a system of quality standards that are practical for laboratory applications. A system is needed because there will never be agreement on a single approach for defining standards of quality. There is a need to define a hierarchy of quality standards in a manner analogous to the hierarchy of standard materials and methods for documenting the traceability of a field method. Different standards are needed and are appropriate in different situations. Rather than arguing about the best way to define a quality standard, we need to establish the relationships between the different types of quality standards and identify the proper application of each.

For example, there is a natural hierarchy for certain types of quality standards, such as clinical outcome criteria, analytical outcome criteria, and analytical operating criteria. Clinical outcome criteria encompass the highest number of variables or factors that affect the value observed for a test result. Analytical outcome criteria encompass all the analytical factors, but do not consider the effects of pre- or post- analytical factors. Operating criteria provide specifications for individual components of error, such as imprecision and inaccuracy, as well as specifications for quality control. Thus, there is a natural hierarchy from broad clinical criteria to overall analytical performance to specific error characteristics on the basis of the factors encompassed by the different criteria.

Here are some other differences that can be considered in a system of quality standards:

Different quality standards require different formats, e.g., clinical outcome criteria can be defined in terms of medically important changes in test values, analytical outcome criteria can be stated in the form of allowable total errors, and analytical operating specifications are stated in terms of the allowable imprecision (CV), allowable inaccuracy (bias), and QC (control rules, number of control measurements) that are needed in the daily operation of a method.
Different quality standards are needed for different applications, e.g., clinical outcome criteria are used in guidelines for interpretation of patient test results, analytical total error criteria are used to score proficiency testing results, and criteria for imprecision, bias, and QC are used to manage the routine operation of a method.
Different sources of information are appropriate for defining different types of criteria, i.e., physician practice guidelines and standard clinical pathways may be useful for defining clinical outcome criteria, population biological variation may be useful for defining the allowable total error, individual biological variation may be useful for defining the allowable imprecision, and the sensitivity of diagnostic patient classifications may be useful for defining the allowable bias.
Different sources of information may be available, or may be more reliable, at different times during the evolution of a testing process, e.g., for new diagnostic tests, it may be possible to define medically important changes in the test results from the initial clinical studies of a method's diagnostic sensitivity, specificity, and predictive value; for well-established tests, there is already available an extensive "data-bank" of estimates of biologic variability.
Different quality standards may take priority in different situations, e.g., government regulations may place a high priority on satisfying proficiency testing criteria in certain laboratory situations, whereas special patient needs may set more demanding clinical criteria in other settings.

For all these reasons, there is a need to develop a system that integrates the different types of quality standards, different sources of information or data for defining those standards, and different applications of those standards

An example system

The accompanying figure shows the relationships between certain kinds of quality criteria. Starting at the top left of the figure, standard treatment guidelines (clinical pathways, clinical practice guidelines, etc.) can define the medically important changes that establish clinical outcome criteria (or decision intervals, D_int). Such clinical criteria can be converted to laboratory operating specifications for imprecision (s_meas), inaccuracy (bias_meas), and QC (control rules, N) by a clinical quality-planning model [1] that accounts for preanalytical factors, such as within-individual biologic. Biologic goals based on within-subject biologic variability should set a boundary condition on these operating specifications, defining the most demanding condition for stable performance that would be required to monitor changes in individual subjects. The right side of the figure shows how proficiency testing criteria define analytical outcome criteria in the form of allowable total errors (TE_a), which can likewise be translated to operating specifications (smeas, biasmeas, control rules, N) via an analytical quality-planning model [2]. Note that the allowable total error can also be set on the basis of total biologic goals [3], therefore the extensive database of individual biologic variation can also be useful in this situation.

What's the point?

Quality standards for clinical outcome, analytical outcome, and analytical performance characteristics are all part of a system for analytical quality management. With the systems approach outlined here, both clinical and the analytical outcome criteria can be converted to operating specifications that define the performance characteristics required by the testing process at the bench level of operation. The bottom line is the imprecision, inaccuracy, and QC that are necessary for the laboratory to manage and assure the quality of the testing process. One common limitation of current approaches for defining specifications for allowable imprecision and allowable inaccuracy is that the known "insensitivity" of common laboratory QC procedures is not adequately considered, therefore the specifications apply only to stable methods that do not need any quality control. Until operating specifications - including QC specifications - can be defined, quality standards will have little practical value for managing and assuring the quality of the test results produced in routine laboratory service.

References

Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program. Clin Chem 1991;37:656-661.
Westgard JO, Wiebe DA. Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing. Clin Chem 1991;37:1938-1944.
Hyltoft Petersen P, Ricos C, Stockl D, Libeer J-C, Baadenhuijsen H. Fraser CG, Thienpont L. Proposed guidelines for the internal quality control of analytical results in the medical laboratory. Eur J Clin Chem Clin Biochem 1996;34:983-999.

Tools, Technologies and Training for Healthcare Laboratories

Quality Requirements and Standards