Links

October 2005

The second part of a discussion of Dr. Westgard's four-city tour of India.

• Summary of Presentations
• Three estimates of Sigma performance
• Links to Clinical and Analytical QC

Dr. Westgard with Eminent Lab personnel in Mumbai

Given that many laboratories in India are interested in ISO certification under the guidelines of 15189, there is a need to understand how to define the “intended use” and “intended quality” of “examination procedures,” the latter being ISO’s term for analytical tests. “Intended use” is the phrase that appears in ISO section 5.5: “performance specifications for each procedure used in an examination should relate to the intended use of that procedure.” “Intended quality” is the term found in section 5.6, Assuring the Quality of Examination Procedures, which provides more specific guideline for IQC, as follows: “The laboratory shall design internal quality control systems that verify the attainment of the intended quality of results. It is important that the control system provide staff members with clear and easily understood information on which to base technical and medical decisions…”

Summary of presentation

Laboratory scientists recognize that the analytical performance needed for a measurement procedure depends on “medical usefulness” of the test result, which might also be termed “clinical quality”. Physicians and patients assume that the necessary clinical quality has been objectively defined and is being carefully managed by instrument manufacturers and healthcare laboratories to guarantee the quality of laboratory tests. In this age of Evidence-Based Medicine, there is little evidence that manufacturers set the specifications for analytical methods objectively on the basis of clinical quality or that laboratories evaluate measurement procedures objectively and establish QC procedures that will assure the clinical quality required for patient care. Furthermore, the ISO/GUM approach for characterizing measurement uncertainty masks the real-world consequences of pre-analytic factors such as within-subject biologic variation and actually conflicts with the development and application of a practical model for relating measurement performance specifications to the clinical quality needed for patient care.

The concepts and principles of Six Sigma Quality Management can be applied to provide guidance for clinical laboratory testing. The concept of “tolerance limits” corresponds to an analytical quality requirement in the form of an allowable total error (TE_a). An analytical error budget can be formulated to account for imprecision and bias of the measurement procedure and can also accommodate the uncertainty of the IQC procedure for detecting changes in process performance. Quality requirements in the form TE_a are readily available from proficiency testing programs and external quality assessment schemes, plus an allowable biologic total error can be calculated from the biologic goals for imprecision and bias, thus this analytical error model can be readily applied to translate existing quality goals and requirements into measurement specifications for imprecision, bias, and IQC rules and numbers of control measurements. Expansion of the analytical error model to include pre-analytic components of variation provides a clinical error budget that can be directly related to the clinical decision interval or gray-zone for medical interpretation of a test result. This expansion takes into account within-subject biologic variation, which is an important variable that must be considered when assessing the uncertainty in a clinical application. This clinical quality model can be used to translate test interpretation guidelines directly into specifications for imprecision, bias, and IQC.

While we have been using these predictive models for more then ten years, I have found that many people have difficulty in accepting the results. They don’t believe that mathematical predictions could reveal what happens in the real world. To address those doubts, we have recently assessed the actual quality being achieved in US laboratories based on outcome studies, in this case, proficiency testing results. In this age of evidence based medicine, this is the best evidence of the quality of laboratory testing today.

Three different estimates of method sigmas have been made:

• National Test Quality (NTQ) =TE_a/SD_group
• National Method Quality (NMQ) = wt.av. [(TE_a-bias_subgroup)/SD_subgroup]
• Local Method Quality (LMQ) = wt.av. [TE_a/SD_subgroup]

NTQ and NMQ are the relevant estimates when there are national test interpretation guidelines, such as for cholesterol where the NCEP prescribes national treatment guidelines in the US. LMQ is relevant only when there are local reference limits or cutoffs that have been determined by the laboratory for the patient populations being served. For common tests such as cholesterol, glucose, and calcium, the estimates of NTQ and NMQ are about 3 to 4 sigma, which confirms the predictions from quality-planning models.

The applications of quality-planning models are made practical via computer programs, which can automatically translate clinical or analytical quality requirements into specifications for imprecision, bias, and IQC. Such computer technology can be embedded in instrument systems, analytic workstations, EQC data analysis, or PD programs. Given the emerging QC technology, what is needed next is a professional interest in assessing the quality of laboratory tests and a commitment to making improvements as needed.

Links to Clinical Quality and Analytical QC

with Dr. Suhasini - of Apollo Hospital - Hyderabad

This 2nd lecture covered more advanced topics, particularly the idea of “clinical quality” and how that issue might be addressed. This presentation was divided into two parts: the 1st dealing with “evidence-based medicine” (EBM), the development of treatment guidelines, and the underlying models (what I call quality-planning models) that relate treatment criteria to analytical performance and laboratory QC; the 2nd part dealing with observational evidence on the quality of laboratory testing today in light of national quality requirements, such as represented by the US CLIA requirements, NCEP treatment guidelines for cholesterol, and recent NACB EBM guidelines for glycated hemoglobin. The 1st part is “what we need” and the 2nd part is “what we’ve got” in the quality of laboratory testing today.

• Quality requirements are confusing because there are many different ways to define quality goals. For an overview of the different types of quality requirements and how they fit into a system, see “Defining Quality Requirements” .
• Quality requirements need to be translated into operating specifications for allowable precision, allowable bias, and necessary QC, i.e., the operating conditions needed at the bench. The relationship between quality and operating specifications can be described by “quality-planning models,” which are analogous to error-budgets.
• A mathematical description of the quality-planning models
• A graphical tool that describes the relationship between quality, allowable bias, allowable precision, and necessary QC is the “Chart of Operating Specifications” or “OPSpecs chart.”
• Use of the OPSpecs tool to select QC for a cholesterol test having an analytical quality requirement in the form of an allowable total error (TE_a) is illustrated in detail.
• Use of the OPSpecs tool to select appropriate QC for a cholesterol test having a clinical quality requirement (D_int=20%) is illustrated in detail.
• The issue of current test quality – myths vs metrics – is presented in the introductory essay on Quality in Laboratory Testing Today, part I.
• The methodology for estimating sigma-metrics from EQA or PT data.
• Results of this assessment the quality of laboratory testing today are provided for cholesterol, calcium, prothrombin time and INR.
• A detailed discussion of glycated hemoglobin and the national guidelines in the US.
• The implementation of quality-planning models using a PC program called EZ Rules 3.

James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.