SIX SIGMA - MEDICAL CUTOFFS AS TOLERANCE LIMITS
James O. Westgard, PhD
SIX SIGMA -
The definition of the tolerance limits is critical in Six Sigma Quality Management for the design of new processes, the assessment of performance of current processes, and quality control of current production processes. For analytical testing processes, several approaches have been advocated for defining tolerance limits or quality requirements of laboratory tests. The consensus conference on “Strategies to set global analytical quality specifications in laboratory medicine” recommended the following hierarchy of models for setting analytical quality specifications [1]:1. Evaluation of the effect of analytical performance on the clinical outcomes in specific clinical settings.
2. Evaluation of the effect of analytical performance on clinical decisions in general:
a. Data based on components of biological variation.
b. Data based on analysis of clinician’s opinions.3. Published professional recommendations.
a. From national and international expert bodies.
b. From expert local groups or individuals.4. Performance goals set by
a. Regulatory bodies
b. Organizers of External Quality Assessment (EQA) schemes5. Goals based on the current state of the art
a. As demonstrated by data from EQA or Proficiency Testing schemes.
b. As found in current publications on methodology.It is important to understand that the different types of quality requirements may not be directly comparable [2]. For example, a clinical quality requirement (decision interval) often encompasses preanalytical and analytical factors of variation, whereas an analytical requirement encompasses only analytical factors; furthermore, an analytical quality requirement may encompass both imprecision and inaccuracy (allowable total error), or may be specific for either imprecision (allowable SD or CV) or inaccuracy (allowable bias).
“When available, and when appropriate for the intended purpose, models higher in the hierarchy are to be preferred to those at lower levels” [3]. This means that quality requirements for interpretation in a specific clinical situation are preferred over general clinical requirements, which in turn are preferred over professional recommendations, regulatory requirements, and state of the art performance. The ability to utilize clinical treatment guidelines as quality requirements is one of the important advantages of the clinical quality-planning model and the OPSpecs quality design and control tool. Here’s a good example of the use of medical cutoff points in the selection of QC procedures.
Cardiac markers is an area of laboratory testing that is “hot” in the the new millennium. So hot that the journal of Clinical Chemistry published its first theme issue on the topic “Heart Health” in March 2001. The lead editorial emphasized the redefinition of myocardial infarction on the basis of laboratory measurement of cardiac markers, particularly Troponin, and also recommended the use of CK-MB isoforms and myoglobin for early triage of patients [4].
Application of the consensus hierarchy of quality requirements to cardiac markers depends on the data and information available for the individual markers. The preferred approach is to use a specific clinical requirement that relates to the diagnostic application for myocardial infarction. Current recommendations on medical cutoff points can be used to define quality requirements for these tests. Here’s how.
Troponin I. Given a TnI cutoff of 0.40 ug/L, a measurement of this value should represent a medically important change from the normal amount of TnI that would be present in a healthly patient, which is very low and can be represented by the detection limit of the method or 0.09 ug/L. This difference of 0.31 ug/L, or 77% at the cutoff point, is the medically important change or clinical decision interval that needs to be correctly measured by the test method. This corresponds to a model 1 specification, or the highest model in the hierarchy, which is the preferred clinical quality requirement. The National Academy of Clinical Biochemistry’s (NACB) recommendation of a CV of 10% for all cardiac markers corresponds to a model 3 specification from a professional expert group [5].
Myoglobin. Given a Myoglobin cutoff of 100 ug/L and an upper limit of the reference interval for healthy individuals of approximately 50 ug/L, a clinical decision interval of 50 ug/L or 50% at the cutoff point describes the medically important change that must be detected by the test. This corresponds to a model 1 specification, or the preferred clinical quality requirement. The National Academy of Clinical Biochemistry’s (NACB) recommendation of a CV of 10% for all cardiac markers corresponds to a model 3 specification from a professional expert group [5].
CK-MB. Given a cutoff of 7.0 ug/L and an upper limit of the reference interval of 3.7 ug/L, a clinical decision interval of 3.3 ug/L or 47% describes the medically important change that must be detected by the test. The upper limit of the reference interval for males is somewhat higher, 4.3 ug/L, giving a decision interval of 2.7 ug/L or 39%. Both of these correspond to model 1 specifications. A general clinical quality requirement can also be described on the basis of biologic variation and is given as 31.2% by Ricos [6], which corresponds to a model 2 specification. The National Academy of Clinical Biochemistry’s (NACB) recommendation of a CV of 10% for all cardiac markers corresponds to a model 3 specification from a professional expert group [5].
A next-generation analyzer provides a good example of the delivery of cardiac marker testing at the point-of-care (POC), in this case, the emergency room department where patients with chest pain must be triaged. Performance information is provided in an evaluation study published by Apple et. al [7]. The First Medical Alpha Dx analyzer provides on-board QC using two levels of control materials and also supports the data analysis for external QC. Effects of analytical bias are minimized by the determination of detection limits and cutoff points for the tests performed, therefore bias can be considered to be minimal and near zero. The table below summarizes the precision performance that can be used in selecting statistical QC procedures that are appropriate for the tests.
| Type of material | Myo | CK-MB | TnI |
| Blood duplicates | 3.9% | 3.9% | 7.3% |
| Serum duplicates | 4.0% | 2.7% | 2.4% |
| On-board Level I |
6.7% (58 ng/mL) |
4.8% (5.0 ng/mL) |
7.7% (0.28 ng/mL) |
| On-board Level II |
4.4% (626 ng/mL) |
4.6% (32.1 ng/mL) |
8.4% (9.06 ng/mL) |
| External frozen LO serum pool |
7.7% (106 ug/L) |
6.3% (8.4 ug/L) |
7.4% (0.30 ug/L) |
| External frozen HI serum pool | 6.2% (400 ug/L) | 5.2% (133 ug/L) | 8.8% (2.38 ug/L) |
The quality requirements and precision performance described above are utilized with the QC Validator® computer program.
A clinical decision interval of 77% represents the clinical quality requirement for the detection of a change from the detection limit of 0.09 ug/L to the cutoff point of 0.4 ug/L. In applying the clinical quality-planning model, there are no available data on the expected within-subject biologic variation, therefore the imprecision at the detection limit (CV=11%) is used to represent preanalytic variability. Using the largest CV observed for on-board controls (8.4%), an appropriate QC design would be a 14s rule with 2 control measurements per run, which should provide greater than 90% detection of any medically important analytical error while keeping the false rejections at less than 1%. See the OPSpecs chart below for documentation of this recommendation.
A clinical decision interval of 50% represents the clinical quality requirement for detection of a change from the upper limit of the reference range of 50 ug/L to the cutoff point of 100 ug/L. The expected within-subject biologic variability of 5.6% [6] is accounted for in the clinical quality-planning model. Using the largest CV (6.7%) observed for on-board controls, the appropriate QC design is a 14s control rule with 2 control measurements per run, which should provide greater than 90% detection of medically important systematic errors while keeping false rejections at less than 1%. See the OPSpecs chart below for documentation of this recommendation.
A clinical decision interval of 47% represents the clinical quality requirement for detection of a change from the upper limit of the reference range of 3.7 ug/L (applies to the population of both males and females as a whole) and the cutoff point of 7.0 ug/L. The expected within-subject biologic variation is 9.3% [6]. Using the largest CV (4.8%) observed for on-board controls, the appropriate QC design is a 14s control rule with 2 control measurements per run, which should provide greater than 90% detection of medically important systematic errors while keeping false rejections at less than 1%. See the OPSpecs chart below for documentation of this recommendation.
A clinical decision interval of 77.5% represents the clinical quality requirement for the detection of a change from the detection limit of 0.09 ug/L to the cutoff point of 0.4 ug/L. In applying the clinical quality-planning model, there are no available data on the expected within-subject biologic variation, therefore the imprecision at the detection limit (CV=11%) is used to represent preanalytic variability. A CV of 7.4% was observed for external controls near the cutoff point [7]. An appropriate QC design would be a 14s rule with 2 control measurements per run, which should provide greater than 90% detection of any medically important analytical error while keeping the false rejections at less than 1%. See the OPSpecs chart below for documentation of this recommendation.
A clinical decision interval of 50% represents the clinical quality requirement for detection of a change from the upper limit of the reference range of 50 ug/L to the cutoff point of 100 ug/L. The expected within-subject biologic variability of 5.6% [6] is accounted for in the clinical quality-planning model. A CV of 7.7% represents the precision performance observed for external controls at the decision level closest to the cutoff point [7]. The appropriate QC design is a 13.5s control rule with 2 control measurements per run, which should provide greater than 90% detection of medically important systematic errors while keeping false rejections at less than 1%. See the OPSpecs chart below for documentation of this recommendation.
A clinical decision interval of 47% represents the clinical quality requirement for detection of a change from the upper limit of the reference range of 3.7 ug/L (applies to the population of both males and females as a whole) and the cutoff point of 7.0 ug/L. The expected within-subject biologic variation is 9.3% [6]. A CV of 6.3% represents the analytical performance on external controls near the cutoff point [7]. The appropriate QC design is a 13.5s control rule with 2 control measurements per run, which should provide greater than 90% detection of medically important systematic errors while keeping false rejections at less than 1%. See the OPSpecs chart below for documentation of this recommendation.
For on-board QC, a simple single-rule control procedure with 4s control limits and 2 control measurements is appropriate for troponin I, myoglobin, and CK-MB. This recommendation is based on clinical quality requirements for each test and the analytical performance observed for the analytical system. This QC design is advantageous for minimizing false rejections and reducing the waste of reagent disks, as well as providing simplicity in implementation.
For external QC, single-rule procedures could again be used for troponin I, myoglobin, and CK-MB. Although 4s control limits would be appropriate for troponin I, it would be more practical to utilize 3.5s control limits for all three of these tests.
The frequency of analyzing external controls should be at least once per week for frequent operators, according to the NCCLS EP-18P guideline [8, page 7]. Given that the minimum QC requirement by CLIA is two levels of control per day, the on-board controls should satisfy the daily QC requirement and the external controls may be used to document operator proficiency.
QC is expected to be easy when process performance achieves the 6-sigma goal! These tests for cardiac markers, as provided by this analyzer, all perform at better than 6-sigma capability. This assessment is based on the medical requirements for interpretation of the tests and the precision performance documented in a published performance evaluation study. This application also shows the benefit of having a Six Sigma tool that encompasses both pre-analytical and analytical factors. While the clinical quality-planning model is more complicated than the analytical model, it greatly expands the potential applications. And it expands the applications on the basis of medically relevant quality or the medical usefulness of a laboratory test.
Here’s a real-world example that demonstrates that it is indeed possible to deliver world class quality with today’s laboratory analyzers, even in a point-of-care application!
- Petersen PH, Fraser CG, Kallner A, Kenny D. Strategies to Set Global Analytical Quality Specifications in Laboratory Medicine. Scand J Clin Lab Invest 1999;59:475-586.
- Westgard JO. The need for a system of quality standards for modern quality management. Scand J Clin Lab Invest 1999;59:483-486.
- Kenny D, Fraser CG, Petersen PH, Kallner A. Consensus agreement. Scand J Clin Lab Invest 1999;59:585.
- Apple FS, Wu AHB. Myocardial infarction redefined: Role of cardiac Troponin testing. Clin Chem 2001;47:377-379.
- Wu AHB, Apple FS, Gibler WB, Jess RL, Warshaw MM, Waldes R. National Academy of Clinical Biochemistry Standards of Laboratory Practice: Recommendations for the use of cardiac markers in coronary artery diseases. Clin Chem 1999;45:1104-1121.
- Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. Current databases on biological variation: pros, cons, and progress. Scand J Clin Lab Invest 1999;59:497.
- Apple FS, Anderson FP, Collinson P, Jesse RL, Kontos MC, Levitt MA, Miller EA, Murakami MM. Clinical evaluation of the First Medical whole blood, point-of-care testing device for detection of myocardial infarction. Clin Chem 2000;46:1604-1609.
- NCCLS. Quality Management for Unit-Use Testing: Proposed Guideline. NCCLS Document EP18-P. NCCLS, 940 West Valley Road, Suite 1400, Wayne, PA 19087 USA, 1999.
