Desirable Laboratory Precision |
|
Desirable Laboratory Precision |
|
An updated version of this article is available in Six Sigma QC Design and Control
If you haven't heard about it yet, there's a good chance you'll hear about it soon. "Six Sigma Quality Management" is the latest buzz coming from industry.
Six Sigma is the next wave in quality management and you'll be able to catch it soon at a bookstore near you. One of the first books aimed at the popular business and management audience is Six Sigma: The breakthrough management strategy revolutionizing the world's top corporations [1], written by Mikel Harry and Richard Schroeder, former Motorola employees who founded the Six Sigma Academy. The American Society for Quality (ASQ) has partnered with the Six Sigma Academy to provide training in Six Sigma Quality Management. The December 2000 issue of ASQ's Quality Progress features an advertorial theme on Six Sigma and a directory of companies that provide Six Sigma training. A year from now, you won't be able to avoid talking about Six Sigma if you want to be considered a knowledgeable manager.
It would be tempting to suggest that "this too will pass," in the same manner that other management programs come and go. Witness the changes in JCAHO's quality management programs from TQM to CQI to IOP and ORYX. In the time it takes to implement a new program, the program is often replaced by something new and apparently different.
The truth of the matter is that the names of the programs are often changed to reflect a change in emphasis, not a change in the overall principles. All these programs have evolved from the quality management philosophy espoused by Deming, Juran, and others. The names are usually important in marketing the program.
- Total Quality Management emphasizes a broad perspective for quality management, identifies the importance of customers and understanding their needs or requirements, and focuses on processes as the critical mechanism for producing the desired quality.
- Continuous Quality Improvement emphasizes that quality isn't static but needs to be improved on an on-going basis. It provides problem solving methodology to support the identification and resolution of chronic problems, particularly those that occur across departments. Teamwork and group problem solving are important elements.
- Improving Organizational Performance emphasizes that quality management applies not only to work processes, but also to management processes that determine how the various parts of an organization work together to deliver quality to the customers.
- ORYX emphasizes outcome measures that will help an organization measure how well they're doing. The name ORYX has no practical meaning in relation to the underlying principles and concepts of quality management; it reflects only a marketing faux pas (pardon my French).
Now comes Six Sigma Quality Management. This is, in fact, a new marketing approach for TQM! There's a lot of hoopla about Champions, Master Black Belts, Black Belts, and Green Belts to describe the training and implementation structure [1]. However, the emphasis on Six Sigma is also truly important for making quality management a more quantitative science. Six Sigma provides a quantitative definition of the desired specifications for production processes and allows those specifications to be related to the customer needs or requirements. When six-sigma performance is recognized as a fundamental goal for processes, quality can truly be measured and managed in a more quantitative way.
The Six Sigma concept was the backbone of Motorola's quality management strategy in the 1980s. The idea was to develop manufacturing processes that were so good that virtually no defective products would be produced. So good was defined as having six sigmas of process variation fit within the product "tolerances," as illustrated in the accompanying figure.
Assuming a gaussian distribution for the variation of a process, the area in the tails of the distribution can be used to estimate the expected defects. For example, if the product specifications enclose plus/minus 2 SDs, the area in the tails would correspond to a 4.5% defect rate or 45,400 defects per million (DPM). The 4.5% figure doesn't sound so bad, but 45,400 DPM doesn't sound very good. For plus/minus 3SDs, the defect rate would be less than 0.27% or 2,700 DPM; for plus/minus 4SDs, the defect rate would be 0.0063% or 63 DPM; for plus/minus 5SDs, the defect rate would be only 0.57 DPM; and with plus/minus 6 SDs, the defect rate would be only 0.002 DPM [see reference 1, page 145].
There would seem to be little to be gained from improving process performance beyond five sigma, however, the advantage is that small shifts in the process mean can actually be tolerated without increasing the defect rate significantly. As shown in the next figure, a shift or bias of 1.5 sigma would hardly cause any defects in a six sigma process. The actual rates that are expected are as follows [from reference 1, page 145]:
- 3.4 DPM for a six-sigma process;
- 233 DPM for a five-sigma process;
- 6210 DPM for a four-sigma process;
- 66,807 DPM for three-sigma; and
- 308,537 DPM for a two-sigma process.
Since shifts or biases equivalent to 1.5s are difficult to detect by statistical QC, a six-sigma process provides a better guarantee that products will be produced within the desired specifications and with a low defect rate.
Another way of looking at this is that a six-sigma process can be monitored with any QC procedure, e.g., with 3 SD limits and low N, and any important problems or errors will be detected and can be corrected. As process capability decreases to five-sigma to four-sigma to three-sigma, the choice of QC procedure becomes more and more important in order to detect important problems. Processes with lower capability may not even be controllable to a defined level of quality!
Process capability is an industrial term that characterizes how the tolerance specification of a product relates to the centering (bias) and variation (standard deviation, SD or s) of the process. High capability means that the process can readily produce a product within the tolerance specifications. Low capability means that the process will likely produce products outside the tolerance specifications (i.e., defective products or defects).
One common measure of process capability is called Cpk [2], which is calculated as Cpk = (Tolerance specification - bias)/3SD.
- If the tolerance specification were 12%, SD 2%, and Bias 0.0%, Cpk would be 2.00, which is considered the ideal capability, i.e., a six-sigma process because six multiples of the SD fit within the tolerance specification.
- If the tolerance were 12%, SD 4%, and Bias 0.0%, Cpk would be 1.00, which is considered the minimum capability for a production process and corresponds to a three-sigma process.
- If the tolerance were 12%, SD 2%, and Bias 3.0%, Cpk would be 1.50. Although this initially starts out as a six-sigma process when there is no bias, the effect of a bias of 1.5 sigma actually reduces the process capability and makes this equivalent to a four-point-five process. This would still be considered a good production process if adequately controlled, but it would still be desirable to eliminate the bias if possible.
This latter situation is illustrated in the accompanying figure, where the tolerance specification is replaced by a Total Error specification, which is a common form of a quality specification for a laboratory test. For example, the CLIA criteria for acceptable performance in proficiency testing events are given in the form of an allowable total error, TEa, thus there is a published list of TEa specifications for regulated analytes. In terms of TEa, Six Sigma Quality Management sets a precision goal of TEa/6 and an accuracy goal of 1.5(TEa/6) or TEa/4. In terms of the industrial process capability, , the combination of the six-sigma precision and accuracy goals results results in a Cpk of 1.5.
Laboratories evaluate process capability when they perform method validation studies. Although they don't calculate an index such as Cpk, they do combine the effects of inaccuracy and imprecision for comparison with the allowable total error. Commonly used TE criteria include TEa > bias + 4SD, TEa > bias + 3SD, and TEa > bias + 2SD, all of which are used on a decision-making tool called the Method Decision Chart [3].
- If the criterion requires that TEa > bias + 4SD, this corresponds to a four-sigma process if there is no bias, e.g., if TEa is 12%, bias is 0%, and SD is 3%, Cpk would be 1.33, which is a good production process that should be controllable to the desired quality.
- If the criterion requires that TEa > bias + 3SD, this corresponds to a three-sigma process if there is no bias, e.g., if TEa is 12%, bias is 0%, and SD is 4%, Cpk would be 1.00, which is the minimal capability needed for a production process.
- If the criterion requires that TEa > bias + 2SD, this corresponds to a two-sigma process if there is no bias, e.g., if TEa is 12%, bias is 0%, and SD is 6%, Cpk would be 0.67, which is unacceptable for production according to industrial guidelines.
Process performance, as evaluated by commonly used laboratory TE criteria, does not approach the six-sigma capability desired for industrial processes. Improvements in laboratory methods are still needed to achieve five-sigma to six-sigma capability.
CLIA has defined the acceptability criteria for performance in proficiency testing surveys for approximately 80 regulated analytes. These criteria are often thought to be "loose" and not very demanding for analytical performance, but that conclusion is based on a goal of two-sigma to three-sigma process capability. If the goal were to establish five-sigma and six-sigma processes, improvements in precision would be needed for many tests today [4].
- Cholesterol has a 10% tolerance specification, as set by the CLIA criterion for acceptable performance in proficiency testing events. A five-sigma process should have a CV of 2.0%. A six-sigma process should have a CV of 1.7%. These performance specifications are considerably more demanding than the NCEP's 3.0% specification for imprecision [5]. If the NCEP allowance for a 3.0% bias is also considered, the NCEP process would have a Cpk of only 0.78 [10-3)/3*3], which corresponds to little better than a two-sigma process capability (actually 2.3).
This is rather dismal performance for a national performance standard! We have pointed this out repeatedly in discussions in the scientific literature [6-8,10], however, the NCEP recommendations are steadfastly defended by our public health service [9,11]. I hope Six Sigma Quality Management brings them to their senses, but I suspect it will take something even more rigorous and quantitative, like an act of congress!
The following table shows some of the CLIA criteria and the corresponding precision that would be needed to establish five sigma and six-sigma processes for common chemistry, hematology, and toxicology tests. These figures should be useful for evaluating the current performance of laboratory methods. Six Sigma Quality Management sets demanding standards of performance for laboratory testing processes.
Chemistry
Test or AnalyteCLIA Acceptable
PerformanceFive-Sigma
PrecisionSix-Sigma
PrecisionALT 20% 4.0% 3.3% Albumin 10% 2.0% 1.7% Alkaline Phosphatase 30% 6.0% 5.0% Amylase 30% 6.0% 5.0% Bilirubin, total 0.4 mg/dL
or 20% (greater)0.08 mg/dL
or 4%0.067 mg/dL
or 3.3%Blood gas pCO2 5 mm Hg
or 8% (greater)1 mm Hg
or 1.6%0.8 mm Hg
or 1.3%Blood gas pH 0.04 pH units 0.008 pH units 0.00067 pH units Calcium, total 1.0 mg/dL 0.2 mg/dL 0.17 mg/dL Chloride 5% 1.0% 0.83% Cholesterol, total 10% 2.0% 1.7% Cholesterol, HDL 30% 6.0% 5.0% Creatine kinase 30% 6.0% 5.0% Creatinine 0.3 mg/dl
or 15% (greater)0.06 mg/dL
or 3.0%0.05 mg/dL
or 2.5%Glucose 6 mg/dL
or 10% (greater)1.2 mg/dL
or 2.0%1.0 mg/dL
or 1.7%Iron, total 20% 4.0% 3.3% LDH 20% 4.0% 3.3% Magnesium 25% 5.0% 4.2% Potassium 0.5 mmol/L 0.1 mmol/L 0.08 mmol/L Sodium 4 mmol/L 0.8 mmol/L 0.67 mmol/L Total protein 10% 2.0% 1.7% Urea Nitrogen 2 mg/dL
or 9% (greater)0.4 mg/dL
or 1.8%0.33 mg/dL
or 1.5%Uric acid 17% 3.4% 2.8% Toxicology Test or Analyte Alcohol, blood 25% 5.0% 4.2% Blood lead 10% or
4 mcg/dL (greater)2.0% or
0.8 mcg/dL1.7% or
0.67 mcg/dLCarbamazepine 25% 5.0% 4.2% Digoxin 20% or
0.2 ng/mL (greater)4.0% or
0.04 ng/mL3.3% or
0.033 ng/mLEthosuximide 20% 4.0% 3.3% Gentamicin 20% 4.0% 3.3% Lithium 0.3 mmol/L or
20% (greater)0.06 mmol/L or
4.0%0.05 mmol/L or
3.3%Phenobarbital 20% 4.0% 3.3% Phenytoin 25% 5.0% 4.2% Primidone 25% 5.0% 4.2% Procainamide 25% 5.0% 4.2% Quinidine 25% 5.0% 4.2% Theophylline 25% 5.0% 4.2% Tobramycin 25% 5.0% 4.2% Valproic acid 25% 5.0% 4.2% Hematology Test or Analyte Erythrocyte count 6% 1.2% 1.0% Hematocrit 6% 1.2% 1.0% Hemoglobin 7% 1.4% 1.2% Leukocyte count 15% 3.0% 2.5% Platelet count 25% 5.0% 4.2% Fibrinogen 25% 5.0% 4.2% Partial thromboplastin time 15% 3.0% 2.5% Prothrombin time 15% 3.0% 2.5%
- Harry M, Schroeder R. Six Sigma The Breakthrough Management Strategy Revolutionizing the World's Top Corporations. A Currency Book, Published by Doubleday. New York, 2000.
- Chesher D, Burnett L. Equivalence of critical error calculations and process capability index Cpk. Clin Chem 1997;43:1100-1101.
- Westgard JO. Basic Method Validation. Chapter 12: The Decision on Method Performance in Basic Method Validation. Madison, WI: Westgard QC, Inc., 1999, pages 125-134.
- Westgard JO, Burnett RW. Precision requirements for cost-effective operation of analytical processes. Clin Chem 1990;36:1629-1632.
- National Cholesterol Education Program Standardization Panel. Current status of blood cholesterol measurements in clinical laboratories in the United States. Clin Chem 1988;34:193-201.
- Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory process specifications for assuring the quality in the US National Cholesterol Education Program. Clin Chem 1991;37:656-661.
- Wiebe DA, Westgard JO. Cholesterol - a model system to relate medical needs with analytical performance. Clin Chem 1993;39:1504-1513.
- Fallest-Strobel PC, Olafsdottir E, Wiebe DA, Westgard JO. Comparison of NCEP performance specifications for triglycerides, HDL-, and LCL-cholesterol with operating specifications based on NCEP clinical and analytical goals. Clin Chem 1997;43:2164-2168.
- Caudill SP, Smith SJ, Cooper GR, Myers GL. Adequacy of NCEP recommendations for total cholesterol, triglycerides, HDLC, and LDLC measurements. Clin Chem 1998;44:1063-1064.
- Westgard JO, Wiebe DA. Response to letter above. Clin Chem 1998;44:1064-1065.
- Caudill SP, Cooper GR, Smith SJ, Myers GL. Assessment of current National Cholesterol Education Guidelines for total cholesterol, triglyceride, HDL-cholesterol, and LDL-cholesterol measurements. Clin Chem 1998;44:1650-1658.
Other Essays by Dr. Westgard:
