A Six Sigma Design Tool

James O. Westgard, Ph.D.

The OPSpecs chart is an ideal Six Sigma assessment and design tool for laboratory testing processes, whether your interest is to establish method performance specifications for imprecision and inaccuracy or to establish appropriate statistical control rules and numbers of control measurements. The next lessons provide a more detailed description of the OPSpecs Chart and a justification for its use as a Six Sigma planning tool.

Some readers may want to skip this material, trusting that the documentation in the peer-reviewed scientific literature provides a sufficient validation of the tool [1-4]. Others may want to see practical applications rather than a theoretical justification. Again, there are peer-reviewed papers that describe how OPSpecs charts can be used to select QC procedures for immunoassays [5], blood gas measurements [6], and lipid tests [7]; evaluate the "state of the art" precision performance [8]; assess the imprecision required by clinical and analytical test outcome criteria [9]; compare the precision required by European biologic goals and US CLIA total error criteria [10]; and even design patient data techniques, such as Average of Normals (AoN) algorithm, for use to measure the length of the analytical run [11]. There are also more detailed applications available in the archives.

For those who want the whole story on how and why OPSpecs charts work, read on! I'll first demonstrate the graphical characteristics or attributes of the OPSpecs chart and how this tool can characterize process performance and also represent the error detection capabilities of different QC procedures. The next lessons will provide new details and explanations:

Feel free to pursue this discussion of "tools and technology" as far as you want. Also feel free to skip to the applications when you need a break from this material, understand the mechanics of using this tool, and are satisfied the OPSpecs chart is a valid design tool for applications in Six Sigma Quality Management.

Graphical display of process performance

The Six Sigma goal for performance is a metric of 6 for the tolerance limit divided by the variation or imprecision of the process. Here's a graph for a process having a tolerance specification, or allowable total error, of 12%.


The y-axis is scaled from 0 to 12% and the x-axis is scaled from 0 to 6%. The y-axis describes the inaccuracy or bias of the testing method in units of percent. The x-axis describes the imprecision or CV of the testing method in units of percent. This graph can be used to describe process performance, or process capability, by drawing the following lines.

Here's what the chart should look like with all these lines.

We now have a graphical tool for relating process performance or process capability in terms of a sigma metric to the bias and CV of an analytical testing process. Here's how it works. You plot the bias and CV of your method, then determine the sigma metric from the location of the point relative to the lines. Here are some examples along with the sigma metric calculated from the tolerance limit minus the method bias, then dividing that quantity by the method CV [metric = (TEa - bias)/CV].

Observe the points on the accompanying graph and compare the graphic sigma value with the calculated sigma capability metric above.

Note that the graphical estimate also properly considers the centering of the distribution (which is related to the method bias) and the width of the distribution (which is related to the method CV). We have a simple graphical tool that can directly relate the precision and accuracy of an analytical method to process capability.

Attributes related to process capability

The graphical demonstration reveals several attributes of the OPSpecs design tool.

Graphical assessment of process performance

World class quality is generally recognized as a 6 sigma process. The initial goal in Six Sigma Quality Management is to bring processes up to 5 sigma capability. In general, production process in use today tend to be about 4 sigma. The minimum performance needed for a process to be considered for routine production or operation is usually set as 3 sigma. The following figure shows the graphical representation of world class quality, the initial 5 sigma performance goal for process improvement, the 4 sigma "state of the art" performance of current production processes, and the 3 sigma performance that represents the minimum standard for a production process and defines the region of unacceptable performance.


All these recommendations can be used to classify method performance and judge the acceptability of new methods for routine operation. The graphical display of these criteria is consistent with the format of the Method Decision Chart developed earlier for use in method validation studies [12,13]. However, the Six Sigma criteria are more demanding and set new standards for method performance. In the 1970s, the criterion for acceptable performance of analytical methods was set at 2 sigma [14], increased to 3 sigma to 4 sigma in the 1990s [15,16], and now are recommended as 5 to 6 sigma.

Performance characteristics of statistical QC

Six Sigma Quality Management recommends the use of a statistical control chart to monitor process performance and hold the gains. In practice, that means that the control chart needs to identify situations where performance is deteriorating and corrective action is necessary. In healthcare laboratories, the control chart commonly employed is a single-value chart where the measured values from stable control samples are plotted directly. This technique is commonly called a Levey-Jennings chart after the pathologists who first introduced quality control in clinical laboratories in the 1950s [17]. Control limits are commonly set at the mean plus/minus 3SD or the mean plus/minus 2SD. Multi-criteria or multi-rule charts [18] are also employed and may include many of the following decision criteria or control rules:

Intuitively, we know that these different control rules must respond differently to changes in process performance. Wide control limits will require a bigger change than narrow limits, therefore suggesting lower sensitivity or lower error detection when wide limits are used. Narrow control limits, on the other hand, may be too sensitive to the inherent variation or stable imprecision of a testing process, suggesting there may be occasional rejections even when process performance is stable. Consecutive control measurements on one side of a control limit would be sensitive to a change in location of the distribution, i.e., systematic errors rather than random errors. The range between the high and low values in a group of control measurements would be sensitive to the width of the distribution, i.e., random error rather than systematic error. It is also intuitive that sensitivity should increase as the number of control measurements increases, thus the performance of a QC procedure depends both on the control rules used to inspect the data and the amount of data available for inspection.

The actual characteristics that describe the performance of a statistical QC procedure are the probabilities for rejection for two situations. The first considers rejections when method performance is stable, in which case these are false rejections or false alarms. The second considers rejections when there is a real change or error, which is described as error detection or true alarms. Ideally the probability for error detection should 1.00 or 100%; ideally the probability for false rejection should be 0.00 or 0%. In practice, it is more practical to set goals of achieving 90% error detection and maintaining false rejections less than 5%.

Graphical display of QC performance

The capability of a QC procedure to detect medically important errors depends on the tolerance limit or quality requirement for the test, the inherent imprecision (CV) and stable inaccuracy (bias) of the method, and sensitivity of the control rules and number of control measurements being used. Typically, with today's highly automated analytical system, we're most concerned with detecting systematic shifts, or accuracy problems. This QC capability can be shown graphically on an OPSpecs chart by a line that defines the allowable bias and allowable inaccuracy for the particular control rules and number of control measurements for the conditions where there will be a probability of 0.90 or a 90% chance of detecting a medically important systematic error.

The Six Sigma QC guideline is to use a control chart having 3 standard deviation (SD, s) control limits. Therefore, let's consider the capability of a Levey-Jenning's control chart having 3 SD limits.


In this figure, the lowest line shows an almost perfect match between the error detection capabilities of a 13s control rule with N=1 and a process that achieves 6 sigma performance. This line represents conditions where there is a 90% chance of detecting a medically important systematic error. A medically important systematic error is defined as a shift in the distribution sufficient to cause 5% of the tail to exceed the tolerance limit or quality requirement, which corresponds to rejecting an analytical run when the rate of defective results reaches 5%. A more quantitative and mathematical explanation of this line will be provided in the next chapters. For now, it's only important to understand what the line represents - the allowable range of bias and CV for which the QC procedure can provide a 90% guarantee of detecting systematic errors that would cause a maximum defect rate of 5%.

The next figure shows the effect of increasing N from 1 to 8, which would provide appropriate control for processes with performance from 6 sigma to 4 sigma (the line whose x-intercept is 3 units).



Intuitively, we know that increasing the number of control measurements increases the error detection capability, which means smaller errors can be detected. As process capability decreases, smaller errors will become medically important and the QC procedure needs to be adjusted appropriately, in this example by increasing N. Note that testing processes whose performance capabilities are between 4 sigma and 3 sigma can not be adequately controlled by 3 SD control limits with N up to 8.
Analytical processes with capability as low as 3 sigma might be considered acceptable methods in some laboratories, therefore there is a need for more sensitive QC procedures. One approach is to use narrower control limits, such as the 12.5s control rule. Another approach is to use multi-rule procedures that improve error detection by using a series of control rules.



This figure shows a wider range of controllable methods from 6 sigma to approximately 3.3 sigma using single-rule and multi-rule procedures with Ns up to 6. Multi-rule procedures with Ns of 8 are capable of detecting medically important errors in processes with 3 sigma capability. For s process whose capability is less than 3 sigma, the real issue is not to do more QC, but rather to make improvements that will reduce the bias and/or CV of the method.

Attributes related to QC capability or performance

This graphical demonstration reveals additional attributes of the OPSpecs chart that are useful in matching QC capability with process performance.

Summary of the attributes of an OPSpecs Chart

The OPSpecs chart shows the relationship between the tolerance limit or quality requirement for a test, the precision (CV) and accuracy (bias) of an analytical method or testing process, and the error detection capability (probability for rejection) for the control rules and numbers of control measurements in the QC procedure. Additional information is needed about the QC procedure's false rejection rate or probability of false rejections, but that can be included in the key of the chart along with the identification of the control rules and the number of control measurements. OPSpecs charts can also be prepared for clinical quality requirements, as will be discussed in later chapters.

References

  1. Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program. Clin Chem 1991;37:656-61.
  2. Westgard JO, Wiebe DA. Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing. Clin Chem 1991;37:1938-44.
  3. Westgard JO. Assuring analytical quality through process planning and quality control. Arch Path Lab Med 1992;116:765-769.
  4. Westgard JO. Charts of operating specifications (OPSpecs charts) for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing criteria. Clin Chem 1992;38:1226-33.
  5. Mugan K, Carlson IH, Westgard JO. Planning QC procedures for immunoassays. J Clin Immunoassay 1994;17:216-22.
  6. Olafsdottir E, Westgard JO, Ehrmeyer SS, Fallon KD. Matrix effects on the performance and selection of QC procedures to monitor PO2 in blood gas measurements. Clin Chem 1996;42:392-6.
  7. Fallest-Strobl PC, Olafsdottir E, Wiebe DA, Westgard JO. Comparison of NCEP performance specifications for triglycerides, HDL, and LDL cholesterol with operating specifications based on NCEP clinical and analytical goals. Clin Chem 1997:43:2164-2168.
  8. Westgard JO, Bawa N, Ross JW, Lawson NS. Laboratory precision performance: State of the art versus operating specifications that assure the analytical quality required by proficiency testing criteria. Arch Path Lab Med 1996;120:621-625.
  9. Westgard JO, Seehafer JJ, Barry PL. Allowable imprecision for laboratory tests based on clinical and analytical test outcome criteria. Clin Chem 1994;40;1909-14.
  10. Westgard JO, Seehafer JJ, Barry PL. European specifications for imprecision and inaccuracy compared with operating specifications that assure the quality required by U.S. CLIA proficiency testing criteria. Clin Chem 1994;40:1228-32.
  11. Westgard JO, Smith FA, Mountain PJ, Boss S. Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control. Clin Chem 1996;42:1683-1688.
  12. Westgard JO. A method evaluation decision chart (MEDx Chart) for judging method performance. Clin Lab Science 1995;8:277-283.
  13. Westgard JO. Basic Method Validation. Madison, WI:Westgard QC, Inc., 1999, 125-134.
  14. Westgard JO, Carey RN, Wold S. Criteria for judging precision and accuracy in method development and evaluation. Clin Chem 1974;20:825-833.
  15. Ehrmeyer SS, Laessig RH, Leinweber JE, Oryall JE. 1990 Medicare/CLIA final rules for proficiency testing: Minimum interlaboratory performance characteristics (CV andBias) needed to pass. Clin Chem 1990;36:1736-1740.
  16. Westgard JO, Burnett RW. Precision requirements for cost-effective operation of analytical processes. Clin Chem 1990;36:1629-1632.
  17. Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J Clin Pathol 1950;20:1059.
  18. Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981;27:493-501.

Other Six Sigma articles:

Six Sigma DPM Table
Six Sigma Quality Management
Six Sigma and Requisite Laboratory QC
Six Sigma Basics: Process improvement, goals, and measurements
Six Sigma: Outcome measurement of process performance
Six Sigma: Quality Design and Control Processes