Six Sigma: Quality Design and Control Processes Analytical quality - an essential characteristic Quality requirements Analytical quality requirements Clinical quality requirements Quality design and control applications Design process for method specifications Design process for QC specifications Tools and technology References

James O. Westgard, Ph.D.

It is a basic principle of Six Sigma Quality Management that well-designed processes are critical to achieving quality. It also follows that there must be a design process that has been carefully structured to plan the quality of new products or services. One well-established approach to process and product design is called Quality Function Deployment (QFD) [1]. QFD is often described as process for translating customer needs into product and production specifications.

The idea of translation suggests the difficulty of understanding customer needs and relating them to product and process specifications. Customers and producers seldom speak the same language, therefore the needs of the customer require interpretation in order to become specifications for products and production processes. This is clearly true when discussing the analytical quality of laboratory tests. Physicians and patients want correct results. Manufacturers are concerned with the imprecision and inaccuracy of their methods and systems. Laboratory analysts are concerned with quality control, as well as the imprecision and inaccuracy of the method. The customers' needs for correct results must be translated into specifications for the imprecision and inaccuracy of a method, as well as the quality control rules and the numbers of control measurements needed to monitor the testing process in a laboratory [2].

Analytical quality - an essential characteristic

Quality design in a laboratory must begin with analytical quality because it is the essential quality characteristic of any laboratory test. It's not the only quality characteristic, but unless analytical quality can be achieved, none of the other characteristics matter. For example, turnaround time is an important quality characteristic, but it doesn't matter how fast the result is reported if the result is wrong. The laboratory must first be able to produce a correct test result before any other quality characteristic becomes important.

A detailed step-by-step planning process is needed to properly consider the critical factors that affect the quality of laboratory test results. Analytical quality is a particularly complex characteristic, involving the imprecision, inaccuracy, and instability of a measurement procedure, as well as the error detection and false rejection characteristics of a statistical QC procedure. Six Sigma texts seldom treat complex characteristics like analytical quality, which is the reason for providing a detailed quality design and control process here.

When focusing on quality design and control for laboratory applications, people will have different interests related to their own jobs.

• If you are a laboratory director, manager, or quality coordinator, your first interest may be to satisfy regulatory and accreditation guidelines for planning your testing processes. A rigorous and well-documented process will be important. Selecting control rules and numbers of control measurements is, of course, an important application in a service laboratory. In addition, performance specifications for methods can also be determined for preferred QC procedures, which should be useful in establishing purchase specifications for methods, instruments, and systems.
• If you are a manufacturer, your interest may be in establishing performance specifications for the precision and accuracy by a quantitative design process. You may also need to develop QC instructions that take into account regulatory quality requirements, the design specifications for imprecision and inaccuracy, and the common QC practices in the marketplace.
• If you are a regulatory analyst, you may need to assess the practicality of proposed proficiency testing criteria by comparison of analytical and clinical requirements for quality. Manufacturer's QC product labeling should be reviewed to assess whether those QC instructions are valid for the intended use. Laboratory QC practices should be reviewed to assess whether they are valid to assure the quality needed for the patient populations and clinical applications in a healthcare organization.

Quality Requirements

The starting point for quality design must be the definition of the tolerance limits for the testing process. In laboratory terms, we will refer to the tolerance limits as the quality requirement for the test. A debate over the best type of quality requirement has been going on for the last twenty years and, unfortunately, has overshadowed the use and application of quality requirements. Finally, in 1999, an international conference in Stockholm led to consensus agreement on a hierarchy or system of quality standards [3]. This hierarchy includes different sources of information, such as studies of diagnostic classification, data on biologic variation, recommendations by expert groups, regulatory requirements for proficiency testing, and summaries of "state of the art" performance. The system includes different formats for requirements, such as the allowable total error (analytical outcome criterion), the clinical decision interval (clinical outcome criterion), or the maximum allowable standard deviation and the maximum allowable bias (analytical performance criteria).

The accompanying figure shows my view of the relationships between these different sources of information, different types of quality requirements, and the operating specifications needed for managing routine testing processes [4]. Starting at the top of the figure, medically important changes in test results can be defined by patient treatment guidelines (clinical pathways, clinical practice guidelines, etc.) to establish clinical outcome criteria (or decision intervals, D_int). Such clinical criteria can be converted to laboratory operating specifications for imprecision (s_meas), inaccuracy (bias_meas), and QC (control rules, N) by a clinical quality-planning model [5] that takes into account pre-analytical factors, such as group (s_biol) or within-subject biologic variation (s_wsub).

The left side of the figure shows how performance criteria for imprecision and inaccuracy can be defined as separate analytical goals for the maximum imprecision and bias that would be allowable for the stable performance of the method. Specifications for maximum imprecision and bias can be derived on the basis of within-subject biological variation [6]. The maximum allowable bias can also be derived from diagnostic classification models [7]. Laboratories can utilize these separate performance criteria by relating observed method performance to the maximum allowable value, calculating the critical-size error that needs to be detected to maintain satisfactory performance, and then selecting appropriate QC procedures by use of power function graphs.

The right side of the figure shows how proficiency testing criteria define analytical outcome criteria in the form of allowable total errors (TE_a), which can be translated into operating specifications (s_meas, bias_meas, control rules, N) using an analytical quality-planning model [8]. Note that the allowable total error can also be established on the basis of total biologic goals that are based on population variation or individual variation [9], therefore an extensive data-bank of individual biologic variation is available for use in calculating an allowable biologic total error [10].

Operating specifications are the bottom line in this system of quality standards. Both clinical and analytical quality requirements, i.e., decision intervals and allowable total errors, resp., can be translated into the practical specifications that are needed to manage routine operations. These operating specifications consist of the imprecision and inaccuracy that are allowable for the method and the control rules and number of control measurements that are necessary to monitor and assure the quality of the testing process. The exact values for the CV, bias, control rules and N are interdependent, permitting many different combinations that will still assure the desired quality will be achieved. The many possible combinations can be shown graphically by OPSpecs charts to help analysts and managers determine how to properly manage the analytical quality of a testing process.

All these different forms of quality standards have some use in the context of a system for analytical quality management. However, until this system is recognized, understood, and applied, the different recommendations in the literature will continue to be incoherent, rather than useful and practical for analytical quality management.

Analytical quality requirements

A statement of an allowable total error most closely represents the industrial tolerance limits for a production process. It considers both inaccuracy (the centering of the process on a target value) and imprecision (the distribution of individual products around that target). The most common sources of these type of requirements are the proficiency testing or external quality assessment programs that specify acceptability limits in the form of a target value plus/minus certain tolerances. In the US, CLIA defines such limits for approximately 80 different tests [11]. In other countries, such as Australia and Canada, the lists and criteria may be even more extensive. These PT limits define minimum levels of quality that must be achieved, therefore, it is always important to plan testing process that will assure PT criteria are achieved in routine operation. This can be accomplished by using an analytical quality-planning model that translates these requirements into the imprecision and inaccuracy that are allowable and the QC that is necessary [8].

A list of the allowable total errors, or proficiency testing criteria for acceptable performance, that have been defined by the US Clinical Laboratory Improvement Amendment. Total error requirements can also be calculated from biologic goals, in the manner recommended by Petersen et al [9]. A data-bank of biologic goals from Ricos [10], provides the calculated biologic total errors for over 300 quantities.

Clinical quality requirements

Practical information can be provided in the form of a medically important change, medically significant change, or clinical decision interval, which are the commonly used terms for this type of quality requirement. One major advantage of this type of quality requirement is that information is directly available from the customers through descriptions of how they use and interpret a laboratory test, through clinical pathways that detail the expected use and interpretation of tests, or through audits of clinical practices. When this information is properly translated to operating specification via a quality-planning model that accounts for pre-analytical factors, it provides a useful and valid approach for defining and managing the quality of the testing process.

One source of information about medically important changes in test values is a paper by Skendzel, Barnett, and Platt [12]. The important information is found in Table 1, which is a summary of physicians' opinions of a significant change in test results. This paper is sometimes criticized for the rather large values recommended for medically useful CVs, which appear in Table 2 and were derived without accounting for within-subject biological variation. When Fraser's figures for within-subject biological variation [13,14] are used in a clinical quality-planning model that accounts for biological variation, the allowable CVs are much smaller [15]. The approach of defining clinical quality in terms of a medically important change is very useful and is valid if the quality-planning model properly accounts for pre-analytical variation, which may be the biologic group variation or the biologic within subject variation, depending on the particular application.

As a starting point for describing the clinical decision interval type of quality requirement, we provide a summary of Skendzel's recommendations, along with Fraser's figures for biologic variation. Later applications will illustrate how to use clinical interpretation guidelines to define the decision interval type of quality requirement.

Quality Design and Control Applications

The main applications involve either the (a) establishment of method
performance specifications for imprecision and inaccuracy, or the selection of a method of measurement in a laboratory, and (b) the establishment of QC
specifications, or the selection of control rules and numbers of control
measurements needed to assure quality in routine operation in a laboratory. Then, as shown in the accompanying figure, there are two variations of the
design process, depending on whether the purpose is method design or QC
design.

• To set performance specifications for a method, the design process
  involves specifying the QC procedure (statistical control rules, number of control measurements or N) that will be employed and then setting the developmental or purchase specifications for the imprecision and inaccuracy of the method.
• To set QC specifications, the process involves assessing method performance (imprecision and inaccuracy) and then selecting the statistical control rules and number of control measurements to be used.

The key step in both applications is the use of an appropriate quality-planning tool that will translate the defined quality requirement into specifications for the imprecision and inaccuracy that are allowable and the QC that is necessary. A chart of operating specifications (or OPSpecs chart) is the most practical tool because it provides all of the necessary information on a single graph [16]. An analytical quality-planning model is available to translate an allowable total error requirement into the imprecision and inaccuracy that are allowable and the QC that is necessary [8]. A clinical model is available that accounts for pre-analytical factors, such as within-subject biologic variation, as well as the analytical factors - imprecision, inaccuracy, and QC [5]. Results from both models can be graphically displayed in the OPSpecs format to show the relationship between a defined quality requirement and the allowable bias (y-axis) and the allowable CV (x-axis) for different quality control procedures. The ability to consider a clinical quality requirement and specific performance characteristics of statistical different QC procedures makes this quality design and control approach much more powerful than the general Six Sigma goal for process performance and the general Six Sigma guideline for quality control.

Design Process for Method Specifications

Setting process specifications seems simple enough with Six Sigma - simply divide the tolerance limit or allowable total error by 6. However, that approach is limited to applications where the quality requirement can be defined as an analytical total error. By using the OPSpecs tool, it is also possible to employ clinical quality requirements in the form of a medically important change in test results. More exacting precision specifications can be established on the basis of both analytical and clinical quality requirements. That provides a much more flexible and comprehensive approach.

1. Define the quality required for the test. For practical purposes, it is easiest to get started with requirements in the form of an allowable total error, such as specified by proficiency testing or external quality assessment programs. The ability to work with clinical decision intervals makes the approach more widely applicable, though a bit more complicated and dependent on having a computer program to carry out the necessary calculations.
2. Specify QC procedures that are commonly employed in the marketplace of interest.
3. Prepare OPSpecs charts for the quality requirement, the control rules, and the numbers of control measurements of interest. For clinical quality requirements, incorporate biologic variation to account for preanalytic variation.
4. Determine the maximum allowable CV from the x-intercept of the operating lines of the various QC procedures. This provides a precision specification for the condition when method bias is zero.
5. Determine the allowable CV at a specified method bias. Read the x-value that corresponds to a y-value that represents the specified method bias.
6. Set performance specifications for CV and bias.

Design Process for QC Specifications

The Six Sigma guideline for QC is very simple - use 3 SD control limits. However, not all measurement processes will achieve the process capability needed to provide a sigma quality metric of 6 or better. For lower capabilities, it is obvious that more QC is necessary. Because laboratories buy their methods from manufacturers, laboratories must determine the process capability of the methods they purchase and provide QC procedures that are appropriate for the observed method performance.

A good starting point for developing a design process for quality control is the NCCLS guideline for planning statistical QC applications [17]. In the step-by-step planning process described here, QC performance will be characterized by the probabilities of rejecting runs having different sizes of errors, therefore there are two probabilities that are of particular interest:

• Probability of false rejection, i.e., the chance of rejecting a run when there are no errors except for the inherent random error of the method;
• Probability for error detection, i.e., the probability or chance of rejecting a run whether there is an error present in addition to the inherent random error of the method.

An eight-step quality-planning process is shown in the accompanying flowchart. Here's a description of each of the steps:

1. Define the quality required for the test. For practical purposes, it is easiest to get started with requirements in the form of an allowable total error, such as specified by proficiency testing or external quality assessment programs. The ability to work with clinical decision intervals makes the approach more widely applicable, though a bit more complicated and dependent on having a computer program to carry out the necessary calculations.
2. Assess method performance in terms of imprecision and inaccuracy. Here's where method validation experiments are important to provide the initial estimates of imprecision (from a replication experiment) and inaccuracy or bias (from a comparison of methods experiment). Later on, the estimates of imprecision can be obtained from routine QC data and estimates of bias can be obtained from monthly peer comparison data and proficiency testing results.
   
3. Prepare an OPSpecs chart for the quality requirement of interest. Assess QC performance of candidate procedures in terms of the rejection characteristics or power curves. This information is available in the scientific literature for most of the commonly used QC procedures [18].
4. Plot the method's imprecision and bias to determine it's operating point on the OPSpecs chart. Prepare critical error graphs [19] to provide further documentation of the error detection capabilities for the conditions of interest.
5. Evaluate the probabilities of rejection for the operating conditions in the laboratory. In the quality-planning process recommended here, the probabilities for false rejection will be minimized (below 0.05 or 5%) and error detection will be maximized (0.90 or 90% and greater).
6. Select appropriate control rules and the total number of control measurements. With the OPSpecs chart [20], this is accomplished by identifying the QC procedures whose lines above the operating point of the method. A wide variety of control rules are available. The rejection characteristics of each QC procedure must be known if it is to be a candidate for implementation. Candidate QC procedures include single-rules such as 1_2s, 1_2.5s, 1_3s, and 1_3.5s with Ns of 2, 3, 4, and 6; multirules such as 1_3s/2_2s/R_4s/4_1s/8_x with Ns of 2 and 4 and 1_3s/2of3_2s/R_4s/3_1s/6_x with Ns of 3 and 6.
7. Adopt a Total QC strategy that provides an appropriate balance of statistical and non-statistical components. This TQC strategy defines the relative amount of effort expended for statistical QC, instrument function checks, method validation tests, patient data QC, preventive maintenance, and operator training. When HIGH error detection is obtained by statistical QC (i.e., P_ed of 0.90 or 90% detection of medically important errors), the HIGH TQC strategy is to depend on statistical QC and perform the minimum other QC required by regulations, accreditation, and good practice guidelines. When MODERATE error detection is obtained (P_ed between 0.50 and 0.90), the MODERATE TQC strategy is to balance the efforts over all the QC components. When LOW error detection is available (i.e., P_ed less than 0.50), the LOW TQC strategy emphasizes preventive measures because problems can not be detected by statistical QC.
8. Reassess the control rules, N, and TQC strategy when method performance or quality requirements change. Given a quality-planning process that is quick and easy to perform, it can be repeated whenever changes occur or when methods are periodically reviewed.

Tools and Technology

A laboratory's ability to do anything efficiently depends on utilizing tools and technology to facilitate a process. Most laboratory procedures have evolved from an initial qualitative manual method (1st generation) that has then been systematized and made more quantitative with tools such as diluters and photometers, then automated through succeeding generations of technology until complete systems are available that are highly efficient and productive (such as todays 4th and 5th generation chemistry and hematology analyzers). Quality planning, likewise, must evolve from a qualitative manual method to a systematic process that utilizes standard tools and, finally, to a quantitative automated process that is quick and effective.

Concerning OPSpecs charts - the quality-planning tool recommended here, different "generations" are available, as follows:

• Manual from scratch using theoretical models available in the scientific literature with implementation via electronic spreadsheets [5,8];
• Kit form using preprinted charts in workbook form, such as the OPSpecs Manual [21], or using a standard set of "normalized" OPSpecs charts [22];
• Semi-automated using Internet calculation tools, such as the "normalized" OPSpecs calculator;
• Automated using a PC computer program - QC Validator(r) - that prepares OPSpecs charts for both analytical total error requirements and clinical decision interval requirements (version 1.1) and fully automates the selection of QC procedures (version 2.0) [23,24].
• Highly automated using a QC rule selection engine that can be embedded in QC software to support the automatic selection and design of QC procedures.

References

1. King B. Better Designs in Half the Time: Implementing QFD Quality Function Deployment in America. 3rd ed. Methuan ME:Goal/QPC, 1989.
2. Westgard JO, Burnett RW, Bowers GN. Quality Management Science in clinical chemistry: a dynamic framework for continuous improvement of quality. Clin Chem 1990;36:1712-1716.
3. Petersen PH, Fraser CG, Kallner A, Kenny D. Strategies to Set Global Analytial Quality Specifications in Laboratory Medicine. Scan J Clin Lab Invest 1999;59(7).
4. Westgard JO. The need for a system of quality standards for modern quality management. Scand J Clin Lab Invest 1999;59:483-486.
5. Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program. Clin Chem 1991;37:656-61.
6. Fraser CG, Hyltoft Petersen P, Ricos C, Haekel R. Proposed quality specifications for the imprecision and inaccuracy of analytical systems for clinical chemistry. Eur J Clin Chem Clin Biochem 1992;30:311-317.
7. Klee GG. Tolerance limits for short-term analytical bias and analytical imprecision derived from clinical assay specificity. Clin Chem 1993;39:1514-1518.
8. Westgard JO, Wiebe DA. Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing. Clin Chem 1991;37:1938-44.
9. Hyltoft Petersen P, Ricos C, Stockl D, Libeer JC, Baadenhuijsen H, Fraser C, Thienpont L. Proposed guidelines for the internal quality control of analytical results in the medical laboratory. Eur J Clin Chem Clin Biochem 1996;34:983-999.
10. Ricos C, Alvarez V, Cava F, Garcia-Lario JV, Hernandez A, Jimenez CV, Minchinela J, Perich C, Simon M. Current databases on biological variation: pros, cons and progress. Scand J Clin Lab Invest 1999;59:491-500.
11. U.S. Department of Health and Human Services. Medicare, Medicaid and CLIA programs; Regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). Final rule. Fed Regist 1992;57:7002-186.
12. Skendzel LP, Barnett RN, Platt R. Medically useful criteria for analytic performance of laboratory tests. Am J Clin Pathol 1985;83:200-205.
13. Fraser CG. Biological variation in clinical chemistry. An update: collated data, 1988-1991. Arch Pathol Lab med 1992;116:916-923.
14. Fraser CG. The application of theoretical goals based on biological variation data in clinical chemistry. Arch Pathol Lab Med 1988;112:404-415.
15. Westgard JO, Seehafer JJ, Barry PL. Allowable imprecision for laboratory tests based on clinical and analytical test outcome criteria. Clin Chem 1994;40:1909-1914.
16. Westgard JO. Charts of operating specifications (OPSpecs charts) for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing criteria. Clin Chem 1992;38:1226-33.
17. C24-A2. Statistical Quality Control for Quantitative Measurements: Principles and Definitions; Approved Guideline - Second Edition. National Committee for Clinical Laboratory Standards, Wayne, PA, 1999.
18. Westgard JO, Groth T. Power functions for statistical quality control rules. Clin Chem 1979;25:863-869.
19. Koch DD, Oryall JJ, Quam EF, Feldbruegge DH, Dowd DE, Barry PL, Westgard JO. Selection of medically useful QC procedures for individual tests on a multi-test analytical system. Clin Chem 1990;36:230-233.
20. Mugan K, Carlson IH, Westgard JO. Planning QC procedures for immunoassays. J Clin Immunoassay 1994;17:216-222.
21. Westgard JO. OPSpecs Manual - Expanded Edition. Madison, WI:Westgard QC, Inc., 1996.
22. Westgard JO. Basic Planning for Quality. Madison WI: Westgard QC, Inc., 2000.
23. Westgard JO, Stein B, Westgard SA, Kennedy R. QC Validator 2.0: a computer program for automatic selection of statistical QC procedures for applications in healthcare laboratories. Computer Method Programs Biomed 1997;53:175-86.
24. Westgard JO, Stein B. Automated selection of statistical quality-control procedures to assure meeting clinical or analytical quality requirements. Clin Chem 1997;43:400-3.

Other Six Sigma articles:

Six Sigma DPM Table
Six Sigma Quality Management
Six Sigma and Requisite Laboratory QC
Six Sigma Basics: Process improvement, goals, and measurements
Six Sigma: Outcome measurement of process performance

Copyright © 2001. All rights reserved.
Westgard QC, 7614 Gray Fox Trail, Madison WI 53717
Call 608-833-4718 or e-mail us at westgard@westgard.com

A Message from JOW
QC Lessons | QC Applications | Questions | Multirule
CLIA | What's New? | Catalog | Demo Download
Home | Glossary | ARCHIVES | Links | Feedback