In today's competitive environment, health care organizations often place a high priority on cost control rather than quality control. This preference is based on the assumption that current laboratory quality-control practices are sufficient to achieve s atisfactory analytical quality. It is unlikely that this is still a valid assumption, however, since major organizational and operational changes have been taking place in laboratories throughout the nation, resulting in personnel reductions and changes i n the skill level and training of analysts.

Quality control (QC) depends on having carefully optimized and stable testing processes. We hope that new instrument systems and the next generation of laboratory automation will assure analytical quality in these unstable times, but changes in technology also make it possible to produce more bad test results faster than ever. To achieve improved quality management in the future, laboratory scientists must continue to play an active role in managing and assuring the analytical quality of test results.

State of Laboratory QC Practice

Laboratory QC practices have not kept pace with the evolution of measurement technology. A recent College of American Pathologists' Q-Probe survey on daily quality control exception practices in 505 laboratories (1) documents that the control rules being used today by most laboratories are the same as ten years ago. This is a rather astonishing observation because instrument measurement systems have advanced one to two generations during this same period. Many laboratories still use control limits set as the mean plus or minus 2SD (12s control rule)-a practice that dates to the 1960s when continuous flow analyzers were being introduced. With these 2SD control limits, a false rejection rate of 9% to 14% is expected for the two to three control measurements (N) that are commonly analyzed per run. The CAP survey reports actual laboratory run rejection rates of 0.4% to 0.6% (1), which are not consistent with what is expected if laboratories are doing what they say they are doing. Obviously, something is wrong ! Laboratories appear to be reducing the number of out-of-control situations to eliminate rework and provide more efficient, more productive, and less costly testing processes. While this is a legitimate management objective, it is important that it be acco mplished in the proper way, otherwise analytical quality may suffer. Cost-effective QC means the use of control procedures that maximize both the quality and productivity of laboratory testing processes (2). False rejections need to be reduced, but approp riate error detection must be maintained.

Current QC practices that automatically repeat the controls and require a second out-of-control signal effectively reduce rejections by changing a 12s rule to a 22s rule-one that is not very sensitive for detecting errors. Other QC practices widen the con trol limits by using calculations based on larger medically allowable standard deviation (SD) or inflated estimates of method SD, and also by imposing medical limits or total error limits directly on control charts. These practices will reduce rejections by changing an apparent 12s or 13s control rule to an actual statistical rule of 14s, 15s, or even 16s (3). These practices reduce false rejections, but at the same time they also reduce error detection to a level that may become so low as to be hazardous to the health of the patients being tested.

For example, Figure 1 shows the probabilities of rejecting runs (shown on y-axis) that have various amounts of systematic error (shown on x-axis) for 12s, 22s, 13s, 14s, 15s, and 16s control rules when there are two control measurements per run.

If it were important to detect a systematic error equivalent to three times the size of the method SD (shown by vertical line), the probability of rejection could be from 0.98 to 0.00, depending on the actual statistical control rule being used. The labo ratory might have a 98% chance of detecting a medically important error, or a 0% chance, depending on the actual QC procedure being used.

What's the Right QC Procedure?

When selecting a QC procedure, the primary objective should be to ensure that data points fall outside the control limits when the method is not working properly, not fall within the control limits when there are no problems. By using this as the criterio n, errors in test results that lead to medically important errors can be identified.

Placing a high priority on quality in test results means that laboratorians must decide on a quality goal or standard for their laboratories. The management objective would then be to achieve the desired quality at the minimum cost. This means aiming for 90% error detection and 5% or fewer false rejections-and doing it with the minimum N. If quality is to be properly controlled, laboratorians must define the quality they wish to achieve in order to determine the sizes of medically important errors. This g uides the selection of QC procedures. Otherwise it makes no difference what statistical rules and N are used-it's arbitrary, and this practice would be better described as arbitrary control rather than quality control.

Improving QC Practices

Tetrault and Steindel (1) recommend that 'the best set of control rules will vary from method to method' [or test to test] and that 'the laboratorian has to balance true error-detection capabilities against the probabilities of falsely rejecting a good ru n.' This can be done quite easily today using the step-by-step quality-planning process shown in Figure 2, which is supported by graphical planning tools such as charts of operating specifications ('OPSpecs charts') (4).

The starting point is to define the analytical or clinical quality that is desired. The format for these quality requirements is important if they are to be useful for establishing specifications for the precision, accuracy, and quality control needed in routine operation. An analytical quality requirement should be in the form of an allowable total error, such as those identified under the CLIA proficiency testing (PT) criteria for acceptable performance (e.g., cholesterol should be correct within 10% of a given target value). A clinical quality requirement should be in the form of a medically significant change between two alternative diagnostic decisions or treatment classifications (e.g., a change in a cholesterol test result from 200 mg/dL to 240 mg/ dL would lead to a different diagnostic classification according to treatment guidelines established by the National Cholesterol Education Program).

Analytical or clinical quality requirements that are stated in these formats can be related to the precision and accuracy that are allowable and the QC rules and Ns that are necessary using OPSpecs charts (5). For example, Figure 3 shows an OPSpecs chart for a 10% total error criterion, which corresponds to the CLIA proficiency testing criterion for cholesterol (and is also applicable for total protein, blood lead, calcium at a decision level of 10 mg/dL, glucose >100 mg/dL, potassium at 5 mmol/L, urea ni trogen at 20 mg/dL, and pCO2 at 50 mm Hg).

The chart shows the allowable inaccuracy on the y-axis and the allowable imprecision on the x-axis. The different lines describe the limits of method inaccuracy and imprecision that are allowable for different QC procedures.

To select a QC procedure, simply plot your method's observed imprecision and inaccuracy and identify the lines above your operating point. Any QC procedure where the limits of allowable imprecision and inaccuracy are above your method's operating point wi ll provide the error detection or analytical quality assurance (90% AQA for systematic error) stated on the chart.

Take the example of a cholesterol method that has a 1% bias and 2% CV. The QC procedures that would provide appropriate error detection correspond to the five lines above the operating point, which are identified in the key at the right of the chart as th e 12.5s rule with N = 4; 12s single rule with N = 2; 13s/22s/R4s multirule with N = 4; 12.5s rule with N = 3; and 13s/2of32s/R4s multirule with N = 3. All provide at least 90% error detection. False rejection rates will vary from 1% to 9%, as shown by the Pfr figures in the key. The 13s/2of32s/R4s multirule or the 12.5s rule with N = 3 would be better choices than the 12s with N = 2 because the false rejection rates are much lower (1%-3% vs. 9%).

Selection of appropriate control rules and Ns can be done in a minute or less once you define the quality you want to achieve, have information about your imprecision and inaccuracy, and have the necessary OPSpecs charts. You can prepare the OPSpecs chart s yourself using an electronic spreadsheet (6,7), obtain them in workbook format (8) for commonly used single and multirules and N = 2, 3, 4, or 6, or obtain specialized software that can prepare the charts for a wider range of rules and values of N (9). Because of the need to consider additional preanalytical factors, such as within subject biological variation, a computer program is recommended for use with clinical quality requirements.

By employing a step-by-step quality-planning process, cost-effective quality control procedures can be designed for the methods in any laboratory. With today's tools and technology, Tetrault and Steindel's objectives of selecting the best set of rules for individual applications and balancing the error detection and false rejection can be accomplished quickly and easily.

Implementation of Cost-Effective QC

Application of individualized QC designs is facilitated by QC software that permits implementation of a variety of single and multirule QC procedures on a test-by-test basis. Current QC software has multiple shortcomings: it often restricts the single and multirules available; it lacks the flexibility to use different rules on different tests on a multitest system; it makes it difficult to apply rules across two or three control materials; and it is unable to implement multistage QC designs that utilize d ifferent rules and N values during the initial startup and later monitoring of routine operation.

QC software is not likely to advance until laboratories demand improvements. Instrument manufacturers whose competitive positions can be enhanced by guaranteed quality, improved test yields, and reduced costs of operation may be expected to be more respon sive than suppliers of laboratory or hospital information systems.

Manual implementation is also possible because only three to five different designs are generally needed to cover a wide range of method performance capabilities. These designs can be as simple as using single rules with 3.5s, 3s, 2.5s, and 2s control lim its and common N values such as 2, 3, or 4-a progression that increases error detection as the control limits narrow and N increases. Or, a multirule algorithm can be adapted to three to five designs by varying the number of rules and Ns used with differe nt tests (e.g., 13s with N=3; 13s/2of32s/R4s with N = 3; 13s/2of32s/R4s/31s with N = 3; 13s/2of32s/R4s/31s with N = 6; 13s/2of32s/R4s/31s/6x with N = 6-a progression that increases error detection by increasing both the rules and N).

A plan to implement cost-effective QC is outlined in Table I.

Table I
Achieving Cost-Effective QC

Identify those tests that will benefit from improvements in QC by performing a QC assessment or audit of your present methods (8).

Begin with those regulated tests for which CLIA has defined a PT total error requirement (TEPT). Obtain estimates of the imprecision (smeas) and inaccuracy (biasmeas) of your methods from replication and comparison of methods evaluation studies or from on-going data on control pools and PT survey samples.

Calculate the critical-systematic error [DSEcrit = (TEPT - biasmeas)/smeas - 1.65] to classify your methods and guide your actions.

• If SEcrit is > 4, the test will be easy to QC with single rules, such as 13.5s and 13s, and Ns of 2.
• If SEcrit is 3-4, QC should be relatively easy, but you may need to use narrower control limits such as 2.5s, multirule procedures, and Ns of 2 or 3.
• If SEcrit is 2-3, you will likely need a 2.5s single rule or multirule procedures with Ns of 3 or 4.
• If SEcrit is < 2, these methods will be difficult to control and you will need to use multirule procedures with higher Ns of 4 to 8 or tolerate a higher false rejection rate for 2s control limits and Ns of 2 to 4, appro ximately 10% to 20% false rejections. (Methods in this class have a high priority for improvement or even replacement.)

The biggest cost savings are likely to come from automated, very precise, and highly stable chemistry and hematology analyzers where cost-effective QC should be possible with low Ns and low false rejection rates. For example, in studies to optimize the Q C designs for individual tests on automated multitest chemistry analyzers (10), we estimated savings of $17,400 per year, or $87,000 over an expected five-year lifetime of the analytical system by changing control rules and reducing false rejections (11). Additional changes to lengthen the analytical run more than doubled those savings. While it may appear daunting to start with multitest automated systems, those systems often perform the bulk of the work in a laboratory and their operation has a major ef fect on laboratory costs.

Tips for Implementing Cost-Effective QC

• Start with high-volume, automated chemistry and hematology analyzers.
• Address lower-volume analyzers as you gain experience in optimizing QC.
• Expect that you may need to increase QC on earlier generation analyzers, such as those currently available for immunoassays, toxicology, and coagulation.
• Recognize that manual methods will often be difficult to control because of operator variability.

QC in the Future

Even with the current emphasis in health care on total quality management and continuous quality improvement, there is a pressing need to improve the technical quality management capabilities in laboratories. The increasing use of automation and robotics may make the laboratory's QC process the bottleneck in the production and reporting of test results. We need to address the pro-blems of automating QC and managing the next generation of measurement technology.

To accomplish this, new QC technology is needed! This technology should allow the user to enter the desired quality requirement, either analytical or clinical, and then the technology itself should select the control rules and numbers of control measureme nts that are needed and implement them to achieve the desired quality at minimum cost. This automatic QC process should also be able to adapt to changes in method performance so that control increases when analytical performance deteriorates and costs dec rease when performance improves. A system of this nature would provide a dynamic, cost-effective QC process that guarantees the desired analytical or clinical quality at the lowest cost. As with any new technology, the laboratory will need to understand t he scientific basis of the QC process, evaluate its performance, make a decision on its acceptability for the intended applications, and then apply the new technology to achieve the benefits of dynamic quality management.

These future QC needs also create opportunities for laboratory professionals who are interested in technical quality management. Among the skills needed will be: an understanding of medical decision-making; the ability to assess clinical and analytical qu ality requirements; the ability to evaluate method performance characteristics; the capability to determine QC performance characteristics; the capability to model and simulate testing processes that optimize quality and productivity; the ability to de velop software specifications for process control systems; and the ability to provide training in technical quality management.

For now and the future, analytical quality is fundamental to the core production processes of all laboratories, and only laboratory professionals have the skills to assure that desired quality can be achieved at minimum cost.

REFERENCES

1. Tetrault GA, Steindel SJ. Q-Probe 94-08. Daily quality control exceptions practices. Chicago, Ill.: College of American Pathologists, 1994.
2. Westgard JO, Barry PL. Cost-effective quality control: managing the quality and productivity of analytical processes. Washington, D.C.: AACC Press, 1986.
3. Westgard JO, Quam EF, Barry PL. Establishing and evaluating QC acceptability criteria. Med Lab Observ 1994; 26(2):22-6.
4. Westgard JO. Charts of operational process specifications (OPSpecs charts) for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing performance criteria. Clin Chem 1992;38:1226-33.
5. Wiebe DA, Westgard JO. Cholesterol-a model system to relate medical needs with analytical performance. Clin Chem 1993;39:1504-13.
6. Westgard JO, Wiebe DA. Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing. Clin Chem 1991; 37:1938-44.
7. Westgard JO, Hytoft Petersen P, Wiebe DA. Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program (NCEP). Clin Chem 1991:37:656-61.
8. Westgard JO. OPSpecs manualÑexpanded edition: operating specifications for precision, accuracy, and quality control. Ogunquit, Maine: Westgard QC, 1996.
9. Westgard JO. A program for assessing statistical QC procedures. Med Lab Observ 1994;26(2):55-60.
10. Koch DD, et al. Selection of medically useful QC procedures for individual tests on a multi-test analytical system. Clin Chem 1990;36:230-3.
11. Westgard JO, Oryall JJ, Koch DD. Predicting effects of QC practices on the cost-effective operation of a multitest analytical system. Clin Chem 1990; 36: 1760-4.
12. Westgard JO, Burnett RW. Precision requirements for cost-effective operation of analytical processes. Clin Chem 1990;36:1629-32.
13. Westgard JO. A method evaluation decision chart (MEDx Chart) for judging method performance. Clin Lab Science. 1995;8:277-83.