Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Essays
  4. Quality Requirements and Standards
  5. Good Laboratory Practices for Statistical QC, Part I:

Quality Requirements and Standards

Good Laboratory Practices for Statistical QC, Part I:

Did you know the ASCP held a teleconference in 2006 on " A 'How-Should-I' guide to Laboratory Quality Control." They cited the lack of "Good Laboratory Practice" standards for QC in the laboratory. If only they had looked at the recent CLSI standards, they would have found some. Dr. Westgard explains the new C24-A3 standard and how it can lead you to good laboratory practices in quality control.

 

 

Designing the Right QC

November 2006

In the fall of 2006, ASCP sponsored the teleconference “A How-Should-I Guide to Laboratory Quality Control,” which focused on the following problems:

  • There is no official "Good Laboratory Practice" guideline for setting QC limits;
  • There is no official “Good Laboratory Practice” guideline for monitoring or evaluating QC;
  • There is no official “Good Laboratory Practice” guideline for creating a good sustainable QC program that yields good value for the resources required.

In response, the teleconference set out to achieve the following objectives:

  • Establish quality control (QC) limits based on defensible good laboratory practice;
  • Understand how QC results can be monitored and evaluated to assure good laboratory quality;
  • Create a good QC program that is sustainable and yields good value for the resources required.

I believe that the Clinical Laboratory Standards Institute (CLSI) C24-A3 document provides good guidance for resolving these problems and achieving these objectives.

Background

In July 2006, CLSI published the 3rd revision of the C24 guideline “Statistical Quality Control for Quantitative Measurement Procedures: Principles and Definitions.” This guideline has gotten little publicity relative to the attention given to the CLSI documents on risk assessment for QC (EP22 and EP23), which are currently under development. These risk assessment guidelines relate to “option 4” for “equivalent QC”, now described as “alternative QC.” While laboratories wait for these risk assessment guidelines, they would be well advised to adhere to the standards of practice recommended in C24-A3. In fact, even when these risk assessment guidelines become available, laboratories would be well advised to implement these recommendations for statistical QC.

The 1st edition of C24 goes back to 1989 when the committee was chaired by Dr. Roy Barnett and was notable for replacing the “batch” definition of a run with a more flexible definition based on the “interval (i.e., period of time or series of measurements) over which the accuracy and precision of the method are expected to be stable.”

The 2nd edition was published in 1999 and recommended a general QC-planning approach to select QC rules and numbers of measurements appropriate for the quality required by the test and the performance (precision and accuracy) available from the method.

The 3rd edition is notable for providing more practical advice on QC selection, including a simple “Sigma-metrics QC Planning Tool.” In addition, there are recommendations for selection of control materials, determination of means, SDs, and control limits, as well as interpretation of control status, particularly taking action to solve control problems, rather than just repeating the controls.

Doing the Right QC Right

The quality of quality control depends on “doing the right QC right.”

  • The 1st right has to do with designing or selecting the right control rules and right number of control measurements.
  • The 2nd right has to do with implementing those rules properly, which involves determining control limits in the right way, which in turn depends on obtaining reliable estimates of the mean and SD, which also depends on collecting an appropriate number of control measurements over an appropriate period of time.

C24-A3 recommends a QC design process that takes into account the quality required for a test and the precision and accuracy observed for a method. This leads to the selection of the right QC rules and the right number of control measurements.

QC Design Process

C24-A3 recommends the following process for designing r selecting a QC procedure (or QC strategy):

  1. Define the quality specifications for the test;
  2. Select appropriate control materials;
  3. Determine the stable (in control) performance characteristics of the measurement procedure;
  4. Identify candidate quality control strategies;
  5. Predict the likelihood that candidate quality control strategies will detect out-of-specification performance;
  6. Specify desirable goals for QC performance characteristics;
  7. Select a quality control strategy whose predicted performance meets or exceeds the quality control performance goals.

QC Selection Tool

The appendix of C24-A3 provides some examples using a practical tool, which makes use of a sigma-metric to take into account the quality specification (or allowable total error requirement, TEa) for the test and the observed precision (smeas) and accuracy (biasmeas) of the measurement procedure, as follows:

Sigma-metric = (TEa – biasmeas)/smeas

Note the relationship of this sigma-metric to the critical systematic error (DSEcrit) that must be detected by the QC procedure:

DSEcrit = [(TEa – biasmeas)/smeas] – 1.65

Given the availability of “power curves” that show the probability of rejection as a function of the size of errors, a “critical-error graph” or “sigma-metrics graph” to allow assessment of the probabilities of error detection (Ped) and false rejection (Pfr), which can be used to select appropriate control rules and numbers of control measurements. Reasonable objectives are to achieve a Ped of 0.90, or 90% chance of detecting medically important errors, while maintaining a Pfr of 0.05 or less, i.e., 5% chance, or less, for a false alarm or false rejection.

Cholesterol Example Applications

Given a cholesterol test where the CLIA criterion for acceptable performance is 10.0% and given a measurement procedure that has an observed imprecision of 2.0% and an observed inaccuracy of 0.0%, the sigma-metric would be calculated as follows:

Sigma-metric = (10.0%-0.0%)/2.0% = 5.0

This application is represented by the vertical-line at 5.0 sigma, which shows the condition imposed on the power curves of commonly utilized control rules and numbers of control measurements, which are shown in the key at the right side of the figure. The power curves, top to bottom, related to the control rules and Ns in the key, top to bottom. Given the objective of achieving Ped of 0.90, there are 3 QC procedures that are possible choices: 12.5s with N=2, which provides a Ped of approx. 0.96 and a Pfr of about 0.03; 13s/22s/R4s with N=2, providing a Ped of approx. 0.94 and a Pfr of about 0.01; 13s with N=2, providing a Ped of approx. 0.87 and a Pfr of 0.01. All three are acceptable, and the choice depends on the ease of implementation in the laboratory. That will likely lead to the use of a Levey-Jennings chart with 3s control limits and 1 measurement on each of 2 levels of control materials.

Given a cholesterol test where TEa is 10.0%, biasmeas is 2.0%, and smeas is 2.0%, the sigma-metric would be calculated as follows:

Sigma-metric = (10.0%-2.0%)/2.0% = 4.0

This application is represented by the vertical-line at 4.0 sigma. The two best candidate QC procedures are seen to be 13s/22s/R4s/41s with N=4 and 12.5s with N=4. The multirule procedure provides a Ped of 0.91 and a Pfr of 0.03, whereas the single-rule 2.5s procedure provides a Ped of 0.87 and a Pfr of 0.04. Implementation would most likely involve making 2 control measurements on each of 2 levels of control materials.

Defensibility of Recommended Process

This process is scientifically defensible because the concepts and principles of this approach appear in the peer reviewed literature (1-12), plus the CLSI process itself includes peer-review. This process is also defensible under ISO 15189, which provides “Particular requirements for quality and competence” for medical laboratories:

  • Section 5.5 on “Examination procedures” includes the statement: “performance specifications for each procedure used in an examination shall relate to the intended use of that procedure.” “Intended use” implies that the quality requirement for the test should be defined.
  • Section 5.6 on “Assuring quality of examination procedures” states that “The laboratory shall design internal quality control systems that verify the attainment of the intended quality of results.” Here again is a reference to the quality required for the test and a specific guidance that the QC procedure shall “verify the attainment” of the “intended quality of results.”

Concluding Discussion

It is unfortunate that CLSI document C24-A3 is not more widely available and more widely utilized. The document costs $100 for non-members, which means most laboratories are not likely to have a copy available. However, the Sigma-metrics QC Selection Tool is readily available through materials from Westgard QC. See the following for more discussion and examples.

Additional GLPs for statistical QC will be addressed in this series. Part 2 will consider control limits and limitations. Part 3 will review recommendations and approaches for defining the “intended quality” of laboratory tests. Part 4 will address the monitoring and on-going evaluation of laboratory QC procedures. Part 5 will provide some guidance for establishing a total QC strategy for cost-effective laboratory operation.

References

  1. Westgard JO, Groth T, Aronsson T, Falk H, deVerdier C H. Performance characteristics of rules for internal quality control: Probabilities for false rejection and error detection. Clin Chem 1977;23:1857 67.
  2. Westgard JO, Groth T. Power functions for statistical control rules. Clin Chem 1979;25:863 69.
  3. Westgard JO, Barry PL. Cost-Effective Quality Control: Managing the quality and productivity of analytical processes. AACC Press, Washington, DC, 240 p, 1986.
  4. Koch DD, Oryall JJ, Quam EF, Felbruegge DH, Dowd DE, Barry PL, Westgard JO. Selection of medically useful QC procedures for individual tests on a multi-test analytical system. Clin Chem 1990;36:230-3.
  5. Westgard JO. Analytical quality assurance through process planning and quality control. Arch Pathol Lab Med 1992;116:765-769.
  6. Wiebe DA, Westgard JO. Cholesterol - a model system to relate medical needs with analytical performance. Clin Chem 1993;39:1504-1513.
  7. Westgard JO. A QC planning process for selecting and validating statistical QC procedures. Reviews in Clinical Biochemistry 1994;15(iv):155-64.
  8. Westgard JO. Error budgets for quality management: Practical tools for planning and assuring the analytical quality of laboratory testing processes. Clin Lab Manag Review 1996;10:377-403.
  9. Westgard JO, Stein B, Westgard SA, Kennedy R. QC Validator 2.0: a computer program for automatic selection of statistical QC procedures in healthcare laboratories. Comput Method Program Biomed 1997;53:175-186.
  10. Westgard JO, Stein B. An automatic process for selecting statistical QC procedures to assure clinical or analytical quality requirements. Clin Chem 1997;43:400-403.
  11. Westgard JO. Internal quality control: planning and implementation strategies. Ann Clin Biochem 2003;40:593-611.
  12. Westgard JO. Clinical quality vs analytical performance: What are the right targets and target values? Accred Qual Assur 2004;10:10-14.