First do no harm

Risk Management standards are on the way. Some of the old QC techniques are going to be replaced by these new methods. That's all well and good, but in the end, you still need a way to catch those errors that slip through. Dr. Westgard explains how to balance the best of the old with the best of the new.

First, Do No Harm:

A Safety Net to Catch Analytic Errors

• Guidance for Error Management
• Guidance for Risk Management
• What's the Point?

March 2009

James O. Westgard, PhD, Sten A. Westgard, MS

New management approaches continue to be introduced, however, that doesn’t necessarily lead to the continuous improvement in management! One danger is that what is “new” becomes a replacement for what is “old,” whether or not the new is really better than the old. Laboratories face that very issue with the introduction of risk management. The “old” in this case may be described as “error management” which focuses on defining how much error is allowable for the tests of interest, estimating the sizes of errors that may occur in the methods in use in the laboratory, monitoring those errors to detect when they exceed what is allowable, and assuring that the error sources are identified and corrective actions are taken so that erroneous test results do not harm our patients.

 

We advocate that “error management” must be maintained because it is a proven approach that provides laboratories with practical tools that can be readily applied. The application of risk management for development of quality systems may be sound in theory, but laboratories do not yet have much experience using this approach. There is a learning curve with any new management approach and laboratories are at the bottom of the learning curve for risk management.

Guidance for Error Management

In the development of Analytical Quality Control (AQC) systems, laboratories should take advantage of well-established principles and available guidelines for defining analytic quality goals, validating method performance, and designing statistical QC procedures. Those initial steps provide the building blocks upon which risk analysis can be added to expand and improve an AQC system. In fact, these initial steps can help you assess the importance of applying risk analysis to improve your AQC systems.

 

These steps in “error management” also take into account some critical parameters that may NOT be adequately considered in risk management, for example:

 

• Quality goals should be defined to describe the “intended quality of results” that must be attained for a laboratory test to be clinically useful. Goals for analytical quality can be defined in quantitative terms, typically the amount of analytic error that is allowable. There is a long history of scientific discussion and many recommendations in the scientific literature that provides practical guidance for defining analytical quality goals [see reference 3]. In contrast, there is little guidance for defining a clinically acceptable risk, which is necessary to evaluate the effectiveness of any risk mitigation efforts. Risk management by itself does not provide a quantitative framework by which the laboratory can evaluate the acceptability of the analytical performance of a measurement procedures or the effectiveness of statistical QC for monitoring performance.
  
  
• Analytical performance characteristics must be evaluated in all laboratories as part of good laboratory practice and accreditation. For example, ISO 15189 [1] makes the statement in section 5.5 that “performance specifications for each procedure used in an examination shall relate to the intended use of that procedure.” Laboratories in the US are required by CLIA [2] to validate the performance of all new non-waived measurement procedures and analytic systems. Specifically, they must verify the reportable range, bias, precision, and reference intervals that are claimed by the manufacturer before reporting patient test results. Details of the evaluation process and experiments can be found on this website and also in reference 4. The information that is obtained on analytic performance should provide important inputs into the planning of an AQC system. This is readily done using traditional error concepts and an error framework for analytical quality management, but the observed precision and accuracy of a measurement procedure may not be taken into account quantitatively in the risk analysis techniques performed by manufacturers, nor in the risk management information provided to laboratories.
  
  
• QC procedures are likewise required in all laboratories as part of international and national standards of practice. ISO 15189 provides specific guidance in section 5.6.1 that “the laboratory shall design internal quality control procedures that verify the attainment of the intended quality of results.” This can readily be accomplished by the proper design of Statistical QC (SQC) procedures to account for the precision and accuracy observed for a measurement procedure, as well as the rejection characteristics of different decision criteria (control rules) and numbers of control measurements [see reference 5].
  
  
• •AQC strategies can be formulated on the basis of the quality goals, method performance, and the effectiveness of the SQC design. Such AQC strategies determine the balance between statistical and non-statistical QC procedures and alert the laboratory to the level of risks that must be managed. Risk analysis and the mitigation of residual risks are complementary and secondary activities that expand the AQC plan to maximize its effectiveness.

Guidance for Risk Management

One simple way to recognize the potential risk in an analytic testing process is to calculate the sigma-metric, i.e., Sigma = [(TE_a-bias)/SD], where TE_a is the quality goals (tolerance limit in Six Sigma terminology) in the form of an allowable total error, bias represents the observed inaccuracy of the measurement procedure, and SD represents the observed imprecision of the measurement procedure [6].

• Low risk methods: Measurement procedures having a Sigma-metric of 5.0 or higher can be easily monitored by SQC to detect medically important error, thus such methods have low risk of producing analytic errors that may cause harm to patients. You can rely on SQC for your safety net.
  
  
• Moderate risk methods: Measurement procedures having a sigma-metric from 4.0 to 5.0 have higher risks of producing erroneous test results, but their quality can still be adequately managed by an AQC strategy that maximizes SQC, aggressively adheres to manufacturer’s maintenance guidelines, updates, and improvements, and incorporates other non-statistical checks and functions to monitor specific identified risk factors.
  
  
• Hi risk methods: Measurement procedures that have low sigma-metrics, generally 4.0 or less, will be difficult to monitor by SQC alone. These are your highest risk methods for producing test results that may have medically important analytic errors! They require your care and attention to improve AQC, improve method performance, and possibly even replace the methods with a new analytic system that provides higher quality results.

What’s the point?

One of the basic tenants of medical practice is to “first, do no harm.” That means that laboratories should first be concerned with patient safety and eliminate, monitor, and detect errors that may cause harm to the patient. The safest way to approach the design of your AQC system is to start with some traditional error management techniques that allow you to 1st define how good your methods must be, 2nd validate that your methods have the necessary precision and accuracy to achieve the desired quality goals, 3rd quickly identify the right SQC procedure (right rules, right number of control measurements) that must be implemented to detect medically important analytic errors, and 4th help you recognize the level of risk for producing erroneous test results and to formulate a AQC strategy to minimize that risk.

These steps provide a laboratory with a “safety net” that will catch most analytic errors and keep them from harming the patients. This safety net should not be replaced or displaced by new control techniques until such those new techniques are proven to be effective. In many cases, the best AQC plan will be to supplement this basic safety net with techniques that target specific sources of errors and add other QC checks that improve prevention, detection, and corrective actions.

References

1. ISO 15189 Medical Laboratories – Particular requirements for quality and competence. 2007

    

2. US Centers for Medicare & Medicaid Services (CMS). Medicare, Medicaid, and CLIA Programs: Laboratory requirements relating to quality systems and certain personnel qualifications. Final Rule. Fed Regist Jan 24, 2003;16:3640-3714. www.cms.gov/clia

    

3. Hyltoft Petersen P, Fraser CG, Kallner A, Kenny D. Strategies to set global analytical quality specifications in laboratory medicine. Scand J Clin Lab Invest 1999:59:No.7(Nov).

    

4. Westgard JO. Basic Method Validation, 3rd ed. Madison WI:Westgard QC, Inc., 2008, 320pp.

    

5. Westgard JO. Assuring the Right Quality Right: Good laboratory practices for verifying the attainment of the intended quality of test results. Madison WI:Westgard QC, Inc., 2007, 288 pp.

    

6. Westgard JO. Six Sigma Quality Design and Control: Desirable precision and requisite QC for laboratory measurement processes, 2nd ed. Madison WI: Westgard QC, Inc., 2006, 338pp.