The Right QC: "Tell me what you heard"

CLSI presented an education symposium on "Reducing Risk in Healthcare: Present and Future Trends in the Laboratory" during its annual Leadership Conference, March 21st, 2012. Dr. Westgard presented, participated and offers his perspective on where Risk QC is taking the laboratory.

The Right QC: “Tell me what you heard!”

James O. Westgard, PhD
April 2012

• Quality is the "Right Care at the Right time to the Right Patient"
• Doing the Right QC Right
• "The Right QC" according to CLSI and CMS
• The Right QC according to ISO and CLIA
• Doing the Right Risk Analysis Right
• What to do?

I was happy to participate in the CLSI symposium on "Reducing Risk in Healthcare", particularly because this occurred at the height of the cherry blossoms. The many monuments are even more impressive when viewed together with the beauty of the cherry trees. I also visited the 9/ll Memorial at the Pentagon just down the street from the Sheraton Hotel where the CLSI meeting was held. While off the beaten track of the other memorials across the Potomac River, the Pentagon Memorial makes a very strong and lasting impression. It is similar in concept to the memorial in Oklahoma City, but provides benches instead of chairs in memory for every person who died, including the all the passengers in the airplane that crashed into the Pentagon. It is particularly striking in the golden color and dark shadows of the late afternoon sun.

Quality is the “Right Care at the Right Time to the Right Patient”

In leading off a session on Risk Analysis and Laboratory QC Plans, Dr. Carolyn Clancy commented that quality is actually “harder than rocket science.” She made this statement as a representative of the Agency for Healthcare Research and Quality, whose mission is to improve quality in US healthcare. She defined quality in healthcare as providing the “right care at the right time to the right patient.”

In discussing risk analysis and patient safety, Dr. Clancy described the complex medical processes that involve numerous patient handoffs throughout diagnosis and treatment. She emphasized the need for clear communications to assure patient safety, citing a pharmacist who provided directions to a patient, then inquired of the patient: “Tell me what you heard!” That simple statement could be applied to all our communications and is particularly applicable to the changes that are occurring with laboratory QC.

Doing the Right QC Right

Dr. Clancy’s definition of quality struck a familiar chord. For many years, we have discussed “doing the right QC right” on this website and in many of our training materials. With respect to QC, the 1st right has to do with defining what QC should be implemented and the 2nd right has to do with implementing QC correctly.

For example, we have a book titled “Assuring the Right Quality Right: Good Laboratory Practices for Verifying the Attainment of the Intended Quality of Test Results.” “Right Quality” here means the quality needed for the intended clinical use of a laboratory test. This book deals with the need for improvements in the quality of laboratory testing, fundamentals of QC design, practical tools for selecting the right QC procedures, and guidance for implementing QC right. The focus is primarily on Statistical QC procedures that can be widely applied to many methods and analytic systems in healthcare laboratories.

“The Right QC” according to CLSI and CMS

With adoption of the CLSI EP23A guideline for use of risk analysis to develop laboratory QC Plans, both CLSI and CMS are now emphasizing “The Right QC”.

According to CLSI:

“’The Right QC’ in today’s hospital laboratory is the Quality Control Plan that’s right for you! It’s made up of many controls that are customized to your individual needs. Controls based on your institution. Controls that capitalize on a variety of measures aimed at preventing errors, detecting weaknesses, and minimizing risk. And, it’s from CLSI!”

According to CMS:

• “The ‘Right QC’ is IQCP [Individualized Quality Control Plan].
• Permits laboratories to develop an IQCP using many of their existing quality practices/information.
• Is based on laboratories’ patient population, environment, test system, clinical uses, etc.
• Applies to CMS-certified, non-waived laboratories.
• IQCP is a choice, and the default is two external QC/day.
• Laboratories must follow manufacturers’ instructions if they are more stringent than CLIA…”

What do these descriptions convey to you about the Right QC? Will you know it when you see it? Can you identify what’s necessary in your own laboratory? Tell me what you heard!

The Right QC according to ISO and CLIA

The use of risk analysis is being driven by ISO guidelines for industry. Diagnostics manufacturers, in turn, are the driving force for introducing risk analysis for laboratory QC Plans. Such QC Plans must also satisfy both accreditation and regulatory requirements, as found in ISO 15189 and CLIA.

The need to define the right QC is clearly implied in ISO 15189 section 5.6.1 which requires:

“…the laboratory shall design internal quality control systems that verify the attainment of the intended quality of results.”

CLIA provides an even more detailed definition in section 493.1256 which states that:

(a) Laboratory is responsible for having control procedures that monitor the accuracy and precision of the complete analytical process;

(b) …must establish the number, type, and frequency of testing control materials…;

(c) The control procedures must detect immediate errors that occur due to test system failure, adverse environmental conditions, and operator performance; and monitor over time the accuracy and precision of test performance…”

Are these definitions the same as the earlier CLSI/CMS descriptions of The Right QC? Or, are they consistent with the CLSI/CMS descriptions, but provide more detailed technical specifications of The Right QC? Tell me what you heard!

Doing the Right Risk Analysis Right

We have followed the evolution of CLIA rules for QC from the original 1992 Rule through the 2003 Final Rule, through QC Clearance to Default QC to Equivalent QC and now to Alternative QC in the form of Risk Analysis and IQCP. We have continued to hear that QC must get simpler because laboratories are having too many problems doing QC right, particularly because testing is becoming more decentralized and there is an expanding shortage of well-trained analysts. While we have many reservations about the difficulties of learning to do risk analysis correctly, we accept that risk analysis provides an appropriate approach for developing a proper QC Plan. But, like any technique or tool, there are many issues with defining the right risk analysis approach and implementing it right.

Doing the right risk analysis first involves consideration of safety characteristics through a rigorous process for validating a method’s precision, bias, reportable range, reference intervals, detection limit, and potential interferences. To assure safety characteristics are acceptable, performance must be compared to a defined requirement for quality, such as the allowable total error (TEa) at a specified critical medical decision concentration. Next the right risk model is required, which should be a 3-factor model that includes occurrence, severity, and detection. Integration of Six Sigma concepts facilitates improvements in the ranking or scoring of risk factors and estimation of residual risk in meaningful terms, such a residual defect rate that can be translated into the number of potentially harmful patient test results.

Doing risk analysis right requires a well-defined methodology that is properly tailored to the needs and applications of medical laboratories. The Joint Commission methodology for Proactive Risk Reduction provides a good starting point because it allows for both 2-factor and 3-factor models. Use of 2-factor models should be adequate for pre-analytic and post-analytic processes, but 3-factor models are preferred for analytic processes because assessment of the detection capability of controls is a critical part of doing Risk Analysis right. Information about the performance characteristics of Statistical QC procedures is readily available, but laboratories will need help from manufacturers to assess the performance of built-in and procedural controls. Manufacturers may be reluctant to provide such information, as evidenced by their rejection of a proposed companion document CLSI EP22P: Presentation of a Manufacturer’s Risk Mitigation Information for Users of In Vitro Diagnostic Devices. Laboratories will need to make a special request for a manufacturer’s risk analysis results because they are not part of current labeling and documentation.

What to do?

Laboratories face a steep learning curve in implementing risk analysis and Individualized Quality Control Plans. IQCPs are the right direction to go, but the exact path is not yet clear and a more detailed roadmap is needed! CMS expects there will be a two-year educational period for transition from Equivalent QC to IQCP. CLSI is committed to supporting the educational needs of medical laboratories to learn risk analysis. Their intention is to provide a coordinated effort to get this right, but they are trying to establish a new standard for QC practice based on principle and theory, without any practical experience in the field! That’s an unusual way of developing a new standard of practice, particularly for a consensus organization such as CLSI.

Given this situation, there is a danger that QC may get worse before it gets better. The lack of careful definition of “The Right QC” and the suggestion that it will be simple to do this because laboratories can make use of what they’re already doing is problematic. It reminds me of a story where Hager and Lucky Eddy arrive at a bar and Hagar asked Lucky Eddy “what will you have?” To which Lucky Eddy replies “My usual.” Hagar asks “What’s your usual?” And Luck Eddy responds “It’s what I usually have.” The analogy here is if you ask a laboratory what’s the right QC, the answer will be “It’s what I usually do - Why would I do the wrong QC?” Improvements in quality come from changing our processes to make them better, not just doing our usual!

I find myself in the same situation as the pharmacist mentioned by Dr. Clancy. The prescription for use of risk analysis is appropriate, but I feel a responsibility to advise you of the potential complications and side-effects for your laboratory and your patients.

Tell me what you heard!

Even more important, tell me what you intend to do!