Will Bad Risk drive out Good QC?
With the expiration date on EQC drawing ever closer, US labs still await the details about its replacement. While CLSI EP23 has been public for more than a year, and CMS has made press releases about the new IQCP (Individualized Quality Control Plans) and has announced the benefits of this new forthcoming policy, the actual regulatory language has yet to be released or described.
Will Bad Risk push out Good QC?
Sten Westgard, MS
November 2012
Gresham's Law- What does Gresham's Law tell us about Quality Control Plans?
- The myth of choice in Risk QC Plans
We are poised to learn more about the new Risk QC approach from CLSI and CMS. While the CLSI guideline EP23 has been out for more than a year, we have not yet heard how CMS plans to implement, interpret and inspect laboratories based on this new guideline. Given that EP23 itself is almost deliberately vague – it describes a “toolbox” full of QC techniques but does not mandate the implementation of any particular one – there is an even stronger need to have a clearly described plan for laboratories seeking to modify their QC practices.
Just as “one size fits all” is not the right approach for QC, “one size fits all” is not the right approach for Risk QC. This has been hinted at in the promotional and advertorial material generated by CLSI and CMS. CMS has described an “IQCP” – an Individualized Quality Control Plan in a recent brochure. Potentially for each test, for each device, for each laboratory, there will be a separate Quality Control Plan, guided by Risk Management.
Thus, what we’re facing in the laboratory is a beast with many heads. Each test may have a different QC approach that has been developed using different Risk Analysis techniques. This has lead more than one person to conclude that Risk Management promises more work, not less, for the laboratory. One of the open questions is how each Risk QC plan will be judged. That is, will any QC plan be acceptable? Will inspectors be empowered to cite laboratories for inadequate Risk Management or IQCP’s that leave too much “residual risk” to be passed on to the clinician and patient?
This generates still another dilemma for laboratories and the regulators who inspect them – if there is a choice between performing a slipshod, minimally compliant Risk Assessment , and doing a more rigorous, thorough, safe Risk Assessment, what will laboratories choose? If an incomplete Risk Analysis and an inferior IQCP is nevertheless “compliant” in the eyes of CLIA and CMS, what is the incentive for laboratories to develop the Right IQCP Right?
This leads us to a digression into monetary policy and a centuries-old concept called “Gresham’s Law.” Bear with me.
Gresham’s Law
In a letter to Queen Elizabeth in the mid-16th century, a merchant named Thomas Gresham attempted to explain why gold was disappearing from her kingdom. The problem had its roots in the debasement of currency started by Henry VIII, who began reducing the gold content in the coins of the realm. As a result, coins with more gold in them could be exchanged for coins with less gold in them, and it was quickly apparent that it was best to take the “good” coins out of the kingdom, and only use “bad” or debased coinage when making transactions with the British. As the British monarchs further debased their currency, “good” coins with more gold content became scarcer still. Indeed, the coins with the best gold content were hoarded, to prevent them from being exchanged for the debased coins. Thus, the market inevitably became dominated by debased inferior coins.
It’s easy to summarize the message of Gresham’s Law: bad money drives out good money. And there are centuries of examples to prove the truth of it.
One of the key underpinnings of Gresham’s Law is an enforced exchange rate. That is, some government or regulator requires that the two coins – one debased and one more pure – equal to each other, despite the reality of their different values.
In modern times, we have abandoned the metallic currency system and replaced it with a fiat currency. The intrinsic value of our coins, bills, and deposits is zero. So a dollar is a dollar, and individual dollars can’t be debased (although the entire currency can be inflated and become less valuable – but that’s an argument for another time and place)
What does Gresham’s Law tell us about Quality Control Plans?
This brings us back to that critical question about IQCPs: how will an IQCP be judged? If you do a poor job, making quick guesses about failure modes, assuming wildly optimistic performance in the factors of occurrence and severity, and generate a slapdash IQCP, will that be just as acceptable as an IQCP done correctly? If a good IQCP requires a committee of subject matter experts, several hours of hazard identification, process-mapping, data collection, and deliberation, etc, is this taken into account during inspection and accreditation?
If CMS sets the "exchange rate" such that a Bad IQCP is still the equivalent of a Good IQCP, then we can predict which way the market will drive Risk QC Plans. Production pressures on the laboratory will force them to do less with less, and generate IQCP plans that are compliant, but not necessarily sufficient or even safe.
In other words, if CMS seeks to displace “one size fits all” QC and replace it with “any size fits all” Risk QC Plans, this is not a better alternative. In fact, such a policy is likely to sound the starting pistol on a new race to the bottom. If labs can’t be failed for poor IQCPs, then their IQCPs will get poorer.
If inspectors and accreditors are not empowered to assess, judge and even reject the IQCPs generated by laboratories, essentially there is no limit to the debasement that could be done to an IQCP. To take this to absurd extreme, perhaps all a laboratory needs is a folder with the words “RISK MANAGEMENT IQCP” on the outside, and it won’t matter what’s actually contained on the inside. Clearly, that would be a mockery of the concept of Risk Management. If CMS doesn’t provide any minimum standards for the proper composition of an IQCP, it’s not a stretch to think that a laboratory may generate a document or fill in a “canned” worksheet that, while inadequate, will nevertheless be compliant.
The myth of choice in Risk QC Plans
It has been stated repeatedly that choosing EP23A and Risk Management IQCPs is a choice. Laboratories are not required to make changes to their standard QC practices. Similarly, the laboratory can choose the level of effort they invest in an IQCP. We don’t expect laboratories to purposely do a poor job in Risk Management, so the hope is that they will choose to assess the risks correctly and generate an appropriate IQCP.
In a similar vein, whether or not a laboratory even decides to use Risk QC at all is a choice. A laboratory will always be able to stick with the “default QC” option of 2 controls once a day. So if a laboratory chooses to implement EP23A, that’s their choice. Laboratories that don’t like EP23 and the concept of Risk QC / IQCP can continue to perform the default approach. However, laboratories that had implemented EQC policies face a more difficult choice: they must return to default QC or implement Risk QC. For those laboratories, which have already reduced their QC frequency based on the flawed protocols of “Equivalent QC”, it will represent a tough choice: implementing Risk QC will be a steep learning curve, but returning to default QC will mean a large increase in the amount of QC they have to perform.
History tells us that laboratories will likely choose the path of least resistance. When CLIA set minimum quality requirements back in 1992, those became the effective maximum amount of QC that a laboratory would perform. A combination of production pressures and a compliance mentality helped drive QC practices to this lowered threshold. When CLIA set the bar low for compliance, laboratories lowered their practices to meet it. Witness how few laboratories out there do more than the minimum QC required by CLIA. Professional standards are nice, but the bottom line is the bottom line, and cost conscious managers won’t “over-invest” in quality processes if they aren’t “necessary.” So in both of the choices above, you can predict that laboratories will (1) choose to implement Risk QC on their tests where they were previously using EQC, and (2) apply the least amount of effort necessary in generating their IQCPs.
Perhaps this is a cynical prediction, or simply a bitterly realistic one. Having traveled to many parts of the world and talked to thousands of laboratory professionals from a variety of settings, I’m still pretty sure this take is dead on. I’ll be the first to admit that there is bias in my sample, but whenever I discuss problems in the laboratory, the agreement is global: our routine practices are far from ideal, and it’s naïve to presume that we won’t experience the same problems with Risk QC. Laboratories struggling with tight budgets and under-staffing are not likely to have the willpower to insist on quality above the compliance threshold. True, there are exceptions to this rule, and I’ve been privileged to meet many professionals who set the bar far, far higher than simple compliance. But the majority of labs aren’t the exception or the exceptional, they are the silent, over-stressed, under-resourced majority. For those laboratories, giving them permission to use Bad Risk Management techniques may eliminate the possibility that they perform good QC.