Six Sigma

It's 6/6/2023... are you still afraid of Six Sigma?

SoSS? (Scared of Six Sigma)
Confronting Six Fears of Quality Metrics

Sten Westgard, MS
June 2023

An old joke: Why is Six afraid of Seven? Because 7…8…9 (this works only in English).
A new phenomenon: Why are laboratories afraid of Six Sigma? Unfortunately, it’s not a joke.
As the pulse of scientific conferences quickens and begins to recapture that speed of pre-pandemic activity, I’ve been encountering more objections to Six Sigma. To help dispel some of the anxiety around Six Sigma metrics, I’m going to share some of the things I’ve heard this year.

1. Six Sigma is not possible in healthcare. No laboratory can actually achieve Six Sigma, and Six Sigma should not be applied to laboratory processes.

For a moment, let’s not worry about the phrase “Six Sigma.” Sometimes the brand name carries such baggage that we need to unpack before we can get anywhere. Six Sigma is just a catchy way of saying “3.4 defects per million opportunities.” It’s an error rate, and we deal with error rates in the laboratory every day, every hour, every second. We know errors happen, we don’t like it, we try to minimize them whenever and wherever possible. The Six Sigma scale simply makes it a bit more clear about what error rates you are experiencing as well as setting some goals for organizations.

Also note, Six Sigma does not ask for perfection or zero defect rates, just as close as possible to that. And even within the practice of Six Sigma, it’s accepted that some processes will never be able to achieve the Six. Instead, it’s often more practical to make improvements that can reduce the defect rate. For example, moving from 3 Sigma to 3.4 Sigma will cut your defects in half. So while perfection may be out of reach, climbing up the Sigma metric ladder will still bring benefits.

As for what Sigma metrics are possible in healthcare and the laboratory, respectfully, Six Sigma is possible in many processes, analytical, pre-analytical, and post-analytical. And the IFCC WG-LEPS recommends measuring Quality Indicators on the Six Sigma scale. Just one recent paper measured 15 quality metrics and found 33% of them were at the Six Sigma level.

[Source: Nergiz Zorbozan and Orçun Zorbozan, Evaluation of preanalytical and postanalytical phases in clinical biochemistry laboratory according to IFCC laboratory errors and patient safety specifications. Biochem Med (Zagreb). 2022 Oct 1; 32(3): 030701. Published online 2022 Aug 5. doi: 10.11613/BM.2022.030701 PMCID: PMC9344872 PMID: 35966260 ]

Moral: Even if you don’t believe Six Sigma is possible in healthcare, Six Sigma believes in you. Six Sigma is just a way to frame the error rates in a process. There’s nothing magical or mysterious about it that makes it impossible to use.

2. Six Sigma is not meant to be used on narrow process measurements. It should only be measured at the level of total testing process, or patient outcome process. Knowing the Sigma metric of one step - analytical quality - isn’t relevant or helpful.

The merit of this argument is that the patient doesn’t care whether or not you have a Six Sigma method - if your turn-around time is never. Success at one part of the total testing process can be lost, the gains wiped out, by poor performance elsewhere. If your laboratory has a hideous rate of hemolysis, the samples that reach your perfect analytical method are already compromised, and the result is less than 6 Sigma.

Nevertheless, we still gain by knowing where we are making errors, and can focus our efforts on making the greatest gains. And again, if we take the term “Six Sigma” out of the argument, it becomes clear that we shouldn’t stop measuring the error rates of processes and sub-processes. Should we stop measuring hemolysis, icterus, and lipemia separately? Should we stop measuring imprecision, or stop participating in PT programs? All of those error rates will only deliver a narrow view of the total testing process, and yet we benefit from knowing them. If we know we have a great analytical method, that helps us redirect our efforts toward other process steps that have more errors. If we know we have solved our hemolysis problems, we can address another challenge in the method.

Even if we are focused only on the last output, we still need to know the error rate or DPM of each sub-step.

Moral: Every improvement helps. If we can reduce many errors in one sub-step, that still helps the overall outcome. Debating about where we can make the biggest bang for our buck in improvements, that’s always up for debate. For analytical processes, there are low-hanging fruit that can deliver quick ROI.

3. Six Sigma is a flawed model.

Put aside the success that Six Sigma has demonstrated in many other industries and business sectors. The application of Six Sigma in medical laboratories, especially at the analytical phase, is unique. Still, the objection raised here is not truly about Six Sigma, it’s at heart a disagreement not with Six Sigma, but with the Total Allowable Error (TEa). A small coterie of critics have call this a “flawed model” primarily because bias is added as a linear element, instead of as a variance. In contrast, the measurement uncertainty (mu) approach, enshrined in ISO and exalted as the purest choice, requires the elimination of biases anywhere they are found. Of course, this is impossible to do in laboratory medicine. Bias exists, and it keeps popping up time and again. Trying to ignore bias is equally distasteful, so the mu proponents came up with another approach – they treat bias like any other variance, and add it together with all the other variances. That satisfies the mathematics, but distorts reality in the process.

Let’s translate a simple situation into a mu variance approach. This is an analogy first described by Hassan Bayat, our colleague. If the time of my commute to work is one hour with an uncertainty of +/- 5 minutes, I have mu of 5 minutes. But one day, when I’m an hour late, my lateness is not an uncertainty – my arrival times doesn’t have an uncertainty of one hour combined with an uncertainty of 5 minutes. Treating my commute as 1 hour +/-  1 hour and 12 seconds is a way to express uncertainty, but it’s also wrong on its face, because I can’t arrive earlier that I leave. Surely, we can agree this mu approach is at least equally flawed, if not more flawed, as the Total Error model.

Moral: George Box said, "All models are wrong. Some models are useful." Some models are more useful than others.

4. Six Sigma is heavily dependent on the total allowable error.

How you measure success is always dependent on the benchmarks you choose. Football success is entirely dependent on the size of the goal. Make it larger, as anyone who has watched a strike on the post knows, you’ll change the outcome of many matches. Make it smaller, and you’ll have more draws, more nil-nil games, and less excitement in the leagues.

This argument has more validity in expressing the frustration many suffer from the lack of standardization of performance specifications across the world and across different programs. The RCPA and the Rilibaek don’t agree. The CLIA and the EFLM don’t agree. We have a hierarchy that explicitly defines three distinct types of performance specifications, and acknowledges that analytes will have different goals from different levels. There is no grand unification of performance specifications in our future. If anything, we’ve seen more confusion in the last year, as CLIA tightens goals while EFLM goals relaxes its default recommendation to minimum specifications.

It certainly detracts from a claim of Six Sigma to have to question, “But what goal did you use to calculate Sigma?” But that’s a problem we must learn to live with. Some goals are coming close to a global consensus, for example, HbA1c. But even if we agree today on the goal for an analyte, that’s not carved in stone. New measurement technologies, new clinical uses, and new ways to interpret results may emerge that will place new demands on the analyte. With HbA1c, we went from having an unreliable assay to having something with which we can now diagnosis diabetes, over the space of a few decades.

As with any statistic, we have to be wary of numbers presented without context or reference. For our part, we used to encourage laboratories to ask their vendors and prospective vendors for Six Sigma metrics. Now we advise laboratories to ask for the “raw” data, and calculate the Sigma metrics themselves. Then you are in control of what goals you choose, and what metrics get calculated.

This is not a problem that is unique to the Six Sigma metric. There are different performance specifications for measurement uncertainty, too. When laboratories report their success rate for TAT, for example, notice how they might vary the success rate from 99% to 95% or 90%. When we look at a process, we must define the performance standard as well as the success rate.

Moral: Metrics are what you make them. Six Sigma is universal as a rate of success, but the goals are always evolving. With a little due diligence, you can find a metric benchmark you can compare across instruments and laboratories anywhere in the world.

5. If Six Sigma was really a problem for methods, manufacturers would do something about it.
6. If Six Sigma was really important, it would be mandated by CLIA or ISO 15189

There’s a validity to this argument, but also a learned helplessness. We complain about regulations, wishing to be left alone to make our own decisions and judgments. Many espouse that the regulations hold us back, that we should be unfettered in our exercise of our business. So why, in this case, are we throwing up our hands and refusing to make any decisions? We don’t trust the regulations to properly address so many things, but when it comes to quality benchmarking, now we long for the government or agency or international standard to force us to address quality?

Diagnostic companies are not perfect. Some of them try hard, but they make mistakes, they build methods that are not as good as they need to be, and they have every incentive to obscure the quality of those methods. If we acknowledge that diagnostic companies make errors, wouldn’t it be nice to be able to identify those errors before a recall occurs? Wouldn’t it be convenient to avoid purchasing a bad instrument entirely? If you have a tool that can predict which instruments and assays are going to generate more outliers, more repeats, more recalls, more trouble-shooting, more PT failures, more costs, and more lost time, wouldn’t you like to use it? Are you really waiting for a recall, or a consent decree, or a bankruptcy, before you decide to act?

CLIA is focused on minimum standards, and in its own way, ISO 15189 does the same. PT “success” can be achieved even with a failure rate of nearly 20%. ISO 15189, by avoiding any performance specifications or quantitative standards for success, leaves the door open for interpretation that can permit equally high error rates. You can be in compliance with CLIA and have poor Sigma metrics. You can be certified in ISO 15189 and have poor Sigma metrics.

Six Sigma is a higher bar than ISO 15189 or CLIA or CAP or Joint Commission. Both CLIA and ISO 15189 have a built-in incentive to generate as much success as possible in those who participate. It’s called the Regulator’s Dilemma – they can’t afford to put too many labs out of business. Again, for some analytes, like HbA1c, Six Sigma has been explicitly recommended.

There is an exhausted tone to these final objections. It’s not enough that the tool might improve success, cut costs, or save time. It has to be mandated, or the laboratory won’t follow it. Waiting passively to be forced to improve your laboratory guarantees your last-mover position in the marketplace. There’s an old saying, “We’re so tired of doing it wrong over and over, we don’t have the energy to do it right.” Whoever competes with you, if they have even a slightly better attitude, is going to beat you.

Laboratories are overwhelmed, we get it. There are a million things happening every day, and some of them are wrong, but there’s no bandwidth to grapple with the right corrective action. We hear that. But the acceptance of errors, knowing that these errors are going to occur, is a tragedy. The benefits of putting a little time and effort into benchmarking performance and redesigning QC will generate a return multiple times greater than the investment.

Moral: Waiting for mandates means ceding the marketplace to your competitors. Aim for excellence, you’ll always succeed in compliance. Focus only on the lowest of bars, you’ll always fail the next change in regulations.

The good news is that Six Sigma metrics are easier than ever, because control vendors like Bio-Rad and Technopath and Thermofisher/MAS have embedded the calculations in their peer group software. The data is gathered for you, the calculations are done for you, all you have to do is turn it on and reap the benefits.