NOTE: The following article is a revised and updated version of one of the chapters from the Six Sigma Quality Design and Control manual. Business and industry have demonstrated significant cost savings as a result of implementing Six Sigma Quality Management. Can you expect similar savings in healthcare applications? We think so and here’s why! The implementation of Six Sigma will actually save you money that is already being spent due to poor quality. This is money that is being wasted and therefore can be reclaimed. While it’s difficult to measure these savings, it’s easy to illustrate the magnitude of the savings that might be made. Within a laboratory, defective quality control procedures are already costing you money, particularly if you’re using the “tried and true” Levey-Jennings chart with 2 SD control limits. Careful design of QC procedures will reduce the false rejection of analytical runs which otherwise waste personnel time and laboratory materials and resources. Defective test results also cost your organization money when laboratory tests are misinterpreted and misused. Appropriate QC will assure the detection of medically important errors that would otherwise cause improper diagnosis and treatment of patients, wasting the time of physicians and nurses and consuming resources of the organization.
Business and industry have demonstrated significant cost savings as a result of implementing Six Sigma Quality Management. Can you expect similar savings in healthcare applications? We think so and here’s why! The implementation of Six Sigma will actually save you money that is already being spent due to poor quality. This is money that is being wasted and therefore can be reclaimed. While it’s difficult to measure these savings, it’s easy to illustrate the magnitude of the savings that might be made.
Within a laboratory, defective quality control procedures are already costing you money, particularly if you’re using the “tried and true” Levey-Jennings chart with 2 SD control limits. Careful design of QC procedures will reduce the false rejection of analytical runs which otherwise waste personnel time and laboratory materials and resources. Defective test results also cost your organization money when laboratory tests are misinterpreted and misused. Appropriate QC will assure the detection of medically important errors that would otherwise cause improper diagnosis and treatment of patients, wasting the time of physicians and nurses and consuming resources of the organization.
These savings are understandable if you consider all the costs associated with quality. These costs are described by the industrial model for “quality-costs”[1], which includes the costs of good quality (preventive-costs and appraisal-costs) and the costs of poor quality (internal and external failure-costs), as shown below. In industry, the costs of good quality are the planning and design of the process, the training of the line workers, and the time and effort in measuring and monitoring the quality of the product. The costs of poor quality are the rework and waste of a production process – doing things over to get the product right, or scrapping the product altogether. For a laboratory, the costs of good quality include the cost of planning for quality (a preventive-cost) and the cost doing QC (an appraisal-cost). The cost of doing QC is probably the only cost the laboratory keeps track of. The costs of poor quality include the wasted time, effort, and materials due to repeat analysis of controls, new controls, and patients (internal failure-costs) and the cost of repeat orders and additional test orders by the physician to confirm laboratory results, as well as the costs associated with the wrong diagnosis and treatment due to erroneous test results (external failure-costs). These failure-costs would be high if they were measured in laboratories and hospitals, but they usually aren’t. Careful planning of QC minimizes internal failure-costs by reducing the false rejections of analytical runs and reduces external failure-costs by assuring that medically important errors are detected. We adopted the industrial quality-costs model in the mid-1980s [2] and that concept is embedded in our thinking and also in the quality-planning process and tools that have been developed since then.
These savings are understandable if you consider all the costs associated with quality. These costs are described by the industrial model for “quality-costs”[1], which includes the costs of good quality (preventive-costs and appraisal-costs) and the costs of poor quality (internal and external failure-costs), as shown below. In industry, the costs of good quality are the planning and design of the process, the training of the line workers, and the time and effort in measuring and monitoring the quality of the product. The costs of poor quality are the rework and waste of a production process – doing things over to get the product right, or scrapping the product altogether.
For a laboratory, the costs of good quality include the cost of planning for quality (a preventive-cost) and the cost doing QC (an appraisal-cost). The cost of doing QC is probably the only cost the laboratory keeps track of. The costs of poor quality include the wasted time, effort, and materials due to repeat analysis of controls, new controls, and patients (internal failure-costs) and the cost of repeat orders and additional test orders by the physician to confirm laboratory results, as well as the costs associated with the wrong diagnosis and treatment due to erroneous test results (external failure-costs). These failure-costs would be high if they were measured in laboratories and hospitals, but they usually aren’t.
Careful planning of QC minimizes internal failure-costs by reducing the false rejections of analytical runs and reduces external failure-costs by assuring that medically important errors are detected. We adopted the industrial quality-costs model in the mid-1980s [2] and that concept is embedded in our thinking and also in the quality-planning process and tools that have been developed since then.
Poor QC is costing you money right now. It’s a little known and oft-denied fact that most laboratories in the US and around the world are performing poor QC. I’ve heard too many personal anecdotes and read too many studies to say otherwise. I’m sorry to be the bearer of bad news, but someone has to speak out. Before you protest, “not in my laboratory,” answer these questions honestly: Does your laboratory still use 2 SD control limits? Does your laboratory repeat controls again and again, analyze new controls, and then re-analyze the new controls until they’re finally “in”? If so, most likely you have a false rejection problem that is costing you time and money. It’s also a dangerous practice because analysts get accustomed to false alarms and no longer respond to true alarms, which leads to the reporting of erroneous test results. These actions are a result of poor quality planning, are costing you time and money, and are most likely giving your laboratory a bad reputation. We must realize that many of the repeat runs, repeat controls, and new controls that we use to replace “bad” controls are the result of false rejection. The reason we don’t immediately reject the run, but take these other actions first, is because we know instinctively that there really wasn’t something wrong with the run in the first place. There isn’t a problem with the samples – there’s a problem with the control rules being used to detect the problem. Here’s a table with some hopefully eye-opening numbers about false rejection: Control Rules Number of Controls per Run 1 2 3 4 12s 5% 9% 14% 18% 12.5s 1% 2% 3% 4% 13s 0% 0% 1% 1% 13.5s 0% 0% 0% 0% 13s/22s/R4s 0% 1% 2% 2% Notice anything startling about the first row of this table? The 12s rule (i.e., 2 SD control limits) has a terribly high level of false rejections. There’s no wonder this rule causes so many rejections and repeat runs. If you’re running 2 controls and everything is working perfectly, you should be getting an out-of-control flag almost once in every ten runs (9%); with 3 controls, it’s one out of every 6 or 7 runs; with 4 controls, it should be one out of every 5 runs. If there’s actually a problem with the method, there will be even more flags, but who can tell a real rejection from a false rejection? For those who aren’t yet convinced that this is a serious problem, let’s put the same condition into another context. If your laboratory has a fire alarm that goes off once a week even though there’s no fire, would it save the laboratory time and money to get a better fire detector – one that only goes off when there’s actually a real fire in the building? Obviously, a more reliable alarm or detector would save a lot of time if we’ve been evacuating the laboratory once a week. If we aren’t evacuating the laboratory, then we’re running the risk of a terribly high cost if a real fire occurs.
Poor QC is costing you money right now. It’s a little known and oft-denied fact that most laboratories in the US and around the world are performing poor QC. I’ve heard too many personal anecdotes and read too many studies to say otherwise. I’m sorry to be the bearer of bad news, but someone has to speak out.
Before you protest, “not in my laboratory,” answer these questions honestly: Does your laboratory still use 2 SD control limits? Does your laboratory repeat controls again and again, analyze new controls, and then re-analyze the new controls until they’re finally “in”? If so, most likely you have a false rejection problem that is costing you time and money. It’s also a dangerous practice because analysts get accustomed to false alarms and no longer respond to true alarms, which leads to the reporting of erroneous test results. These actions are a result of poor quality planning, are costing you time and money, and are most likely giving your laboratory a bad reputation.
We must realize that many of the repeat runs, repeat controls, and new controls that we use to replace “bad” controls are the result of false rejection. The reason we don’t immediately reject the run, but take these other actions first, is because we know instinctively that there really wasn’t something wrong with the run in the first place. There isn’t a problem with the samples – there’s a problem with the control rules being used to detect the problem.
Here’s a table with some hopefully eye-opening numbers about false rejection:
For those who aren’t yet convinced that this is a serious problem, let’s put the same condition into another context. If your laboratory has a fire alarm that goes off once a week even though there’s no fire, would it save the laboratory time and money to get a better fire detector – one that only goes off when there’s actually a real fire in the building? Obviously, a more reliable alarm or detector would save a lot of time if we’ve been evacuating the laboratory once a week. If we aren’t evacuating the laboratory, then we’re running the risk of a terribly high cost if a real fire occurs.
If you’re still not convinced, then maybe some actual figures in $$$$ will be persuasive. Look at the next table, where the yearly costs of repeat runs have been calculated. The first column shows how many runs (from 1 to 4) are performed per day. The second column shows the total runs per year, assuming the laboratory operates all 365 days per year. The probability of false rejection is given in the 3rd column (0.09 for 2 control measurements per run and 0.14 for 3 per run) and is multiplied by the number of runs per year to give the extra runs needed, as shown in column 4. The remaining columns show the failure-costs or waste due to false rejections for 1 test or method, 5 methods, and 20 methods. The cost of analysis per sample is assumed to be $0.50. Sections A and B estimate the costs of repeating all patients and controls, assuming there is an average of 20 patients per run. Sections C and D show the costs if only the controls are repeated. Quality-Costs I: Internal Failure-Costs (Waste & Rework) Runs/day Runs/year Pfr Ex.runs/year Cost/run 1 method 5 methods 20 methods A. Cost of repeating run of 20 specimens and 2 controls when cost of each is $0.50 1 365 0.09 32.85 $ 11.00 $ 361.35 $ 1,806.75 $ 7,227.00 2 730 0.09 65.7 $ 11.00 $ 722.70 $ 3,613.50 $ 14,454.00 3 1095 0.09 98.55 $ 11.00 $ 1,084.05 $ 5,420.25 $ 21,681.00 4 1460 0.09 131.4 $ 11.00 $ 1,445.40 $ 7,227.00 $ 28,908.00 B. Cost of repeating run of 20 specimens and 3 controls when cost of each is $0.50 1 365 0.14 51.1 $ 11.50 $ 587.65 $ 2,938.25 $ 11,753.00 2 730 0.14 102.2 $ 11.50 $ 1,175.30 $ 5,876.50 $ 23,506.00 3 1095 0.14 153.3 $ 11.50 $ 1,762.95 $ 8,814.75 $ 35,259.00 4 1460 0.14 204.4 $ 11.50 $ 2,350.60 $11,753.00 $ 47,012.00 C. Cost of repeating 2 controls only when each control costs $0.50 1 365 0.09 32.85 $ 1.00 $ 32.85 $ 164.25 $ 657.00 2 730 0.09 65.7 $ 1.00 $ 65.70 $ 328.50 $ 1,314.00 3 1095 0.09 98.55 $ 1.00 $ 98.55 $ 492.75 $ 1,971.00 4 1460 0.09 131.4 $ 1.00 $ 131.40 $ 657.00 $ 2,628.00 D. Cost of repeating 3 controls only when each control costs $0.50 1 365 0.14 51.1 $ 1.50 $ 76.65 $ 383.25 $ 1,533.00 2 730 0.14 102.2 $ 1.50 $ 153.30 $ 766.50 $ 3,066.00 3 1095 0.14 153.3 $ 1.50 $ 229.95 $ 1,149.75 $ 4,599.00 4 1460 0.14 204.4 $ 1.50 $ 306.60 $ 1,533.00 $ 6,132.00 Let’s assume that the laboratory adheres to the QC procedure religiously, i.e, both patients and controls are repeated whenever there’s a rejection. For a small laboratory that performs only a single test or method once a day, the cost is $361 per year. If the laboratory performed 4 different tests, the cost would be $1,806; if there were 2 runs per day, the costs would increase to $3,613. For a laboratory that performed 20 different tests or for an instrument that performs 20 different tests, the cost per year is $7,227 for a single run per day, $14,454 if there are both morning and afternoon runs, and $28,908 if the laboratory operated around the clock, performing a third run on second shift and a fourth run on 3rd shift. For 3 controls per run instead of 2, the cost would be $47,012. If you wondered why many laboratories don’t repeat the patient samples when a run is out-of-control, now you know – the cost is too high. They keep the costs down by repeating only the controls, as shown in sections C and D of the table. But even those costs run into the thousands. It would be better, of course, to utilize a QC procedure that has a lower false rejection rate. A multirule procedure with Ns of 2 and 3 would have only 1-2% false rejections, rather than the 9-14% for 2 SD control limits. You can see that a lot of money is wasted by common practice of using 2 SD control limits. At the same time, you can also see how the false rejection problem can be reduced by selecting other QC procedures, such as a 13s, 12.5s, or multirule procedure. You can do the math and figure out the savings. In summary, the costs of false rejections are a big problem in many laboratories. This cost can be reduced by selecting QC procedures that have a low probability for false rejection. Any laboratory that uses 2 SD control limits will experience false rejections; if not, they’re somehow widening their control limits, perhaps by using the manufacturers’ limits, bottle values that include lab to lab variation, supposedly clinical limits, or even drawing the CLIA total error criteria directly on control charts – all bad practices.
If you’re still not convinced, then maybe some actual figures in $$$$ will be persuasive.
Look at the next table, where the yearly costs of repeat runs have been calculated. The first column shows how many runs (from 1 to 4) are performed per day. The second column shows the total runs per year, assuming the laboratory operates all 365 days per year. The probability of false rejection is given in the 3rd column (0.09 for 2 control measurements per run and 0.14 for 3 per run) and is multiplied by the number of runs per year to give the extra runs needed, as shown in column 4. The remaining columns show the failure-costs or waste due to false rejections for 1 test or method, 5 methods, and 20 methods. The cost of analysis per sample is assumed to be $0.50. Sections A and B estimate the costs of repeating all patients and controls, assuming there is an average of 20 patients per run. Sections C and D show the costs if only the controls are repeated.
Let’s assume that the laboratory adheres to the QC procedure religiously, i.e, both patients and controls are repeated whenever there’s a rejection. For a small laboratory that performs only a single test or method once a day, the cost is $361 per year. If the laboratory performed 4 different tests, the cost would be $1,806; if there were 2 runs per day, the costs would increase to $3,613. For a laboratory that performed 20 different tests or for an instrument that performs 20 different tests, the cost per year is $7,227 for a single run per day, $14,454 if there are both morning and afternoon runs, and $28,908 if the laboratory operated around the clock, performing a third run on second shift and a fourth run on 3rd shift. For 3 controls per run instead of 2, the cost would be $47,012. If you wondered why many laboratories don’t repeat the patient samples when a run is out-of-control, now you know – the cost is too high. They keep the costs down by repeating only the controls, as shown in sections C and D of the table. But even those costs run into the thousands.
It would be better, of course, to utilize a QC procedure that has a lower false rejection rate. A multirule procedure with Ns of 2 and 3 would have only 1-2% false rejections, rather than the 9-14% for 2 SD control limits. You can see that a lot of money is wasted by common practice of using 2 SD control limits. At the same time, you can also see how the false rejection problem can be reduced by selecting other QC procedures, such as a 13s, 12.5s, or multirule procedure. You can do the math and figure out the savings.
In summary, the costs of false rejections are a big problem in many laboratories. This cost can be reduced by selecting QC procedures that have a low probability for false rejection. Any laboratory that uses 2 SD control limits will experience false rejections; if not, they’re somehow widening their control limits, perhaps by using the manufacturers’ limits, bottle values that include lab to lab variation, supposedly clinical limits, or even drawing the CLIA total error criteria directly on control charts – all bad practices.
Another problem is that many laboratories really don’t know if their QC procedures will detect medically important errors. A lack of rejections may mean the method is working perfectly, or it may reflect that the QC procedure is unable to detect errors. If that occurs, the laboratory may be reporting erroneous test results, which could lead to improper diagnoses and treatments of patients. The potential cost of erroneous test results are hard to quantify – they might be small, but they could also be huge. Let’s consider two situations where erroneous test results cause a delay in treatment. In the first situation, perhaps an outpatient setting, the delay costs $100 in additional time and effort to get things right. In the second situation, the delay costs $1000 for an additional day in the hospital. The next table shows what happens when there are medically important errors that go undetected in the laboratory. Quality-Costs II: External Failure-Costs Runs/Year Patients/Run Error Frequency Ped Errors NOT Detected Cost/error Failure-cost A. Lab Cost of each error = $100 due to delay, retesting, & extra effort 365 20 0.01 0.9 7.3 $100.00 $730.00 365 20 0.01 0.5 36.5 “ $3,650.00 365 20 0.01 0.2 58.4 “ $5,840.00 365 20 0.05 0.9 36.5 $100.00 $3,650.00 365 20 0.05 0.5 182.5 “ $18,250.00 365 20 0.05 0.2 292 “ $29,200.00 730 20 0.01 0.9 14.6 $100.00 $1,460.00 730 20 0.01 0.5 73 “ $7,300.00 730 20 0.01 0.2 116.8 “ $11,680.00 730 20 0.05 0.9 73 $100.00 $7,300.00 730 20 0.05 0.5 365 “ $36,500.00 730 20 0.05 0.2 584 “ $58,400.00 B. Hospital Cost of each error is $1000 due to extra day in the hospital 365 20 0.01 0.9 7.3 $1,000.00 $ 7,300.00 365 20 0.01 0.5 36.5 “ $36,500.00 365 20 0.01 0.2 58.4 “ $58,400.00 365 20 0.05 0.9 36.5 $1,000.00 $6,500.00 365 20 0.05 0.5 182.5 “ $182,500.00 365 20 0.05 0.2 292 “ $292,000.00 730 20 0.01 0.9 14.6 $1,000.00 $14,600.00 730 20 0.01 0.5 73 “ $73,000.00 730 20 0.01 0.2 116.8 “ $116,800.00 730 20 0.05 0.9 73 $1,000.00 $ 73,000.00 730 20 0.05 0.5 365 “ $365,000.00 730 20 0.05 0.2 584 “ $584,000.00 The first two columns show the number of runs per year and number of patients per run, which are similar to the earlier table. Column 3 shows the frequency of errors that occur with the analytical method, e.g., a figure of 0.01 means that 1 run out of 100 has a problem that leads to medically important errors; a figure of 0.05 means that 1 run out of 20 has a problem. Column 4 shows the probability of error detection for a laboratory’s QC procedure, e.g., 0.90 means that 90% of runs with medically important errors would be detected; 0.50 means that 50% of runs would be detected; 0.20 means that 20% of runs would be detected. Column 5 is the number of errors that go undetected. Column 6 is the cost per error and column 7 the total failure-cost per year. As you can see, a delay in treatment can lead to a significant waste in the resources of a healthcare organization. The worst case in the outpatient setting is almost $60,000 per year and we’re only talking about a single test that is not done well. The worst case in the hospital setting is almost $600,000 per year, and again we’re only talking about a single test. It’s easy to see that analytical errors could cost hundreds of thousands, even millions of dollars. That cost affects the overall bottom line of the healthcare organization. No one really knows what this cost is in a particular organization. As Deming said, there are some figures that are unknowable, but they must still be considered. Laboratories should not knowingly operate with poor quality and poor quality control. The secret to knowing about quality is the Six Sigma techniques and the associated planning tools like the OPSpecs chart. The cornerstone of quality-planning is the selection of a QC procedure that will detect medically important errors. Built into the quality-planning tools, graphs, and calculations is the selection of a QC procedure on the basis of the quality required by the test and the imprecision and inaccuracy observed for the method; the process accomplishes this at the lowest cost by minimizing the probability for false rejection and the number of control measurements. Thus, the quality-planning process optimizes your QC so you aren’t over-controlling a test and wasting resources, or under-controlling a test and suffering potentially high failure-costs. You do the right amount of QC for the test and for your method, with the assurance that you will detect analytical errors that would otherwise have a costly impact on patient care. Optimizing your QC allows you to spend only the minimum amount on QC, while maximizing your “through-put” of tests. With fewer false rejects, you increase the productivity or “test-yield” of your processes; you can run more tests and have more staff time to devote to other tasks. With better error detection, you have fewer repeat tests ordered by puzzled physicians; you reduce medical errors and contribute to better patient care. If we could speak a moment in business terms, the quality-planning process leads to improved productivity, reduced costs, and dare we say, a competitive advantage. In summary, the current QC practices in laboratories today contribute to two kinds of financial waste through high false rejection and low error detection. The quality-planning process will help you choose control rules that are high in error detection and low in false rejection, optimizing your quality and minimizing your costs.
Another problem is that many laboratories really don’t know if their QC procedures will detect medically important errors. A lack of rejections may mean the method is working perfectly, or it may reflect that the QC procedure is unable to detect errors. If that occurs, the laboratory may be reporting erroneous test results, which could lead to improper diagnoses and treatments of patients. The potential cost of erroneous test results are hard to quantify – they might be small, but they could also be huge.
Let’s consider two situations where erroneous test results cause a delay in treatment. In the first situation, perhaps an outpatient setting, the delay costs $100 in additional time and effort to get things right. In the second situation, the delay costs $1000 for an additional day in the hospital. The next table shows what happens when there are medically important errors that go undetected in the laboratory.
$100.00
$730.00
“
$3,650.00
$5,840.00
$18,250.00
$29,200.00
$1,460.00
$7,300.00
$11,680.00
$36,500.00
$58,400.00
$1,000.00
$ 7,300.00
$6,500.00
$182,500.00
$292,000.00
$14,600.00
$73,000.00
$116,800.00
$ 73,000.00
$365,000.00
$584,000.00
The first two columns show the number of runs per year and number of patients per run, which are similar to the earlier table. Column 3 shows the frequency of errors that occur with the analytical method, e.g., a figure of 0.01 means that 1 run out of 100 has a problem that leads to medically important errors; a figure of 0.05 means that 1 run out of 20 has a problem. Column 4 shows the probability of error detection for a laboratory’s QC procedure, e.g., 0.90 means that 90% of runs with medically important errors would be detected; 0.50 means that 50% of runs would be detected; 0.20 means that 20% of runs would be detected. Column 5 is the number of errors that go undetected. Column 6 is the cost per error and column 7 the total failure-cost per year.
As you can see, a delay in treatment can lead to a significant waste in the resources of a healthcare organization. The worst case in the outpatient setting is almost $60,000 per year and we’re only talking about a single test that is not done well. The worst case in the hospital setting is almost $600,000 per year, and again we’re only talking about a single test. It’s easy to see that analytical errors could cost hundreds of thousands, even millions of dollars. That cost affects the overall bottom line of the healthcare organization. No one really knows what this cost is in a particular organization. As Deming said, there are some figures that are unknowable, but they must still be considered.
Laboratories should not knowingly operate with poor quality and poor quality control. The secret to knowing about quality is the Six Sigma techniques and the associated planning tools like the OPSpecs chart. The cornerstone of quality-planning is the selection of a QC procedure that will detect medically important errors. Built into the quality-planning tools, graphs, and calculations is the selection of a QC procedure on the basis of the quality required by the test and the imprecision and inaccuracy observed for the method; the process accomplishes this at the lowest cost by minimizing the probability for false rejection and the number of control measurements. Thus, the quality-planning process optimizes your QC so you aren’t over-controlling a test and wasting resources, or under-controlling a test and suffering potentially high failure-costs. You do the right amount of QC for the test and for your method, with the assurance that you will detect analytical errors that would otherwise have a costly impact on patient care.
Optimizing your QC allows you to spend only the minimum amount on QC, while maximizing your “through-put” of tests. With fewer false rejects, you increase the productivity or “test-yield” of your processes; you can run more tests and have more staff time to devote to other tasks. With better error detection, you have fewer repeat tests ordered by puzzled physicians; you reduce medical errors and contribute to better patient care. If we could speak a moment in business terms, the quality-planning process leads to improved productivity, reduced costs, and dare we say, a competitive advantage.
In summary, the current QC practices in laboratories today contribute to two kinds of financial waste through high false rejection and low error detection. The quality-planning process will help you choose control rules that are high in error detection and low in false rejection, optimizing your quality and minimizing your costs.
Just as there’s no magic set of control rules for all tests and all laboratories, there’s no one magic way to save money on all tests in all laboratories. Most likely there are tests that are being over-controlled (e.g., highly automated chemistry and hematology analyzers) and other tests that are being under-controlled (e.g., immunoassays, POC, manual assays). That’s the outcome from using the same QC procedure on all the tests in the laboratory. With careful selection of QC procedure, you will save money in some areas while possibly spending more in other areas. Ultimately, what’s important is that you’ll be doing the right amount of QC on all of your tests, thereby making the optimal investment of time, effort, and materials. There are some laboratories doing such poor QC, or so little QC, that the quality-planning process may require them to do more QC for all tests, which will inevitably cost them more than what they’re spending now. But if a laboratory is operating that poorly, it’s imperative to make an investment in better QC. There are also some tests that are very difficult to control because current analytical methods lack the desired performance. Those methods will require extra control measurements together with the most powerful control rules, implementation of non-statistical checks and an aggressive preventive maintenance schedule, increased education and training, staffing with more experienced analysts, and less rotation of staff. Is it worth implementing Six Sigma if you won’t always save money? For most of us who have dedicated ourselves to clinical laboratory services, the knowledge of the quality of our tests is the key to managing the laboratory services effectively. There’s no satisfaction in providing cheap test results if those results might be in error! We’d rather know if there are analytical problems so we have a chance to solve those problems. We’d rather assure the quality of the test results instead of assuming the quality is okay. We’re healthcare professionals who take responsibility for the quality of our services!
Just as there’s no magic set of control rules for all tests and all laboratories, there’s no one magic way to save money on all tests in all laboratories. Most likely there are tests that are being over-controlled (e.g., highly automated chemistry and hematology analyzers) and other tests that are being under-controlled (e.g., immunoassays, POC, manual assays). That’s the outcome from using the same QC procedure on all the tests in the laboratory. With careful selection of QC procedure, you will save money in some areas while possibly spending more in other areas. Ultimately, what’s important is that you’ll be doing the right amount of QC on all of your tests, thereby making the optimal investment of time, effort, and materials.
There are some laboratories doing such poor QC, or so little QC, that the quality-planning process may require them to do more QC for all tests, which will inevitably cost them more than what they’re spending now. But if a laboratory is operating that poorly, it’s imperative to make an investment in better QC. There are also some tests that are very difficult to control because current analytical methods lack the desired performance. Those methods will require extra control measurements together with the most powerful control rules, implementation of non-statistical checks and an aggressive preventive maintenance schedule, increased education and training, staffing with more experienced analysts, and less rotation of staff.
Is it worth implementing Six Sigma if you won’t always save money? For most of us who have dedicated ourselves to clinical laboratory services, the knowledge of the quality of our tests is the key to managing the laboratory services effectively. There’s no satisfaction in providing cheap test results if those results might be in error! We’d rather know if there are analytical problems so we have a chance to solve those problems. We’d rather assure the quality of the test results instead of assuming the quality is okay. We’re healthcare professionals who take responsibility for the quality of our services!
We’ve included a set of cost worksheets in this chapter to help you get rough estimates on the amount of money you can save by improving your quality design and control. There are no guarantees in life, and I can’t promise that these worksheets can take into account all the factors involved in Six Sigma Quality Management. However, I hope these worksheets help you get started. Even a rough estimate of the waste generated by poor quality is illuminating. I’m sure if you plug in a few numbers, you’ll be motivated to make improvements. If you’re truly going to calculate the cost savings from improved QC, you’ll need to fill out a set of worksheets for your current practices, then work through the quality-planning process to select a more appropriate QC procedure, and fill out the worksheets based on the new QC procedure. Then you can see the difference between your current QC costs and your potential QC costs. The savings you can realize can be determined by • (Worksheet I) Internal Failure Cost Savings = Current – New • (Worksheet II) External Failure Cost Savings = Current – New Working through the quality-planning process and selecting a new QC procedure can be accomplished in a variety of ways: (1) By using a manual method like Normalized OPSpecs charts (2) By using online quality-planning tools at http://www.westgard.com/qptools.html or (3) By using software support such as QC Validator® 2.0 or EZ Rules™. If you don’t work through the quality-planning process, then you will only be able to determine your current costs. You won’t be able to compare those costs to your potential future costs, so you won’t know the size of your potential savings, etc. Identifying the cost of a wasteful QC procedure is important, but ultimately it’s only useful if you replace it with a more efficient QC procedure. So remember to take the next step: use the quality-planning process to find a more efficient QC procedure that meets the quality required by the test.
We’ve included a set of cost worksheets in this chapter to help you get rough estimates on the amount of money you can save by improving your quality design and control. There are no guarantees in life, and I can’t promise that these worksheets can take into account all the factors involved in Six Sigma Quality Management. However, I hope these worksheets help you get started. Even a rough estimate of the waste generated by poor quality is illuminating. I’m sure if you plug in a few numbers, you’ll be motivated to make improvements.
If you’re truly going to calculate the cost savings from improved QC, you’ll need to fill out a set of worksheets for your current practices, then work through the quality-planning process to select a more appropriate QC procedure, and fill out the worksheets based on the new QC procedure. Then you can see the difference between your current QC costs and your potential QC costs.
The savings you can realize can be determined by • (Worksheet I) Internal Failure Cost Savings = Current – New • (Worksheet II) External Failure Cost Savings = Current – New
Working through the quality-planning process and selecting a new QC procedure can be accomplished in a variety of ways: (1) By using a manual method like Normalized OPSpecs charts (2) By using online quality-planning tools at http://www.westgard.com/qptools.html or (3) By using software support such as QC Validator® 2.0 or EZ Rules™.
If you don’t work through the quality-planning process, then you will only be able to determine your current costs. You won’t be able to compare those costs to your potential future costs, so you won’t know the size of your potential savings, etc. Identifying the cost of a wasteful QC procedure is important, but ultimately it’s only useful if you replace it with a more efficient QC procedure. So remember to take the next step: use the quality-planning process to find a more efficient QC procedure that meets the quality required by the test.
This worksheet doesn’t require any hard work. Just pick a test and start filling out the worksheet. You should already know the answers to these simple questions: How many tests are in a run? How many runs are there in a year of operation? What control rule are you using? If you don’t know these details off the top of your head, they should be available somewhere in your laboratory. The only piece of information that you might not already have is presented in the table below: the false rejection rate of the control rule that you’re using for the test. You probably know what control rule you are using and how many controls you are running. For example, if you are using the 2s control rule and running 2 controls, you will look at the intersection of the first row (12s) and the second column (2) and find the number 9%. This means that 9% of the runs performed on this test will be falsely rejected (i.e. rejected even though there is nothing wrong with them). This cost can’t be reduced, unless you use a better control rule! False Rejection Rates of Common Control Rules Quality Cost Worksheet I: Internal Failure Costs (Waste and Rework) Example Worksheet: Cholesterol with 12s control rule Example Worksheet: Cholesterol with "Westgard Rules" The conclusion of the Internal Failure Worksheet is clear: the 12s control rule wastes a lot of money. Switching to the "Westgard Rules" can provide an immediate reduction in waste and rework, because the multirule QC procedure has much a lower false rejection rate.
This worksheet doesn’t require any hard work. Just pick a test and start filling out the worksheet. You should already know the answers to these simple questions: How many tests are in a run? How many runs are there in a year of operation? What control rule are you using? If you don’t know these details off the top of your head, they should be available somewhere in your laboratory.
The only piece of information that you might not already have is presented in the table below: the false rejection rate of the control rule that you’re using for the test. You probably know what control rule you are using and how many controls you are running. For example, if you are using the 2s control rule and running 2 controls, you will look at the intersection of the first row (12s) and the second column (2) and find the number 9%. This means that 9% of the runs performed on this test will be falsely rejected (i.e. rejected even though there is nothing wrong with them).
This cost can’t be reduced, unless you use a better control rule!
False Rejection Rates of Common Control Rules
The conclusion of the Internal Failure Worksheet is clear: the 12s control rule wastes a lot of money. Switching to the "Westgard Rules" can provide an immediate reduction in waste and rework, because the multirule QC procedure has much a lower false rejection rate.
Estimating Error Frequency Estimating the frequency of errors should be straightforward if you have previous records about the test and method. The number of true out-of-control flags (remember, discount false alarms) as a percentage of the total number of runs will give you the frequency of errors. If you don’t know for sure what the frequency of errors is for a good test, use this rule of thumb: An excellent method has only about a 1% occurrence of errors. A normal method has somewhere between 1 and 5% occurrence of errors. A poor method has more than 5% occurrence of errors. Start with an estimate of the error frequency and see what the numbers say. Calculating Error Detection, Ped You need to do some work in order to figure this out. In order to accurately find the error detection you’re achieving with your current QC procedure, you need to go through the quality-planning process. You will need to use an OPSpecs chart and possibly a critical-error graph. As stated elsewhere, these graphic tools are available in manuals, as online tools, and as software. If you know at least the analytical quality requirement, the imprecision, and inaccuracy, you can use an OPSpecs chart to select a QC procedure. The OPSpecs charts are commonly available for 90%, 50% and 25% error detection. Obviously, it’s best to find a control rule on an OPSpecs chart for 90% error detection. And if you find a control rule above your operating point on that chart, you’ve succeeded and can enter 0.90 in the worksheet. If you use a critical-error graph, you’ll still need the same details as the OPSpecs chart — but you’ll get a specific number for error detection, not just 0.90 or 0.50 or 0.25. Then you can enter that number in the worksheet. If you really must, start with an estimate of 0.90 in the worksheet. This will give you the “best case” scenario for your external failure costs. But even the best case may surprise you... Quality Cost Worksheet II: External Failure Costs Example Worksheet: Cholesterol with current method (assuming inadequate error detection) Example Worksheet: Cholesterol with "Westgard Rules" The conclusion of the External Failure Worksheets for this example: switching from a 12s control rule with low error detection to the "Westgard Rules" dramatically reduces the possible (or existing but hidden) costs. But even a control rule with 90% error detection is not cost-free. Striving toward Six Sigma performance can reduce these costs even more.
Estimating the frequency of errors should be straightforward if you have previous records about the test and method. The number of true out-of-control flags (remember, discount false alarms) as a percentage of the total number of runs will give you the frequency of errors.
If you don’t know for sure what the frequency of errors is for a good test, use this rule of thumb: An excellent method has only about a 1% occurrence of errors. A normal method has somewhere between 1 and 5% occurrence of errors. A poor method has more than 5% occurrence of errors. Start with an estimate of the error frequency and see what the numbers say.
Calculating Error Detection, Ped
You need to do some work in order to figure this out. In order to accurately find the error detection you’re achieving with your current QC procedure, you need to go through the quality-planning process. You will need to use an OPSpecs chart and possibly a critical-error graph. As stated elsewhere, these graphic tools are available in manuals, as online tools, and as software.
If you know at least the analytical quality requirement, the imprecision, and inaccuracy, you can use an OPSpecs chart to select a QC procedure. The OPSpecs charts are commonly available for 90%, 50% and 25% error detection. Obviously, it’s best to find a control rule on an OPSpecs chart for 90% error detection. And if you find a control rule above your operating point on that chart, you’ve succeeded and can enter 0.90 in the worksheet. If you use a critical-error graph, you’ll still need the same details as the OPSpecs chart — but you’ll get a specific number for error detection, not just 0.90 or 0.50 or 0.25. Then you can enter that number in the worksheet. If you really must, start with an estimate of 0.90 in the worksheet. This will give you the “best case” scenario for your external failure costs. But even the best case may surprise you...
The conclusion of the External Failure Worksheets for this example: switching from a 12s control rule with low error detection to the "Westgard Rules" dramatically reduces the possible (or existing but hidden) costs. But even a control rule with 90% error detection is not cost-free. Striving toward Six Sigma performance can reduce these costs even more.
Six Sigma Quality Management is a big job in a laboratory that performs hundreds of different tests. Where should you start? Begin with tests where you can make the biggest impact, such as automated chemistry and hematology analyzers that perform a significant portion of the laboratory workload. Place a high priority on any tests where the current QC practice uses 2 SD control limits.
1. Feigenbaum AV. Total quality control. Harvard Bus Rev 1954;34(6):93-101. 2. Westgard JO, Barry PL. Cost-Effective Quality Control: Managing the Quality and Productivity of Analytical Processes. Washington DC: AACC Press, 1986, pp 138-159.
If you would like to use these cost worksheets on applications in your laboratory, you can download a PDF of the worksheets and print out as many copies as you would like. Just fill out the form below:
Other Essays: