|
|
|
|
December 2000
An updated version of this essay appears in the Nothing but the Truth about Quality manualJames O. Westgard, PhD, FACB
I just returned from the Spotlight on Quality conference in Antwerp. This conference is organized by Dr. Henk Goldschmidt from The Netherlands and Dr. Jean Claude Libeer from Belgium. It continues to provide some of the most stimulating discussions on issues in quality management. This year the focus was on the quality management model developed by the European Foundation for Quality Management (EFQM), which emphasizes important components of Total Quality Management, such as processes, the inputs to processes, and the results from processes. The EFQM model is a logical construct that shows the importance of leadership, people, policy and strategy, and partnerships and resources to implement and support processes. The output of processes is considered in the context of people results, customer results, society results, and key performance results. It's nice to see there still is a strong interest in TQM in Europe, even though the interest is waning in the US.
Dr. Goldschmidt began by asking some tough questions to challenge the logical thinking of the participants and to get them involved in the discussion.
- How do you put a giraffe in a refrigerator?
- How do you put an elephant in a refrigerator?
- When the lion called a meeting of the animal kingdom, who didn't participate?
Needless to say, the audience was at a loss for words and there were few participants willing to volunteer any answers. Dr. Goldschmidt noted that these questions had been asked in formal studies to evaluate logical reasoning. Most kids can answer these questions correctly, but not many adults. With more education, the questions are actually more difficult to answer.
What are your answers? Are these your final answers?
And the correct answers are...To put a giraffe in a refrigerator, you open the door, put the giraffe in, and close the door. To put an elephant in a refrigerator, you open the door, take the giraffe out, then put the elephant in. When the lion called the meeting of the animal kingdom, the elephant couldn't come because he was in the refrigerator.
Let's examine the some of our own reasoning about the important issue of laboratory errors. During the last decade, there has been a growing opinion that laboratory errors are no longer a problem. More specifically, analytical errors are no longer a problem and laboratories should focus their attention on pre-analytical and post-analytical sources of errors. Here's a sampling of some comments from laboratory scientists.
- "While it would be useful to define the amount of inaccuracy and imprecision that would limit the medical relevance of test results, the error inherent in the analytical phase of the total testing process will probably be considered acceptable for most clinical purposes…. Of greater concern is the location of errors within the total testing process, the frequency of their occurrence, and the impact of these errors on patient care…. Today, mistakes in the preanalytical and postanalytical phases of testing seem more likely to affect the usefulness of laboratory results in patient care than does variability in the analytical phase of testing." [Proceedings of the 1992 Clinical Chemistry Forum on "Accuracy, Precision Goals, and Medical Relevance", 1]
- "Most laboratory mistakes occur not during the analytical phase but before and after testing… For example, [in a review of] 363 incidents that occurred in a tertiary-care hospital in 1987,… preanalytical (missed or incorrectly interpreted laboratory orders, improper patient preparation, incorrect patient identification, wrong specimen container, and mislabeled or mishandled specimens) and postanalytical (delayed, unavailable, or incomplete results) mistakes account for 46% and 47% of the total incidents, respectively." [Proceedings of the 1996 AACC forum "Quality for Tomorrow", 2]
- "Laboratories rarely make an analytical error. As [the speaker above] commented, of all the major errors maybe only 7% are analytical. Is our focus on common causes of variability wrong as we talk about quality for tomorrow? When we investigate proficiency test errors or patients' reports errors, 95% of them are mistakes, not analytical errors." [Discussion included in the proceedings of the 1996 AACC forum "Quality for Tomorrow" forum, 2]
Emerging strategies and practices in analytical quality management in the US have been influenced very much by the opinions described above. Here are some examples of extension of these opinions about laboratory quality and extrapolations about how laboratory testing should be managed.
- Analytic performance is better than needed for medical relevance. WRONG! The logic requires that medically relevant performance be defined. Requirements for analytical performance were not defined in the past, therefore no conclusion can be drawn that analytical performance is better than needed. What is a medically relevant error? What difference between the reported test result and the correct value can be tolerated without invalidating the use and interpretation of the test result? These issues have to be addressed before any conclusion can be made that analytical performance is better than needed. Only in 1999 have we seen a consensus develop about how to define quality specifications for laboratory medicine [3].
- Laboratories should focus their efforts on pre and post analytic errors rather than analytic errors. WRONG! All errors are critical. Laboratories need to expand their monitoring to cover pre and post analytic errors, but must continue to monitor analytic errors.
- Laboratory QC efforts can be reduced. WRONG! The logic depends on the erroneous reasoning that analytical performance is better than needed for medical relevance PLUS the erroneous idea that customer complaints provide a reliable and acceptable monitor of problems. Do you think that the Firestone tire problem was adequately monitored by depending on customer complaints, or should Firestone have provided a more reliable QC system to measure of faults during the production of the tires? Customer can't see technical characteristics, such as precision and accuracy, until they blow up on them. Then it's too late!
- Laboratory errors can be reduced by moving to Point-of-Care testing. WRONG! The logic is that moving the testing closer to the patient will reduce the pre-analytic errors, but it's erroneous to think that the error rates for the analytic and post-analytic phase will remain the same. The quoted figures apply only to a well-controlled clinical laboratory, not a POC setting. Analytic errors most likely will increase. Post-analytic errors may increase even more if you consider that test results must be documented in the medical record before the testing process is considered complete.
- Statistical QC is not needed when testing is performed in a POC setting. WRONG! The logic again draws on the erroneous assumptions that analytic performance is better than needed for medical relevance and that analytic errors will be the smallest fraction of errors in the testing process, PLUS the additional speculation that electronic QC provides an effective way to monitor instrument performance. Electronic QC only tells you if the instrument readout is still working - important when instruments might be dropped off the bed or thrown against the wall, but it's not sufficient to tell you whether the analysis really works okay. We need to be more concerned about analytic errors than ever before.
What do these opinions have in common? They all reference the same study and expand the recommendations and conclusions of that study! Here's the complete unabridged study as published in the literature [4].
"A review of 363 incidents captured by the Hospital Laboratory's Quality Assurance Program during 1987 was conducted. The medical records of 336 patients involved in 304 of the incidents were reviewed. The medical records reviewed were those in which physician orders for laboratory tests were missed or incorrectly interpreted; patients were not properly prepared for testing or were incorrectly identified; specimens were collected in the wrong containers or mislabeled or mishandled; the analysis was incorrect; data were entered improperly; and results were delayed, not available, or incomplete, of they conflicted with clinical expectations."
"No effect on patient care could be found for 233 patients, 78 patients were not harmed, but were subjected to an unnecessary procedure not associated with increased patient risk, and 25 patients were not harmed, but were subject to an additional risk of inappropriate care. Of the total number, pre-analytic mistakes accounted for 218 (45.5%), analytic for 35 (7.3%), and post-analytic, 226 (47.2%). Nonlaboratory personnel were responsible for 28.6% of the mistakes. An average of 37.5 patients per 100,000 treated were placed at increased risk because of mistakes in the testing process."
That's right, folks! It's an abstract, not a peer-reviewed study. It's a retrospective review of incident reports by the quality assurance committee in one clinical laboratory in 1987 - the good olds days of laboratory testing before CLIA-88 reduced quality control and personnel standards to the minimums used in practice today. We may trust in God, but we should have better data and a well-documented study before expanding its conclusions and extending it's recommendations.
For a study to become the cornerstone of evolving strategies and practices in laboratory quality management, it should have been critically reviewed and published in the peer-reviewed literature. Over time, as the references are made to papers that reference an earlier reference to an earlier study, the study becomes less and less traceable, but somehow becomes more and more creditable. Who can even find the original abstract in the CDC proceedings today?In spite of that and the shortcomings of the actual study design, this study exerts tremendous influence on laboratory quality management practices today. It lets us hope and believe, through a semblance of logical reasoning, that we no longer have to worry about analytical quality at a time when we're doing less and less to actually manage and control the quality of laboratory testing.
Dr. Goldschmidt left us with one other challenge. How do you get to the other side of a river infested with crocadiles? Answer - you jump into the river and swim to the other side. This is the right method because all crocodiles are at the meeting called by the lion. Don't you remember?
Logic is not always so logical. Common sense is not so commonly found. Laboratory errors won't go away just because it seems logical that there should be fewer errors with more expensive equipment, more computerization, and more automation. Improving the order of our processes is offset by the turbulence and disorder in laboratories and healthcare organizations today. Orderly processes, when operated in an environment of disorder, will still be subject to errors. All errors are important and we must start by eliminating those most directly under our control - analytical errors, then ever expand our efforts to eliminate errors from the beginning to the end of the testing process - the pre-analytical and post-analytical errors.
- Proceedings of the 1992 Clinical Chemistry Forum. Clin Chem 1993;39:1446-1553.
- Proceedings of the 1996 Clinical Chemistry Forum "Quality for Tomorrow." Clin Chem 1997;43:864-912.
- Strategies to set global analytical quality specifications in laboratory medicine. Scand J Clin Lab Invest 1999;59(7):475-586.
- Ross JW, Boone DJ Assessing the effect of mistakes in the total testing process on the quality of patient care. Proceedings of the 1989 Institute on Critical Issues in Health Laboratory Practice. Centers for Disease Control, Atlanta, GA, 1991, page 173.
Other Essays:
