Quality Requirements and Standards

Are instruments so good that we don't have to worry about quality anymore? Or are they so bad there's no point trying to differentiate one from another? In light of the events of 2010, a reflection on the state of quality in diagnostics and the laboratory.

The Quality Fade

October 2010
Sten Westgard, MS

After a recent lecture, I overheard a laboratory manager talking about his latest troubles with an instrument. Despite significant problems with some of the analytes on his instrument, he seemed resigned to tolerate the disruption. While he conceded that the problems were not good for the laboratory or the patients, he did not intend to blame the instrument manufacturer or consider changing to a new instrument. In essence, he shrugged his shoulders and explained that it was just that manufacturer's "turn" to have problems. His tacit implication was that all manufacturers had poor methods at one time or another, so there was no point in changing instruments, even when problems came to light.

We have encountered this attitude before, but usually it manifests itself in the reverse orientation: all tests are equally good in quality, so you can choose any instrument and you'll be fine. Here, the attitude was: ultimately, all tests are equally bad, so there's little point in choosing one method or instrument over another.
Either attitude is a dangerous sign of the times for laboratories.

Those of us in healthcare know that perfection is a long way off. We aim to eliminate "never events" while knowing that even the worst case scenarios seem to occur with depressing frequency. Absolute perfection is often too much to ask: the sheer volume of testing means that even low probability events stand a good chance of happening over the long run. Even quality management approaches like Six Sigma don't aim for perfection - instead the goal is to improve processes so that only a few defects occur per million opportunities (Six Sigma is 3.4 DPMO).  In an era where many hospitals are generating millions of reportable results per year or even per month, that means even world class quality isn't totally defect free.
But while it may be realistic to acknowledge that some errors will occur in our processes, that doesn't mean it is acceptable to surrender to that inevitability. Laboratories should challenge and verify the quality of all their methods. Skepticism is healthy, but too often it degrades into cynicism:

"In its most extreme form, cynicism enshrines as eternal truth the notion that every institution and every person is for sale. It assumes that human motives are debased and always will be. Implying that no traditions, truths, or community bonds are worth fighting for, it occupies the mental and psychological space we could reserve for hope....Painting a categorically bleak portrait of human existence, with no redemption, cynicism convinces us that all businessmen and politicians are dishonest, all religious leaders charlatans, and all journalists cheap-shot hacks, all social activists fools. Better to expect nothing, in this view, than to set ourselves up for certain disappointment."[1]

Cynicism can be tempting for the laboratory, because once embraced, the laboratory is off the hook; when all methods are equally bad, there's no point and no need to make any effort to find a good method. Unfortunately, the consequences of expecting nothing from method performance impacts the patients worse than it does the laboratory. Unreliable results mean sub-optimal care for patients, but may only result in increased activity for the laboratory, as tests are run and repeated and repeated.

One of the daunting challenges for the laboratory is the sheer size and complexity of today's instrumentation. It is no longer a trivial thing to change instruments. It's not just a major decision anymore; it's an enterprise-wide decision, involving months of evaluation, numerous committees and senior decision makers. The cost of a new instrument is so high, often the final decision is made outside the laboratory, by hospital or health system administration officials who may not have a grasp of the reality of method performance and the importance of quality. When a decision takes such effort, it builds up an inertia of its own - it becomes very difficult to change the decision once it has been made, even in the face of strong evidence.

[Furthermore, once an instrument is installed, the laboratory may not have easy ways to improve the method performance. They are at the mercy of reagent lots, service visits, and technical support response. While the promises before the sale might have been earnest, the reality of routine operation may be something different. Clinicians and Administration are often the push and pull of the testing decision. Clinicians want a test result and prefer a fast one. Administration wants a cheap test result. Between faster, cheaper, better, usually the first two priorities win out, and the laboratory gets left holding the quality bag.]

I recently finished a book about somewhat similar problems in a different field. Poorly Made in China recounts the experiences of a Western translator who lived in China and acted as the representative and agent for multiple US companies seeking to outsource their manufacturing to China. The main thrust of the book is that US manufacturers find themselves at a disadvantage when dealing with Chinese manufacturers, who may deceitfully modify the production of goods in a way that degrades quality (and  increases profits for the manufacturing plant). In a specific example, he describes how plastic bottles manufactured by one Chinese plant were progressively made thinner, without the customer's consent or knowledge, until one shipment was so thin the bottles collapsed:

"Chinese factories often engaged in this sort of quality fade - the incremental degradation of a product over time. They quietly reduced the amount of materials or else manipulated the quality of raw inputs. The changes were gradual, almost imperceptible. The importer was neither asked for permission nor told." [2]

In the case of Chinese manufacturers, the excessive greed of US outsourcers overwhelmed their concern for quality. When the US businessmen were too eager to generate profits, and too unwilling to stop and insist on better quality, the manufacturing plants took advantage of that. The scandals with lead paint in toys, toxic tooth paste, etc., are just the most public examples of this situation.

In the laboratory, greed isn't the motivating factor. Instead it's the pressure to produce test results quickly and cheaply. Even as methods are revealed to perform worse than clinical need, the demand for test results doesn't diminish.

James Reason describes the resulting organizational culture in terms of learned helplessness: "a condition in which people learn that attempts to change their situation are fruitless so that they simply give up trying."[3] 'Learned helplessness' is a psychological state where "a person perceives that he can no longer control his own destiny.... The energy and will to resolve problems and attain goals drains away."[4] For decades, we have known that his behavior manifests itself not only in individuals, but also in organizations. When people or even departments within a company believe they can no longer impact the organization's decisions, they learn to stop trying.

This is where laboratory helplessness can lead. If laboratories don't insist on better quality, but instead show themselves willing to tolerate poor or even defective methods, they will increasingly find that's what manufacturers deliver to them. If the highest priority of the hospital is cheap testing, labs will find that their instruments and test results begin to resemble cheap goods. Technologists will find less satisfaction in a career that produces progressively worse outputs.

The point here is that laboratories need to be vigilant and demanding and hold manufacturers accountable for instrument failures. If labs don't care about quality, don't expect that manufacturers will care about it either. Manufacturers follow the cues of their customers. When quality isn't important to the customer, the manufacturer will adjust quality downward.

Whatever the latest instance of failed instrumentation, quality in the laboratory won't die in one terrible scandal. But if we engage in cynicism and adopt an attitude of helplessness, quality may indeed fade away.

References:

1. Paul Rogat Loeb, Soul of a Citizen, St. Martin's Griffin, (NY, NY), Revised Edition, 2010, pp.84-85.
2. Paul Midler, Poorly Made in China, Wiley (Hoboken, NJ), 2010. Page 96.
3. James Reason, Managing the Risks of Organizational Accidents, (Ashgate Publishing Company, Burlington VT, 1997), p.193
4. Paul Bate, The Impact of Organizational Culture on Approaches to Organizational Problem-Solving, Organization Studies, 1984, 5/1, p.59.