Quality Control: |
|
Are improvements in quality control (QC) still needed? Do we need more sophisticated QC software? Do we need a more automated QC process? In short, do we need better QC technology in future laboratory systems?
These questions are being asked by many manufacturers as they design their next generation analytical systems. The usual setting for this discussion is a customer focus group, where a small group of key customers are involved with a carefully structured presentation and a carefully moderated discussion. The difficulty is that these questions about quality require a technical analysis, not a market analysis. In truth, most customers would prefer not to run QC at all since it costs time and effort and requires skills and training.
Quality is like safety - an important issue, but not a readily marketable feature. Witness safety in automobiles, where government regulations have driven progress, not customer market demand. Consider safety in the workplace, where government regulations are required to protect workers from hazards and danger. Consider safety in the home, where building codes set the standards for materials and smoke alarms. The market survey approach leads manufacturers to believe that improved quality systems aren't needed because customers don't demand these improvements. While customers may be aware of some of the current limitations, they don't know what might be possible in future technology. Manufacturers actually need to lead the market, rather than wait for customers to express those needs or have regulators impose new rules.
To provide a better perspective on the need for improvements in QC, let's consider what should be done to optimize QC practices. We'll start with a list of bad practices that need to be eliminated, then consider what to do to make improvements.
Probably well over half of laboratories worldwide use control limits that are calculated as the mean plus and minus 2 standard deviations (SD), also known as a 12s rule. This practice is known to have a high level of false rejections - 1 out of every 20 points is expected to exceed 2SD limits, which is a 5% rate of false rejections. However, many laboratorians don't understand that when two controls are analyzed per run, the false rejections rate is 10%, with three control per run it's up to 14%, and with four controls per run it's about 18% [1]. This false rejection rate can potentially cause a 10 to 20% waste of laboratory resources! Certainly the use of 2SD control limits should be avoided with any automated analytical system. However, since customers aren't demanding such a change, manufacturers continue to support the use of 2SD limits in their QC software. Marketing responds to customer "wants", rather than the real "needs" for better QC procedures with lower false rejection rates. Wants and needs are not necessarily the same!
In many laboratories, the common response to an out-of-control situation is just to repeat the controls, rather than to fix the problem [2]. The rationale is that the out-of-control situation is likely a false rejection, therefore, let's check again. And often if the repeats are still out, the response is to prepare new controls for analysis. And, of course, if they're out, they will also be repeated. The problem with this practice is that you're actually changing the control rules and are requiring that two or more consecutive points exceed a 2SD limit (e.g., 22s, 32s, even 42s). While this change in rules does reduce false rejections, it also reduces the error detection, leading to a practice that has little chance of detecting medically important errors.
If we need to eliminate 2SD control limits, should we use 3SD limits or possibly multirule QC instead? First of all, there's no one rule or one set of rules that's right for all tests and methods. Some methods have better precision than others, therefore different QC procedures should be used. The most cost-effective operation is possible when the QC procedures are selected for the individual tests on the basis of the quality required for the test and the performance observed for the method [1].
The basic principle of statistical QC is to assess current performance relative to past performance. The critical measures of past performance are the mean and SD observed for the control material analyzed by your method in your laboratory. Bottle values generally reflect the overall performance of a group of laboratories, and while the bottle value means may be useful for assessing the accuracy of your methods, the bottle value SDs are usually larger and will give control limits that are wider (which causes low error detection).
Another ill-advised practice is the use of "medical decision limits" to transfer medical quality requirements directly to a control chart, which supposedly eliminates statistical rejections and only identifies problems that are medically important [3]. Again, the technique is appealing because it provides wider control limits, which give fewer false rejections, but also causes lower error detection.
Alternatives to statistical QC are desirable, particularly in Point-Of-Care (POC) applications where personnel may have little laboratory background or training. Electronic QC is appealing because of its simplicity, low cost, and few rejections. Unfortunately, electronic QC checks only a few steps in the total testing process - often only the electronic readout step [4]. Therefore, electronic QC by itself is not sufficient to assure the quality of the tests being performed.
Statistical QC can monitor many steps in the analytical process, including the proficiency of the operator. SQC should be considered essential whenever operator skills may affect the results of the testing process [5].
This issue is addressed by the new QC practice guidelines [6] that were published in early 1999 by the National Committee for Clinical Laboratory Standards (NCCLS). These guidelines draw heavily on recommendations by a European working group [7], thus there are some common practice guidelines emerging in both the European and American markets. Some of the important principles are summarized by the list of DO's that follows.
The first step in the objective and quantitative management of quality is to define the quality that is to be achieved. How can quality be managed if we don't know what quality we're trying to achieved? The starting point must be to define the quality that is needed. Several different formats may be utilized, such as the allowable total error format of most proficiency testing and external quality assessment criteria, the medically important change involved in clinical treatment decisions, and the allowable imprecision and allowable inaccuracy based on the biological variation of an individual. Tabulations of these forms of quality requirements are available in the literature and via the Internet. For example, see our lesson on The need for standard processes and standards of quality.
It's best to keep the chance of false rejections below 5%. This requires that you eliminate the use of 2SD control limits whenever the number of control measurements (N) is more than 1, which implies you should almost NEVER use 2SD limits. Use of 3SD limits will give you only a 1% false rejection rate when N is 2 to 4. Multi-rule combinations will usually have between 2 to 4% false rejects as long as N is 2 to 4. When higher Ns from 5 to 8 are used to achieved greater error detection, false rejections of 6 to 8% may have to be tolerated.
A practical goal is to aim for a 90% chance of detecting medically important errors. QC procedures that have appropriate error detection can be selected on the basis of the quality required for the test and the imprecision and inaccuracy observed for the method. The NCCLS C24-A2 document describes the general steps for planning a QC procedure (in section 5), which include (a) defining a quality requirement for the test, (b) determining the imprecision and bias of the method, (c) identifying candidate QC procedures, (d) predicting QC performance, i.e., assessing the rejection characteristics of the candidate QC procedures, (e) setting criteria for QC performance, and (f) selecting the appropriate control rules and number of control measurements.
The selection of appropriate QC procedures can be supported with graphical planning tools, such as critical-error graphs and OPSpecs charts (operating specifications). Adoption of these kinds of tools will make it quick and easy to select QC procedures. Example applications are provided in the scientific literature [8,9] and also on the Internet [http://www.westgard.com/qcapp3.htm, for example]. A PC computer program is also available and provides an automatic QC selection function [10,11].
QC applications themselves must be systematic from the preparation of control materials through the interpretation of control results. Automatic preparation, scheduling, and sampling of control materials is important for standardizing the collection of control data. Computer implementation of control rules is important for standardizing the interpretation of control data.
The NCCLS document makes it very clear that "the mean and standard deviation of a control material should be established on the basis of repeated measurements on those materials by the methods in use in the laboratory." It further recommends the use of cumulative control limits that are based on the means and SDs for several months of laboratory data.
Computer support will facilitate data calculations, such as the calculation of cumulative means, SDs, and control limits. More important, computer support is necessary to individualize the QC procedure for each laboratory test. In addition, online computer support is desirable for interpretation of control data for tests performed by high-volume multi-test analyzers or any analyzer operated by personnel with limited laboratory background and training. Computer support is also essential for complex QC designs, such as multi-stage QC procedures that utilize a "startup" set of rules for high error detection and a "monitor" set of rules for low false rejections, with periodic switching between the two designs as appropriate during the operating cycle of an analyzer.
When QC procedures have been carefully selected to minimize false rejection and maximize detection of medically important errors, the proper response to an out-of-control signal is to reject the analytical run, trouble-shoot the method, identify the problem, eliminate the cause of the problem, and verify the test results for patient specimens.
The Total QC system is a mix of statistical QC, instrument checks, validation tests, preventive maintenance, and patient-data QC procedures [12]. The optimum mix can be defined on the basis of the error detection available by statistical QC. If 90% error detection can be achieved, then the Total QC strategy should rely on statistical QC to detect problems; the time and effort spent on other QC procedures should be minimized. When 90% error detection cannot be achieved, then the Total QC strategy should be to increase the use of non-statistical procedures and components, including increased preventive maintenance, increased operator training, and reduced rotations of operators.
With this background, we can return to the original issues. It should be apparent from the list of DON'Ts that current QC practices are not optimal and that changes are needed. The list of DO's makes it clear that improvements must start with the careful planning and design of QC procedures, support the proper implementation and operation of those QC procedures, and lead to a Total QC system that integrates statistical QC with other QC components. In the next millineum, it will be essential to improve the QC process [13]. Statistical QC will continue to be an essential component in the foreseeable future [14].
Features and capabilities that are desirable include the following:
This improved QC technology may reside in instrument software, data analysis workstations, lab information systems, a laboratory Intranet, or a manufacturer's Internet service, or may be distributed between these different sources. The QC technology of the future will allow the laboratory to specify the quality required for a test, then the QC technology will automatically select an appropriate QC procedure, implement the desired control rules, acquire the necessary number of control measurements, and interpret the control data to release only those patient test results that meet the laboratory's defined requirements for quality.
Other Essays:
