Essays
Technology for Implementing QC Right
What kind of software is available for Quality Control? Is the software you get for "free" up to the job?
Dr. Westgard provides a cogent survey of what's available in today's market. He also highlights what features are important in a QC program, as well as the features that still need to be offered.
Computer software is the "technology" that is needed to implement QC procedures in an effective and efficient way. We seldom think of QC in terms of technology, probably because QC continues to be such a manual operation in many laboratories.
QC software has been available in healthcare laboratories for over twenty years. These QC programs sometimes reside in Laboratory Information Systems (LIS), data interface managers, data analysis workstations, analytical instruments, and personal computers (PC). After all these years, one might assume that QC software has been perfected and that all QC packages are equal. Not true! QC technology has lagged behind the development of measurement technology and laboratories still need to be careful to implement the right QC procedure in the right way.
The discussion here focuses on PC software, rather than QC programs that are part of laboratory information systems or analytical instrument systems. A laboratory that purchases an information system or instrument system with an integral QC package will most likely utilize that package, whether it is good or bad. Managers, supervisors, and analysts often have to live with the QC capabilities available in the products that have been purchased. Even with PC software, users often obtain the QC program as part of the purchase of control materials. Again, users will likely have to get along with the capabilities of that program because it was free.
There seems to be an expectation that QC software should be "cheap" if not entirely free. The lack of financial return for good QC software is partly responsible for the limited advancements and progress in the development of improved technology. The old adage that "you get what you pay for" applies here. Many laboratories utilize cheap QC software and then wonder why they have so many QC problems. Many of those problems are due to the limitations of the software and their inability to implement optimal QC designs for cost-effective operation of laboratory testing processes.
What programs are available?
Here are some typical programs that are available to laboratories in the US. This list is not intended to be comprehensive, but to provide examples of programs that have been tailored to healthcare testing applications. There are many more suppliers of QC software for industrial applications and that software may also be adapted for laboratory applications (see the Quality Progress journal, www.asq.org).
Bio-Rad Unity (www.bio-rad.com) is the leading program in terms of the number of installations. It is a Windows program and can be connected to instrument systems by third-party interface service providers. The software and related peer-comparison service are usually bundled with the purchase of control materials and are limited to the use with Bio-Rad products. Bio-Rad provides one of the most complete lines of control materials available today.
MAS/Dade LabLink (www.mas-inc.com) has been a leading competitor in QC software, beginning with the Dade QAS program that was the first widely distributed QC program. LabLink is a Windows program and also provides a peer-comparison service via the Internet. LabLink is usually free with the purchase of MAS/Dade control materials and also limited to use with materials from the manufacturer.
Sigma Diagnostics Computrol on Line (www.sigma-aldrich.com) is a Windows QC program that provides in-laboratory analysis of control data for immediate decision-making (internal QC) plus provides rapid access to peer-comparison statistics (external QC). This program is available to customers who utilize Sigma's line of control materials.
Fisher Scientific ConCurTRAK (www.fishersci.com/) is a Windows PC program for internal QC plus a peer-comparison service. It is available to customers who purchase Fisher Healthcare control materials.
What features and characteristics are important?
There certainly will be different opinions about the features and capabilities that are most important in QC software. This list emphasizes the needs for flexibility within a well-structured QC environment, i.e., the flexibility to make choices that are beneficial for cost-effective operation in your laboratory along with the structure to help you make the right choices and implement them in the right way.
Ease of use. Keep in mind that most laboratory analysts find QC to be difficult, even confusing. That trend will continue into the future because many personnel who perform tests will have little background in QC. A user-friendly, graphical interface is needed to help the operator perform QC correctly. Most QC software now operates under the Windows operating system and utilizes conventions standard in other common programs.
Rapid setup of new applications. While flexibility is necessary to implement a wide variety of QC designs, simplicity in operation is critical for getting started. This often means that certain default settings must be available in the initial setup. Later on, those settings can be changed to adapt the QC procedure for the characteristics of the test and method. For example, new applications may utilize a default QC design (rules) and a default mode of calculation to establish control limits.
Multiple sources of mean and SD for setting control limits. Different situations may justify the establishment of control limits on the basis of means and SDs from labeled values, assigned values, initial calculations from a limited data set, fixed interval calculations, moving interval calculations, and cumulative to date calculations. Flexibility is needed to switch between these different sources or estimates of means and SDs. For example, an application may begin using assigned values for means and SDs, then switch to the means and SDs calculated from the first month's data (fixed interval), and then update those estimates with each additional month's data (cumulative to date).
On-line and/or keyboard data entry. Some applications require on-line data acquisition because of the large number of control measurements that are collected, whereas keyboard entry of data may be okay for low volume laboratory tests. QC programs may be offered in versions for on-line data acquisition, single user manual entry applications, or network manual entry applications.
Special data calculations. Some applications require raw values to be converted or transformed before being analyzed for control purposes, e.g., viral load data may require a log transform before plotting and display. Some tests may require the calculation of the signal/cutoff ratio, etc. Programs aimed at high-volume laboratories may not provide the calculation features needed for specialized testing.
Flexible rule selection. Cost-effective operation ultimately depends on optimizing QC for each individual test. A wide variety of control rules are useful, both single-rule and multi-rule procedures. These rules can be optimized for an individual test on the basis of the quality required for that test and the imprecision and inaccuracy observed for the method.
Automatic rule selection. Support for the selection of control rules should be built into the program to facilitate the calculations that are necessary to select an optimal QC design and to document the validity of this design. Off-line tools may also be utilized to select appropriate QC designs, but there may be some difficulties in matching up the selected design with those designs available for implementation. The integration of an automatic design function should assure that the selected designs can be implemented.
Multi-stage QC designs. Traditionally, a single set of control rules and a chosen number of control measurements have been used, i.e., a single QC design. Ideally, that design should have a very low probability for false rejection and a high probability for error detection. In situations where it is impossible to achieve both low false rejections and high error detections with a single QC design, the strategy should be to utilize two different QC designs - one for high error detection and another for low false rejection. The ability to implement such multi-stage QC designs is one of the most advanced features of QC software.
Levey-Jennings control charts. The standard way of displaying control data is to prepare a Levey-Jennings control chart for each control material. There are generally two or three different control materials in use simultaneously, which requires the L-J charts to be presented in a manner that facilitates inspection across materials.
Multi-level, multi-stage, single-chart display. The capability to display all materials and multiple QC designs on a single chart facilitates the inspection and interpretation of control results. Generally this is accomplished by calculating a z-value, i.e., the difference between each control measurement and the mean for that material is divided by the SD for that material to calculate the number of SDs from the mean, e.g., a z-value of -2.1 would indicate the control measurement is 2.1 SDs lower than the mean for that material.
Automatic flagging of rule violations. A major feature of QC software is to provide a uniform interpretation of control data and to automatically flag all out-of-control runs. The application of control rules within and across materials and runs should be carefully assessed. Once a QC application has been properly setup, the software should provide consistent inspection of control results and identification of analytical problems.
Automatic entries into action log. Anytime an analytical run is determined to be out-of-control, corrective action must be documented. The automatic entry of each rule violation into an action log facilitates the proper documentation of corrective action.
Exceptions or exclusions log or report. Supervisory review of on-going QC can be facilitated by having a special log that contains all out-of-control events, often called an exceptions or exclusions log. This log can be constructed from the action log entries for a specified group of tests. Supervisory review can be documented in this log.
Summary reports. Many laboratories prepare monthly summaries of QC data for review by supervisors and managers, updating the means and SDs for calculation control limits, and comparison of means and SDs over time. Flexibility is needed for selecting the period of time and the mode of calculating the data.
Data export to electronic spreadsheets. Additional inspection, manipulation, and calculation of control data is facilitated by exporting the data or file to an electronic spreadsheet, such as MicroSoft Excel. Once the data is in Excel, it can be sorted, calculated, or exported to other calculation programs.
Peer-comparison service. The ability to compare the mean observed for a control material in an individual laboratory to the mean of a group of other laboratories is a valuable way to monitor the stability and accuracy of a method. This service is of most value to small laboratories that have limited resources for assessing the accuracy and stability of their testing processes.
Patient data statistics. Patient test results may be useful for monitoring the stability of measurement processes. Techniques such as the Average of Patient Normals (AoN) first exclude patient results outside "normal," then calculate the average of the remaining results, i.e., the average of the normals in the patient population. This feature almost certainly depends on having on-line acquisition of patient test results.
Why are improvements still needed?
Although there are a variety of QC programs available today, users must still be careful in selecting QC software to get the flexibility needed to optimize operations in their laboratory. QC technology has lagged so far behind measurement technology that many QC applications today are both ineffective and inefficient - doing the wrong QC in the wrong way, rather than doing the right QC right.
The limited progress in QC technology in recent years is due in large part to the compliance mentality that has developed in laboratories, along with the government's inability to implement the CLIA regulations as originally intended. With no need to respond to either users or the government, manufacturers have done little to improve QC technology. In our market economy, laboratories are getting what they pay for in QC technology - paying little attention to quality, getting little capability to implement optimal QC procedures, but getting it cheap.
Improvements are urgently needed. Today's staffing shortages are expected to become even more severe in the future. In the past, well-trained and educated professionals were the key to quality. In the future, QC technology will be the key to quality. That technology must provide an automated QC process to guarantee the quality of the test results produced, just like the automated measurement technology that has been developed to assure the desired volume of test results can be produced.
James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.