Tools, Technologies and Training for Healthcare Laboratories

EZ Rules for Quality Assurance

Dr. Westgard describes our latest software. EZ Rules is a simple interview format program that makes it simple, easy, fast, and automatic to plan quality.

In 2000, we developed and demonstrated an advanced QC Design program. This next generation program supported complex multi-stage and multi-decision level QC designs in a single application file. It added the capability to use biologic goals as quality requirements and expands the list of control rules to include Average of Normals (AoN) patient data rules.

One key feature is a new question/answer mode of operation to facilitate learning the quality-planning process and to simplify the use of this very sophisticated program. If you have used a computer program such as TurboTax to prepare your income tax returns, you've probably found the interview mode of operation to be the easiest way to use the program.

We observed that the interview mode of operation was the preferred mode of operation by new users. The reaction was so positive that we offered this new program as EZ Rules. We think that EZ Rules will fulfill a real need in healthcare laboratories to assure that the quality control itself is appropriate for the medical quality required by physicians and patients and the analytical performance observed for methods in the laboratory.

QC Validator Technology

An automatic QC selection process was first demonstrated with version 2.0 of the QC Validator program [1,2]. Version 3.0 has been developed with the Delphi programming tools and a new graphical folders interface. A software "engine" can be prepared in the form of a Dynamic Link Library (*.dll) that can be accessed by any Windows program. The EZ Rules program makes use of the QC Validator engine with the interview or question/answer mode of operation. While providing much of the capability of Validator 3, EZ Rules does not support multi-stage or multi-level QC designs. EZ Rules is more comparable to Validator 2.0 with the addition of biologic goals as a new type of quality requirement and addition of Average of Normals patient data rules. Validator 3 truly represents the next generation of QC design capability by supporting the three-stage structure recommended for advanced QC systems [3].

EZ Rules Interview Mode of Operation

ez9rev1

The accompanying example screen shows the format of the interview. The top panel is used for the question and answer. The space below that panel is used for on-screen instructions and a tutorial example.The user can move through the screens with the aid of a mouse or by keyboard alone. Generally one or two parameters are considered with each input screen in EZ Rules.

The technical information needed for QC selection begins with the type of control rule - traditional statistical rules or Average of Normals patient data QC rules. Traditional statistical rules refers to the rules commonly used on Levey-Jennings control charts or with Westgard multirule procedures [4], e.g., rules such as 13s, 22s, R4s, 41s, etc. Average of Normals (AoN) refers to rules that use the mean of patient results to monitor the stability or systematic changes of a method [5]. AoN procedures are not commonly used today, but are being recommended as one of the new approaches for cost-effective QC [6].

ez9rev3

EZ Rules allows the user to select from three types of quality requirements - allowable total error, clinical decision interval, or biologic goals:

  • CLIA provides a list of approximately 80 tests with analytical total error requirements defined by the acceptability criteria for proficiency testing.
  • Clinical decision intervals come more directly from guidelines for interpreting test results, e.g., the National Education Cholesterol Program guidelines for interpretation of lipid tests.
  • Biologic goals are derived from data on biologic variability and an extensive list of published values is available

The next series of input parameters are critical for QC selection. They include the CV (observed imprecision), bias (observed inaccuracy), expected instability, and number of control materials to be analyzed. Once these parameters have been entered, the QC selection occurs automatically and the control rules and number of control measurements appear on the screen.

In this example, the selected QC procedure is a 12.5s rule with a total of 2 control measurements per run.

On-screen documentation is available in the form of the OPSpecs chart that was used to make the selection, as shown in the accompanying figure where a 12.5s with N=2 procedure has been selected. The OPSpecs chart is the heart of our QC selection methodology. For the quality requirement entered by the user, the chart shows the imprecision (x-axis) and inaccuracy (y-axis) that are allowable for different control procedures, as indicated by the diagonal lines. The key at the right side identifies the different control rules and numbers of control measurements that are shown top to bottom on the chart. The observed performance of the method is displayed by an operating point whose x-coordinate represents the method's CV and whose y-coordinate represents the method's bias. In this example, the operating point has an x-value of 2.0 and a y-value of 0.0, thus is located on the x-axis at a value of 2.0. An appropriate QC procedure is one whose allowable limits for imprecision and inaccuracy provides a line above the method's operating point. In this case, there are several possible QC procedures that will assure the detection of medically important errors. The program chooses the simplest one with the lowest number of control measurements, in this case, 12.5s with N=2.

Further documentation of the selected QC procedure is provided by a QC Validation report that can be printed and includes all input parameters, calculated parameters, an OPSpecs charts (like above), and critical-error graphs that give more detailed estimates of the probabilities of detecting medically important random and systematic errors. In an integrated instrument application, the documentation for the QC selection process could be added to the information stored for the selected QC procedure.

The complete list of EZ Rules interview questions for use of an analytical allowable total error type of quality requirement is shown in the list below. All these parameters are included in the interview process to provide the widest range of flexibility for utilizing the design capabilities of the QC Validator technology. Some additional questions/screens are included when other types of quality requirements are used (i.e., clinical decision intervals and biologic goals have different formats as quality requirements and certain preanalytic factors, such as within subject biologic variability, are needed when a clinical decision interval is used).

List of EZ Rules Interview Questions

1. What is the name of the test, units to be reported, and name of the system or method?
2. What type of control rules do you want to use?
Choice of Traditional statistical rules or Average of Normals patient data rules
3. What type of quality requirement do you want to use?
Choice of allowable total error, clinical decision interval, or biologic goals
4. What is the critical medical concentration or decision level for the test?
5. What is the quality required at this decision level?
6. What imprecision or CV (in percent) is observed at this decision level?
7. What inaccuracy or bias (in percent) is observed at this decision level?
8. How often do you expect the method to have a problem?
Choice for frequency of errors of 10%
9. How many different control materials are analyzed?
Choice of 1, 2, 3, or 4

-----Automatic QC selection occurs when these parameters have been entered

10. Do you want to see the chart of operating specifications that was used to make the QC selection?
11. Do you want to view a report that documents the QC recommendation and add coments to it?
12. Do you want to print the report?
13. What do you want to do now?
Choice of Create a new application file, Open an existing application file, or Exit Validator

References

  1. Westgard JO, Stein B. Westgard SA, Kennedy R. QC Validator 2.0: a computer program for automatic selection of statistical QC procedures in healthcare laboratories. Comput Method Program Biomed 1997;53:175-186.
  2. Westgard JO, Stein B. An automatic process for selecting statistical QC procedures to assure clinical or analytical quality requirements. Clin Chem 1997:43:400-403.
  3. Westgard JO. Sage advice about new approaches to quality control.
  4. Westgard JO, Barry PL, Quam EF. Basic QC Practices. Madison, WI:Westgard QC, Inc., 1998.
  5. Westgard JO, Smith FA, Mountain PJ, Boss S. Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control. Clin Chem 1996;42:1683-1688.
  6. Edutrak 3206. The use of patient data for process control: Its time has arrived. [Abstract] Clin Chem 2000;46:S18.

James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.