Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Essays
  4. QC Design
  5. Principles of QC Planning for Immunoassays

QC Design

Principles of QC Planning for Immunoassays

Dr. Westgard takes the theory and tools of quality planning and applies it to immunoassays. This was part of his presentation to the joint meeting of UK National External Quality Assessment Schemes (UK NEQAS) for Endocrinology and the European Ligand Assay Society (ELAS) in Edinburgh, Scotland, a paper that he delivered "virtually" to the conference.

A presentation for the joint meeting of UK National External Quality Assessment Schemes (UK NEQAS) for Endocrinology and the European Ligand Assay Society (ELAS), Heriot-Watt University, Riccarton, Edinburgh, Scotland, April 14-17, 1998.

Dept. of Pathology and Laboratory Medicine
University of Wisconsin Medical School
Madison, WI 53792 USA

ABSTRACT

Selection of analytical methods and control procedures can be accomplished with a systematic quality-planning process. Key elements in this planning process are the definition of the quality requirement for the test and the translation of that quality requirement into operating specifications for imprecision, inaccuracy, and quality control (QC). A thyroxine example is provided to illustrate the different formats for defining quality requirement and the use of an OPSpecs chart for setting specifications for the imprecision and inaccuracy that are allowable and for selecting the appropriate control rules and the total number of control measurements for a statistical QC procedure. Quality planning for immunoassays is complicated by the need to consider multiple formats of quality requirements, multiple medical decision levels and the analytical performance at each of those decision levels, and multiple QC designs. Improvements in quality-planning technology should be able to overcome these complications. Quality-planning tools, technology, and training can be delivered via the Internet to support applications in individual laboratories.

"Quality control" versus "arbitrary control"

"Quality control" (QC) implies that a defined level of quality is assured or guaranteed for the laboratory tests being performed. Unless the quality to be achieved has been defined, the traditional laboratory practice of analyzing control samples actually amounts to "arbitrary control." However, the term "arbitrary" wouldn't be very comforting to the physicians and patients who are the customers and consumers of the test results. Quality control sounds good, even though the actual control practice in most laboratories is arbitrary.

To put quality into quality control, the desired level of quality must be defined in the beginning to guide the selection of analytical methods AND quality control procedures. Then the management objectives are to select methods with appropriate imprecision and inaccuracy and to select statistical control rules and numbers of control measurements to detect medically important errors.

Some guidelines for planning QC procedures for immunoassays are available in the literature [1-3]. The steps of a systematic planning process that utilizes charts of operating specifications (OPSpecs charts) have been presented by Mugan, Carlson, and Westgard, along with seven specific examples for tests performed on an automated analyzer [1]. Seth has illustrated how QC procedures can be selected with the aid of critical-error graphs. Carey has discussed practical aspects of selecting appropriate QC procedures in an essay on Tips for managing the quality of immunoassays that appears on this website (including four example applications).

Quality planning

Quality planning options to select an appropriate method of analysis or an appropriate QC procedure.

A systematic process is needed to carefully consider the many variables and factors that affect the outcome of an analytical testing process. The first step is to define the quality requirement for the diagnostic test of interest. Then, as shown in Figure 1, there are two variations of the planning process, depending on whether the purpose is to select the method of analysis or to select the QC procedure.

To select a new method of analysis, the process involves specifying the QC procedure (statistical control rules, number of control measurements or N) that will be employed and then setting the developmental or purchase specifications for the imprecision and inaccuracy of the method. To select a QC procedure for a method, the process involves assessing method performance (imprecision and inaccuracy) and then selecting the statistical control rules and number of control measurements to be used.

Step-by-step QC Planning process

A more detailed outline of the QC selection process is provided in Figure 2, which identifies the following steps:

  • Define the quality required for the test
  • Assess the method performance in terms of imprecision and inaccuracy
  • Assess QC performance in terms of the rejection characteristics or power curves of candidate control rules with Ns of interest
  • Utilize QC planning tools such as OPSpecs charts
  • Evaluate the probabilities of rejection for the operating conditions in the laboratory
  • Select appropriate control rules and the total number of control measurements
  • Adopt a Total QC strategy that provides an appropriate balance of statistical and non-statistical components
  • Reassess the control rules, N, and TQC strategy when method performance or quality requirements change.

Applications of this step-by-step process can be found elsewhere on this website.

The two key elements in the planning process are the quality requirement and the OPSpecs chart. The quality requirement guides the selection of the method and the QC procedure. The OPSpecs chart [4,5] is a tool for translating a quality requirement into the imprecision and inaccuracy that are allowable for the method and the control rules and N that are required for the QC procedure, i.e., the operating specifications needed at the bench level to assure the quality of test results.

Quality requirements

The most useful format is an interval statement that encompasses the overall or total variation expected in a test result due to the factors of interest in the analytical and pre-analytical parts of the testing process. For the analytical part of a testing process alone, the quality requirement can be stated as an allowable total error, such as often specified by criteria for acceptability in proficiency testing programs [6] or as calculated from biologic goals for imprecision and inaccuracy as recommended by Hyltoft Petersen et. al [7]. For the pre-analytical plus analytical parts, a quality requirement can be stated in the form of a medically important change or decision interval [8]. This clinical requirement requires that pre-analytical factors such as within-subject biological variation of the patient be taken into account when assessing the operating specifications for imprecision, inaccuracy, and QC.

For thyroxine, for example, the US CLIA criterion for acceptability is give as 20% or plus/minus 1.0 mcg/dL, whichever is greater [6]. The European interim biologic goal for imprecision is 4.1% and for inaccuracy is 6.8% which give a calculated allowable total error of 13.6%. Skendzel, Barnett, and Platt [9] recommend that a change of 33% in a patient's test results would be medically significant. Note that the within-subject biologic variation, which is given as 5.8% by Fraser [10], must be taken into account to properly consider the imprecision and inaccuracy allowable for the method itself [11].

Relationship between types of quality requirements and operating specifications

Figure 3 shows the relationships between these different types of quality requirements and the operating specifications that are needed to manage the analytical quality of a laboratory testing process.

The right side illustrates that proficiency testing criteria or calculated total biologic goals can provide analytical requirements in the form of allowable total errors, which can then be translated into operating specifications for the imprecision and inaccuracy that are allowable and the control rules and N that are necessary.The left side shows how treatment guidelines and critical pathways should be a source of information on medically important changes in test results or decision intervals, which in turn can be translated into operating specifications if pre-analytical factors are taken into account. An understanding of this "systems perspective" is essential for properly applying the different types of quality requirements for selecting methods of analysis and quality control procedures.

Tools for quality planning

The relationship between imprecision, inaccuracy, and QC can be shown graphically by an OPSpecs chart, which is derived from an "error budget" or quality-planning model [12]. An analytical quality-planning model is used to relate the allowable total error to the imprecision, inaccuracy, and sensitivity of the QC procedure [6]. A clinical model relates the decision interval to pre-analytical factors, such as within-subject biologic variation, as well as the analytical factors - imprecision, inaccuracy, and QC [8].

OPSpecs chart for a thyroxine example where CLIA sets an allowable total error of 20%

For example, Figure 4 shows an OPSpecs chart that has been prepared for a TEa of 20% (the CLIA criterion for acceptable performance on proficiency testing surveys), as indicated at top.

The heading also states 90% AQA(SE), which means that the chart is derived for 90% Analytical Quality Assurance, or 90% detection of medically important Systematic Errors.Allowable inaccuracy is plotted on the y-axis vs allowable imprecision on the x-axis - both in percent to eliminate any difficulties with concentration units. The top line in the graph corresponds to the bias plus 2 standard deviation criterion for total error that is commonly used in judging the stable performance of a method during initial evaluations. The lines below describe the allowable imprecision and inaccuracy for different control rules and numbers of control measurements, which are defined in the key area at the right. For example, the top line in this group corresponds to a 5-rule multirule procedure with N of 6, whereas the bottom line corresponds to a single-rule with an N of 3. The fact that all these lines are below the limits for stable performance reveals the relative insensitivity of statistical QC procedures that have a low number of control measurements.

To use an OPSpecs chart for setting specifications for the imprecision and inaccuracy of a method, you define the control rules and N desired for routine laboratory practice. For example, if you want to use a 13s control procedure with N=3 in your laboratory, the method needs to have a CV of 4.0% or better if bias is zero (the x-intercept for the operating limits of the 13s rule with N=3). If you are willing to use a multirule procedure with an N of 6, the method can have a CV as high as 6.0% (the x-intercept of the multirule procedure with N of 6).

To use an OPSpecs chart to select an appropriate QC procedure, you need to know the imprecision and inaccuracy of the method. Given a method with an observed CV of 5.0% and an observed bias of 1.0%, these values become the x and y coordinates of the method's "operating point" as shown in the figure. Any QC procedures whose operating limits are above the point provide appropriate QC. The 4-rule multirule procedure with an N of 6 could be selected here.

Technology for quality planning

OPSpecs charts are very simple to use, as illustrated above. The difficulties are to understand why they work (i.e., the theory) and to prepare the individual charts needed for each individual application (i.e., the calculations and graphic presentation). Obviously, the calculations and the graphics can best be handled by computer, either by an electronic spreadsheet that facilitates manual selection [6,8] or by a specialized PC program that can automate the selection process on the basis of user-defined criteria and logic [13,14].

Even with a computer program, the quality planning for immunoassays poses special difficulties due to multiple formats for quality requirements, multiple decision levels, and multiple estimates of method performance at different decision levels. In addition, because immunoassay measurement procedures are generally not as precise as highly automated chemistry and hematology systems, multiple QC designs for "startup" and "monitoring" may be needed to effectively manage the quality of routine testing processes [1]. Because these "multiple" factors require preparation of additional OPSpecs charts, it would be advantageous to compare their effects side-by-side, all at once, as a single quality-planning exercise, in a single computer file.

An example window from a prototype QC planning program
Click here to see full size graphic of the screen.


Future computer programs should be able to address these needs for improvements in quality-planning technology. For example, Figure 5 shows the QC Instructions window of a prototype PC program that considers all these "multiple" factors (multiple formats of quality requirements, multiple design levels, multiple estimates, of method performance, and multiple designs). Three different QC designs that can be selected simultaneously and each can consider up to three different types of quality requirements, up to four different medical decision levels, and estimates of method performance at each of these decision levels.

In this example, the three designs have been customized to provide "Startup", "Monitor", and "Average of Normals" [15] QC designs for immunoassay applications. Consideration of these "multiple" factors makes this prototype program much more complex, however, the "folder-format" of the user interface simplifies the navigation and the operation of the program.

Training for quality planning

Quality-planning tools and technology will not by themselves improve the management of quality in healthcare laboratories! Training is also needed to support the efforts of the directors, managers, supervisors, and analysts who work in these laboratories. Fortunately, training can now be delivered directly to a laboratory through internet courses, such as the Basic QC Practices course, along with tutorials for learning operation of the program. Thus, the tools, technology, and training can be delivered directly to laboratories anywhere and anytime through the internet. For laboratories where internet access is slow or limited, this course can be provided on CD-ROM.

References

  1. Mugan K, Carlson IH, Westgard JO. Planning QC procedures for immunoassays. J Clin Immunoassay 1994;17:216-222.
  2. Seth J. Quality Assurance. Chapter 10 in Principles and Practices of Immunoassay, Price CP and Newman DJ, 2nd ed. London:Macmillan Reference Ltd, 1997.
  3. Carey RN. Tips for managing the quality of immunoassays.
  4. Westgard JO. Charts of operating specifications (OPSpecs charts) for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing criteria. Clin Chem 1992;38:1226-1233.
  5. Westgard JO. Assuring analytical quality through process planning and quality control. Arch Pathol Lab Med 1992;116:765-769.
  6. Westgard JO, Wiebe DA. Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing. Clin Chem 1991;37:1938-1944.
  7. Hyltoft Petersen P, Ricos C, Stockl D, Libeer J-C, Baadenhuijsen H. Fraser CG, Thienpont L. Proposed guidelines for the internal quality control of analytical results in the medical laboratory. Eur J Clin Chem Clin Biochem 1996;34:983-999.
  8. Westgard JO, Hyltoft Petersen P, Wiebe DA. Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program. Clin Chem 1991;37:656-661.
  9. Skenzel LP, Barnett RN, Platt R. Medically useful criteria for analytic performance of laboratory tests. Am J Clin Pathol 1985;83:200-205.
  10. Fraser CG. The application of theoretical goals based on biological variation data in clinical chemistry. Arch Pathol Lab Med 1988;112:404-415.
  11. Westgard JO. Error budgets for quality management: Practical tools for planning and assusring the analytical quality of laboratory testing processes. Clin Lab Manag Review 1996;10:377-403.
  12. Westgard JO, Seehafer JJ, Barry PL. Allowable imprecision for laboratory tests based on clinical and analytical test outcome criteria. Clin Chem 1994;40:1909-1914.
  13. Westgard JO, Stein B, Westgard SA, Kennedy R. QC Validator 2.0: a computer program for automatic selection of statistical QC procedures for applications in healthcare laboratories. Computer Method Programs Biomed 1997;53:175-186.
  14. Westgard JO, Stein B. Automated selection of statistical quality-control procedures to assure meeting clinical or analytical quality requirements. Clin Chem 1997;43:400-403.
  15. Westgard JO, Smith FA, Mountain PJ, Boss S. Design and assessment of average of normals (AON) patient data algorithms to maximize run lengths for automatic process control. Clin Chem 1996;41:1683-1688.