A simple example planning QC for hemoglobin using OPSpecs charts.
A simple example showing
QC Planning with OPSpecs Charts
This month, we're using a hemoglobin example to illustrate the application of the QC planning process to select appropriate QC procedures for your laboratory tests. We recommend the 8-step process shown below. This QC planning process allows you to select control rules and numbers of control measurements (N) to monitor individual tests on the basis of the quality required by the test, the observed accuracy and precision of a method in your laboratory, and the expected error detection and false rejection characteristics of the QC procedure itself. At first, this planning process will seem formidable, but with practice it will become quick and easy to do.
- Example data
- 1. Define the Quality Requirement
- 2. Evaluate the precision and accuracy of the method
- 3. Look Up OPSpecs Charts
- 4. Plot Operating Point
- 5. Assess probabilities of rejection
- 6. Select the control rules and numbers of control measurements
- 7. Adopt a Total QC Strategy
- 8. Reassess for changes in performance
- Using QC Validator to choose a rule
- For further reading
- Test: Hemoglobin
- Decision level: 9 g/dL (i.e. this is where we are running one of the controls because this is an important level for test interpretation)
- Imprecision SD: 0.117 g/dL
- Inaccuracy (bias): 0 g/dL
- Number of control materials: 3 (that is, we intend to analyze three different concentrations or levels of control materials)
Each individual test has its own quality requirements in clinical or analytical terms. For practical purposes, the CLIA proficiency testing criteria for acceptability provide the minimum total error requirements for all regulated tests. For example, the total error that is allowable for a hemoglobin method at a decision level or target value of 9 g/dL is 7%. This means a test result should be correct within 0.63 g/dL.
Method performance can be assessed from initial method evaluation studies, on-going validation studies, and current performance on internal and external control materials. It is best that these performance characteristics reflect how the method works in your laboratory, however, estimates can also be obtained from evaluation studies in the literature and from manufacturer’s claims in product labeling. In this example, our hemoglobin method has a 0.117 g/dL SD and no bias.
The CV and bias must be expressed in percentages. Use the control level (the level at which you run controls, a target value where a small change becomes important, etc.) as a basis for calculation. For hemoglobin, this level is 9 g/dL. Now if your SD is 0.117 g/dL, then the %CV is 0.117/9 = 1.3%. The calculation for bias is easy - if there is no bias, then the % bias is 0 as well.
Since the analytical quality requirement is 7%, we should look at OPSpecs charts for a 7% allowable total error, or TEa. Furthermore, if the instrument for our method runs three control materials, then we want to look at control rules with numbers of measurement (N's) of 3 and 6. Thus, we want to look at OPSpecs charts for a 7% analytical quality requirement and N's of 3 and 6. There are several ways to obtain these OPSpecs charts. For those with a computer, the QC Validator program will produce OPSpecs charts for any quality requirement and three different levels of error detection (90% AQA, 50% AQA, and 25% AQA). For this example, however, we are going consult an OPSpecs Manual, which is a library of OPSpecs charts for common control rules and quality requirements. An index of charts tells us that we can find the relevant OPSpecs charts on pages 3-50 and 3-51 of the OPSpecs Manual, Expanded Edition. A two page layout displays 4 OPSpecs charts in the following order:
The objective is to achieve 90% error detection with the lowest N possible. Thus, you start by looking at OPSpecs chart with the highest error detection and lowest N.
Once we've obtained the OPSpecs charts, we can plot our operating point. Use the value for CV as the x-coordinate. Use the value for bias for the y coordinate. Thus for 1.3% CV and 0% bias, we have an operating point of x=1.3, y=0. We start with the chart for 90% AQA (top left of the 2 page layout).
OPSpecs chart for a 7% analytical quality requirement, 90% AQA, and N=3. The order of the graph lines matches the order of the rules in the key, top to bottom (i.e. the top line in the graph matches the top control rule in the key). If there are lines above the operating point, the QC procedure will provide the error detection listed in the title (in this case, 90%).
Those QC procedures whose operating limits (lines of the graph) are above the operating point will provide the error detection listed in the title of the chart. For our hemoglobin example, there are five lines above the operating point. Any one of the five control rules guarantees over 90% error detection. So we actually don't have to look at any more OPSpecs charts; we've found all that we need on the first one.
Check the key at the right of the graph to match up the lines that passed above your operating point with the control rules and N's they represent. Note the column labelled Pfr lists the false rejection of each control rule. Ideally, you want to aim for a Pfr of 5% or less.
This is the key from the N=3 OPSpecs chart shown above. Note the column which lists the Pfr, False Rejection.
In our example, we have five choices:
- The single rule 12s with 14% false rejection,
- The multirule 13s/2of32s/R4s/31s multirule with 2% false rejection,
- The single rule 12.5s with 3% false rejection,
- The multirule 13s/2of32s/R4s multirule with 1% false rejection, or
- The single rule 13s with 1% false rejection
Of the five control rules above the operating point, the 12s control rule with N=4 will have an 14% false rejection rate. That means even if your method is stable and has no problems, you will still have an out-of-control signal about every seventh run. This is an unacceptable level of false alarms. The second QC procedure is a 13s/2of32s/R4s/31s multirule with N=4 has a 2% false rejection rate. This is an excellent rate, but the multirule may be too complicated for many laboratories to implement. The third QC procedure, 12.5s has a 3% false rejection rate -- good but we can still do it better (and simpler). The fourth, 13s/2of32s/R4s has a rejection rate of just 1%, but again, a multirule may be too difficult to implement. Finally, the 13s control rule has a false rejection rate of 1%. This is a simple control rule with very low false rejection rate.
Conclusion: The 13s single rule with N=3 satisfies our conditions by providing more than 90% error detection with less than 5% false rejection. This is the control rule we choose for the method.
Note:In situations where 90% error detection cannot be achieved by increasing N and/or using multi-rule criteria, then you may consider optimizing performance for the observed or expected stability of the process (frequency of errors). Consider 50% error detection for moderately stable processes. Consider 25% error detection for highly stable processes that seldom have problems.
The Total QC strategy includes statistical QC, as well as other QC components such as preventive maintenance, system function checks, measurement validation tests, patient data QC, and finally quality improvement. The appropriate balance of these TQC components can be decided based on the error detection that is available from your statistical QC procedure.
Conclusion: In our example, we achieved high error detection (greater than 90%) and low false rejection (less than 5%), with an N of only 3. Therefore, we can rely on statistical QC, so we choose a High Ped TQC strategy. We can now minimize our QC costs for this test and concentrate our energies on the more difficult tests in our laboratory.
This QC planning process should be repeated whenever there are changes in the performance of the method. If performance improves, it may be possible to widen the control limits or lower the N. If performance deteriorates, it may be necessary to increase the amount of QC, increasing N and changing the control rules to narrow our limits, or increasing the rules to form multirule procedures.
Changes in imprecision and inaccuracy can be examined using the same OPSpecs chart. For instance, if the bias increased to 1.0%, we can assess its QC effects by plotting a new operating point of 1.3% CV and 1.0% bias (red dot). With this new bias, it would no longer be possible to control the test with a 13s single rule or a 13s/2of32s/R4s multirule with N=3. If bias increased to 1.5% (blue dot), we would have to use the 13s/2of32s/R4s/31s multirule. If, on the other hand, we could maintain the zero bias and reduce imprecision from 1.3% to 1.0% (green dot), we could use the 13.5s control rule, which has essentially no false rejection.
If we had used QC Validator for the original example, we would have entered all the example data into the parameters screen. Click on the icon at right to see a full-size screen shot of what the parameters screen would look like:
QC Validator also selected the 13s control rule with N of 3 (indicated by the solid black line in the graph and the key; the other rules have dotted or dashed lines). It's the same rule we selected using manual inspection of OPSpecs charts, but Validator has done the work for us. Validator also displays a few more control rules that we can choose if we decide we don't want to use a 13s control rule.
- Westgard JO. "Assuring analytical quality through process planning and quality control." Arch Pathol Lab Med 1992;116:765-769.
- Westgard JO, Wiebe DA. "Cholesterol operational process specifications for assuring the quality required by CLIA proficiency testing." Clin Chem 1991;37:1938-1944.
- Westgard JO. "Charts of operational process specifications (‘OPSpecs charts’) for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing performance criteria." Clin Chem 1992;38:1226-1233.