QC Design

Stockl et al's landmark paper on vitamin D establishes desirable specifications for imprecision and inaccuracy (trueness). But what about total allowable error? With the use of QC Design tools, we can turn individual specifications into a broader quality requirement.

• Total allowable error from Biologic Goals (TE_ba)
  
• Sigma-metric assessment of recommended S/P-25(OH)D performance
  • Maximum allowed Bias and CV
  • Better Bias and CV
  • Optimal Bias and CV
• Conclusion
  

November 2009

As noted in a recent article on Westgard Web, the explosion in use of vitamin D testing (specifically S/P-25(OH)D methods) has not been accompanied by a surge in interest in the quality of said tests. But thankfully Stockl and colleagues finally established some good goals for performance:

Dietmar Stockl, Patrick M. Sluss, Linda M. Thienpont, Specifications for trueness and precision of a reference measurement system for serum/plasma 25-hydroxyvitamin D analysis. Clinical Chimica Acta, 408 (2009): 8-13.[1]

To briefly summarize, Stockl et al applied four different approaches for setting quality specifications for a S/P-25(OH)D test, following the guidance from the 1999 Stockholm consensus conference on quality specifications. [2] They finally determined that the best and most realistic recommendation for routine measurement performance was

"a maximum CV and B of 10% and 5%, respectively. It should, however, be emphasized that for ameasurement procedure to achieve the required quality routinely, its "stable" [that is average] imprecision and bias should not be at the upper limit, but approximately half of it to ensure that the maximum goal is not exceeded."[1]

This statement may seem confusing to the average reader. Why give a maximum recommendation, then recommend that the routine be half? We'll show you why. Plus, we'll explore the combined effects of imprecision and bias and calculate the total allowable error.

Total Allowable Error from Biologic Goals. (TE_BA)

Total Allowable Errors are traditionally set for analytical performance, possibly set without knowledge of biologic variation or the clinical decision interval of the test. Callum Fraser, Per Hytoft Petersen, Carmen Ricos, and a number of other European colleagues came up with another approach - one that calculated the total allowable error based on the known within-subject biologic variation of the individual for that analyte. To see discussions in greater detail of this approach, click here and here.

For S/P-25(OH)D, we can calculate the total error using the traditional calculation of TE = bias + 2s.

That is 5 + 2*(10) = 25%

Sigma-metric assessment of recommended S/P-25(OH)D performance

Now, we're going to make use of a QC Design tool, our EZ Rules 3 software program, to make some more difficult calculations calculations and compare the effects of different method performance.

Let's start by using the recommended specifications for routine performance, a CV of 5% and a bias of 2.5% [The graphic compresses this screen shot, so it looks a bit fuzzy here, but down on the left column, near the bottom, you might be able to see the entered performance characteristics below the words "Operating Point." If not, you may be able to right-click the graphic and view the image by itself in its full size]:

If we focus on the Sigma-metrics graph produced by the program, we can see that EZ Rules has already assesses the Sigma-metric of the method (2.0) and automatically chosen the optimal QC procedure for this method performance. Acutally, there isn't an optimal QC procedure if you have both the maximum allowed CV and Bias - the program reverts to a Maximum QC procedure when performance is low Sigma. This is a "full Westgard Rules" multirule procedure with extra controls (N=6). At Stockl, Sluss, and Thienpont indicated in their essay, their specifications are no good if your method has both the maximum allowed inaccuracy and imprecision.

Let's look at what happens when your method has more better CV and Bias (5% and 2.5%, respectively):

Note that the Sigma-metric of this level of performance is now 4.5 Sigma - good but not great,  definitely above the minimum performance threshhold of 3 Sigma. Also note that the recommended QC procedure has an N of 4. That means either running 4 controls per run, or measuring two controls twice per run. This is more than the usual number of control measurements. But it is a simple 1_2.5s control rule, which maximizes error detection while minimizing false rejection.

If we take the optimum recommendation from Stockl et al's paper: CV of 4% and bias of 2.6%, we get a different result:

Here's a close up of the Sigma-metrics chart:

You can see that EZ Rules 3 assesses performance at 5.6 Sigma, which is quite good. Performance here is so good that you only need 2 controls per run, and can use single control limits of 3s. Essentially, there will be nearly 100% detection of medical significant errors within the first run they occur, and nearly 0% false rejections.

Conclusion

Stockl et al's paper provides us with a wealth of recommendations for performance for vitamin D assays. With the use of QC Design tools, we can turn those recommendations into real-world operating specifications.