Questions
Questions from Breaking all the Rules
In November 2000, we held the “Breaking all the Rules” Zoom. There were many questions asked, not all of which could be answered in an hourlong zoom. So here are a few lingering questions, abstracted so they are easier to understand outside of the zoom.
Questions from "Breaking all the Rules" Zoom
Sten Westgard, MS
November 2020
Q: If my high control is exceeding +3 sd, but my low control is – 2 sd, do I have a 1:3s violation, an R:4s violation, or both?
A: If we think about the “Westgard Rules”, you note there is an order of interpretation, 1:2s/1:3s/2:2s/R:4s/4:1s/8:x for the classic rules, and 1:3s/2:2s/R:4s/4:1s/8:x for the modern recommendation. Thus, if you were implementing the classic Westgard rules, you would set of the 2s warning, then you would look at the 1:3s rule, which would be violated, that’s a rejection. If you were using the modern Westgard rules, you don’t have a warning rule anymore, you just proceed directly to the 1:3s rule, which is violated.
So, in a formal sense, the first rule violated is 1:3s and you reject the run.
As you trouble-shoot the run, certainly you would pick up the fact that the R:4s has also been violated. And it’s rather convenient, since when either the 1:3s or the R:4s are violated, the violation is typically indicative of random error. Having two rules that point to random error violated at the same time is a real sign that you should look at random error causes (bubbles, electric fluctuations, etc.) as the root cause of your error.
Q: Should we consider QC across assays on the same instrument?
A: When we teach Westgard Rules we often discuss across-run interpretation (where we look at the current run and a number of previous runs) and across-level interpretation (where we look at both the high and low controls, or the high, low, and intermediate controls, all together). But the statistics and theory are only able to predict error detection and false rejection within a single test. There is definite advantage to comparing the performance of tests that use the same principle, same sensor, same lamp, etc. If both AST and ALT are out, that’s a really useful clue. If only one of them is out and the other is fine, that will send your troubleshooting in a different direction. If your sodium, chloride, potassium are all going haywire, that’s going to focus you on the ISE elements of the instrument. These are professional judgments that you can add to the interpretation of QC, but they can’t really be captured by statistical theory. So it’s a good thing you’re there looking at the QC of the instrument (they can’t totally replace you with a robot yet).
Q: Are you suggesting that we don’t rerun the control first?
A: As always, whenever we talk QC, we get questions about repeating controls. Despite decades of knowing this is a bad practice, laboratories still do it. It’s an easy way to get back “in” even if it may just be postponing a problem or allowing it to grow worse. Ideally, when selecting or planning SQC, the rules should be chosen to provide the necessary error detection with minimum false rejection. For testing processes with high Sigmas, single rules such as 1:3s or 1:2.5s may be used with Ns of 2 to provide high error detection and low false rejection.
There are justified times to run a control again. If you troubleshoot, if you make a change to the system, if you attempt a fix of the problem, then you run controls again to see if your fix worked. That’s fundamentally different than the wasteful “do over” QC – where one doesn’t make any attempt at troubleshooting or improvements, but simply is rolling the dice to see what happens.
So, if you get an out-of-control flag, and your immediate response is just to repeat the control, that’s a bad practice. That’s wasting your time and money. And if you have poorly designed QC procedures (i.e. you never rationally designed the rules you are using), you are probably following up a false rejection with an unnecessary repeat. It’s a wrong action to try and correct a wrong outlier – double waste.
Given how much pressure there is on laboratories to save time and money, it’s amazing that pure waste like repeating controls is still allowed to exist. (It doesn’t help that some control vendors encourage it.)
Q: If the peer group mean is running below the manufacturer’s package insert control mean and my instrument is still within the package insert control ranges does that mean I have a problem with my instrument?
A: Ok, let’s parse this question, there’s a few issues in here to dissect. First, If there is a package insert control mean, that’s only meant as a starting point. And we might almost expect that half the time, the peer group mean should be above that insert value, and the other half of the time it should be below that value. So whether or not the peer group mean is above or below an expected mean is not so much to worry about – it’s how far away from the expected mean it gets. If the peer group mean is so far below the insert mean that it’s outside the range of the package insert, that probably indicates the package insert was wrong in the first place or something major has happened with the reagent lot or calibration on all the instruments in the peer group.
Second, the package insert control range is NOT something you should use to judge acceptability. At least, you shouldn’t use the package insert data after you have completed the initial period of establishing the actual mean and SD of the new control. The package insert data is what you use when you have no other reliable information. As soon as you have accumulated 20 data points of performance in your own lab, you switch over to that observed mean and SD. And as you get even more data, you update your estimates of mean and SD with one month, two months, three months, etc., of actual performance. That gives you a much better picture of what’s going on with the instrument.
If you have a question that wasn’t answered here, please feel free to send it in to