## Break all the rules (Part 3)

After a significant time elapsed since parts one and two, we continue our series on basic QC interpretation. We are finally tackilng that most complex scenario: what happens when we have 3 controls? How do we interpret across materials/levels? Does that mean we stop interpreting across runs? Here's another simple exercise - if you're told that all the rules have been broken, can you find where the violations occurred?

## Break All the Rules! (Part Three)

#### Sten Westgard, MSJuly 2017

As laboratorians, we are constantly pressured to achieve compliance, to operate with an absence of problems, to generate a stream of data points that land lovingly near the mean. As a contrarian exercise, this article is about the exact opposite of getting all your data to stay in control. We're going to look at data that's out-of-control, breaking all of the various "Westgard Rules." And we're going to see if you can identify which data points are outliers and which rules are being broken.

As the title suggests, this is third installment. Part One was just one control. Part Two was two controls. Now we're going to get about as complicated as most labs gets. We're going to look at rule violations with three controls.

### The data

The following data set is going to have a low control with a mean of 47 and an SD of 3 and a middle control with a mean of 71 and an SD of 8 and a high control of 256 with an SD of 18. I'm not going to tell you which test or what units are involved. They're irrelevant to this exercise. We're also not going to look at the total allowable error or CV or bias. All of that becomes important after we know the basics of QC and charting.

To make it a little easier to understand this data, here are the Levey-Jennings charts for these first 29 values (notice for each control, not only is the raw value given, but the z-score is shown as well - that's really useful):

 Values Control 1 1-z Control 2 2-z Control 3 3 -z RULEBROKEN? Run 1 51.5 1.5 82.2 1.4 272.2 0.9 Run 2 38.9 -2.7 51 -2.5 288.4 1.8 Run 3 47.6 0.2 74.2 0.4 245.2 -0.6 Run 4 46 -0.33333 91.8 2.6 212.8 -2.4 Run 5 50.9 1.3 74.2 0.4 239.8 -0.9 Run 6 53.9 2.3 69 -0.25 211 -2.5 Run 7 52.7 1.9 58.2 -1.6 263.2 0.4 Run 8 49.1 0.7 83.3 1.54 324.4 3.8 Run 9 48.5 0.5 65.4 -0.7 286.6 1.7 Run 10 37 -3.33 90 2.375 230.8 -1.4 Run 11 47 0 63 -1 261.0 0.28 Run 12 54 2.333333 82.2 1.4 239.8 -0.9 Run 13 50.3 1.1 65.4 -0.7 259.6 0.2 Run 14 40 -2.33333 86.2 1.9 223.6 -1.8 Run 15 55.4 2.8 70.2 -0.1 254.2 -0.1 Run 16 49.4 0.8 88.6 2.2 290.2 1.9 Run 17 46.7 -0.1 87.8 2.1 261.4 0.3 Run 18 45.8 -0.4 78 0.875 256.0 0.0 Run 19 52.7 1.9 76.6 0.7 293.8 2.1 Run 20 51.2 1.4 65.4 -0.7 261.4 0.3 Run 21 49 0.667 86.2 1.9 304.6 2.7 Run 22 53 2 71 0 248.8 -0.4 Run 23 49.7 0.9 81.4 1.3 241.6 -0.8 Run 24 48.2 0.4 90.2 2.4 308.2 2.9 Run 25 46.4 -0.2 80.6 1.2 221.8 -1.9 Run 26 54.2 2.4 90 2.375 243.4 -0.7 Run 27 51.2 1.4 53 -2.25 266.8 0.6 Run 28 53.9 2.3 76.6 0.7 308.2 2.9 Run 29 44.3 -0.9 72.6 0.2 263.2 0.4

If the table is too hard to work with, here are the three Levey-Jennings charts (click on the graph to enlarge the figures):

Here is the second table of QC values for runs 30 through 59:

 Values Control 1 1-z Control 2 2-z Control 3 3-z Rule Broken? 30 48.5 0.5 84.6 1.7 279.4 1.3 31 48.8 0.6 79.8 1.1 254.2 -0.1 32 46.7 -0.1 82.2 1.4 250.6 -0.3 33 46.1 -0.3 83.8 1.6 265 0.5 34 49.7 0.9 72.6 0.2 279.4 1.3 35 53.9 2.3 63.8 -0.9 286.6 1.7 36 52.1 1.7 60.6 -1.3 284.8 1.6 37 48.2 0.4 62.2 -1.1 245.8 -0.6 38 52.1 1.7 71 0 286.6 1.7 39 50.6 1.2 69.4 -0.2 261.4 0.3 40 50.3 1.1 64.6 -0.8 245.2 -0.6 41 44.3 -0.9 87.8 2.1 263.2 0.4 42 51.8 1.6 79.8 1.1 281.2 1.4 43 41.9 -1.7 64.6 -0.8 284.8 1.6 44 49.4 0.8 63.8 -0.9 261.4 0.3 45 48.8 0.6 75.8 0.6 241.6 -0.8 46 49.9 0.97 66.2 -0.6 263.2 0.4 47 52.4 1.8 71 0 293.8 2.1 48 52.7 1.9 82.2 1.4 248.8 -0.4 49 49.1 0.7 56.6 -1.8 223.6 -1.8 50 44 -1 66.2 -0.6 229 -1.5 51 48.8 0.6 82.2 1.4 247 -0.5 52 50.6 1.2 77.4 0.8 266.8 0.6 53 43.7 -1.1 78.2 0.9 290.2 1.9 54 47.6 0.2 75 0.5 281.2 1.4 55 45.5 -0.5 85.4 1.8 245.2 -0.6 56 44.3 -0.9 86.2 1.9 261.4 0.3 57 52.4 1.8 55.8 -1.9 221.8 -1.9 58 53.9 2.3 65.4 -0.7 277.6 1.2 59 45.2 -0.6 83.8 1.6 284.8 1.6

And now the Levey-Jennings charts for runs 3 through 59:

Next, we're ready to see the table of data for runs 60 through 89:

 Values Control 1 z 1 Control 2 Z 2 Control 3 z 3 RuleBroken? 60 45.8 -0.4 74.2 0.4 277.6 1.2 61 50.6 1.2 64.6 -0.8 284.8 1.6 62 48 0.3 83 1.5 266.8 0.6 63 49 0.7 83 1.5 259.6 0.2 64 46 -0.3 67.1 -0.49 260.1 0.23 65 53 2.0 74.2 0.4 281.2 1.4 66 48 0.3 80.6 1.2 241.6 -0.8 67 45.8 -0.4 71.8 0.1 272.2 0.9 68 48.2 0.4 69.4 -0.2 288.4 1.8 69 50.9 1.3 81.4 1.3 250.6 -0.3 70 48.2 0.4 65.4 -0.7 277.6 1.2 71 51.2 1.4 70 -0.125 221.8 -1.9 72 49.1 0.7 69 -0.25 263.2 0.4 73 51.5 1.5 66 -0.625 247 -0.5 74 47.9 0.3 68.6 -0.3 254.2 -0.1 75 48.5 0.5 86.2 1.9 261.4 0.3 76 50.9 1.3 59.8 -1.4 266.8 0.6 77 46.7 -0.1 67.8 -0.4 252.4 -0.2 78 47.8 0.3 86.2 1.9 236.2 -1.1 79 45.5 -0.5 72.6 0.2 250.6 -0.3 80 52 1.7 81.4 1.3 259.6 0.2 81 52 1.7 74.2 0.4 243.4 -0.7 82 48.5 0.5 80.6 1.2 275.8 1.1 83 44.3 -0.9 79.8 1.1 281.2 1.4 84 49.4 0.8 75 0.5 252.4 -0.2 85 46.7 -0.1 76.6 0.7 272.2 0.9 86 52.7 1.9 72.6 0.2 281.2 1.4 87 45.2 -0.6 55.8 -1.9 275.8 1.1 88 48.8 0.6 79 1 256 0 89 46.4 -0.2 70.2 -0.1 236.2 -1.1

Now there are some Levey-Jennings charts for those runs 60 through  89:

Now the data table for runs 90 through 119:

 Values Control 1 Z 1 Control 2 Z 2 Control 3 Z 3 Rules 90 49.7 0.9 82.2 1.4 275.8 1.1 91 47 0 68.6 -0.3 265 0.5 92 45.5 -0.5 60.6 -1.3 272.2 0.9 93 46.4 -0.2 79 1 283 1.5 94 51.5 1.5 59.8 -1.4 288.4 1.8 95 48.8 0.6 66.2 -0.6 259.6 0.2 96 44.9 -0.7 77.4 0.8 274 1 97 50.3 1.1 74.2 0.4 281.2 1.4 98 44.3 -0.9 65.4 -0.7 265 0.5 99 46.7 -0.1 63 -1 245.2 -0.6 100 50.6 1.2 74.2 0.4 268.6 0.7 101 49.4 0.8 85.4 1.8 279.4 1.3 102 47.6 0.2 80.6 1.2 288.4 1.8 103 46.7 -0.1 60.6 -1.3 234.4 -1.2 104 50 1 71 0 275.8 1.1 105 51.8 1.6 72.6 0.2 275.8 1.1 106 47.6 0.2 70.2 -0.1 250.6 -0.3 107 50 1 68.6 -0.3 261.4 0.3 108 49.1 0.7 81.4 1.3 230.8 -1.4 109 50.3 1.1 78.2 0.9 232.6 -1.3 110 48.8 0.6 69.4 -0.2 265 0.5 111 51.8 1.6 63 -1 256 0 112 50.6 1.2 82.2 1.4 252.4 -0.2 113 47.3 0.1 80.6 1.2 238 -1 114 48.8 0.6 76.6 0.7 230.8 -1.4 115 47.8 0.3 62.2 -1.1 239.8 -0.9 116 47 0 75 0.5 234.4 -1.2 117 47.6 0.2 81.4 1.3 275.8 1.1 118 46.7 -0.1 83.8 1.6 268.6 0.7 119 44.9 -0.7 72.6 0.2 236.2 -1.1

Here's our final table of data, with values 120 through 150. We're just looking for the 12:x violation now:

 Values Control 1 Z 1 Control 2 Z 2 Control 3 Z 3 RulesBroken? 120 46.7 -0.1 62.2 -1.1 239.8 -0.9 121 47 0 75 0.5 234.4 -1.2 122 47.6 0.2 81.4 1.3 275.8 1.1 123 46.7 -0.1 83.8 1.6 268.6 0.7 124 44.9 -0.7 72.6 0.2 236.2 -1.1 125 50.9 1.3 79 1 263.2 0.4 126 52.7 1.9 76.6 0.7 230.8 -1.4 127 46.4 -0.2 80.6 1.2 232.6 -1.3 128 44 -1 75.8 0.6 270.4 0.8 129 51.2 1.4 86.2 1.9 268.6 0.7 130 48.5 0.5 80.6 1.2 252.4 -0.2 131 49.1 0.7 75.8 0.6 238 -1 132 51.8 1.6 75.8 0.6 248.8 -0.4 133 42.2 -1.6 58.2 -1.6 254.2 -0.1 134 43.4 -1.2 70.2 -0.1 265 0.5 135 50.3 1.1 69.4 -0.2 279.4 1.3 136 49.1 0.7 72.6 0.2 284.8 1.6 137 43.7 -1.1 70.2 -0.1 259.6 0.2 138 48.2 0.4 65.4 -0.7 274 1 139 42.8 -1.4 81.4 1.3 268.6 0.7 140 43.1 -1.3 78.2 0.9 277.6 1.2 141 48.8 0.6 69.4 -0.2 266.8 0.6 142 49.1 0.7 63 -1 290.2 1.9 143 46.4 -0.2 82.2 1.4 277.6 1.2 144 44 -1 75 0.5 266.8 0.6 145 45.8 -0.4 76.6 0.7 266.8 0.6 146 45.5 -0.5 59 -1.5 243.4 -0.7 147 51.2 1.4 81.4 1.3 266.8 0.6 148 48.5 0.5 78.2 0.9 277.6 1.2 149 49.1 0.7 73.4 0.3 266.8 0.6 150 50.3 1.1 75 0.5 270.4 0.8

Here are our last graphs of those runs:

That should be enough data to keep you out of trouble. And the number of outliers you find in this dataset should be equivalent to a year's worth of outliers that you see normally in your own lab data.

Finally, I'm going to add the following conditions:

• We are running THREE controls per run; assume each trio of values represents one run
• We're NOT going to look for 2s violations. Let's break that habit right now. In any case, I didn't generate these data points randomly, so we should not expect to see a certain number of "natural false rejections" in the form of 2s outliers
• We ARE going to try to find outliers of the following "Westgard Rules": 1:3s/2of3:2s/R:4s/3:1s/6:x   (and later in the exercise, we switch to 9:x and12:x)
• We're NOT going to sink into the swamp that is the repeating of controls. So don't assume anything you see is a repeated control - just stick with "in" or "out"

So, at the end of this, can you answer the following questions?

• How many rule violations were there?
• Which rules were violated and how many times?
• What would be different if I told you that only the "Westgard Rules" 1:3s/2of3:2s/R:4s were necessary?
• What would be different if I told you that only the 1:3s control rule was necessary?

These answers, and more, in a few weeks...

[Don't submit this exercise for grading. You'll get to compare your answers to the answer set in a few weeks. But feel free to share this with your colleagues and see if you can all reach the same conclusions and interpretations.]