Break all the rules (Part 3)

After a significant time elapsed since parts one and two, we continue our series on basic QC interpretation. We are finally tackilng that most complex scenario: what happens when we have 3 controls? How do we interpret across materials/levels? Does that mean we stop interpreting across runs? Here's another simple exercise - if you're told that all the rules have been broken, can you find where the violations occurred?

Break All the Rules! (Part Three)

Sten Westgard, MS
July 2017

As laboratorians, we are constantly pressured to achieve compliance, to operate with an absence of problems, to generate a stream of data points that land lovingly near the mean. As a contrarian exercise, this article is about the exact opposite of getting all your data to stay in control. We're going to look at data that's out-of-control, breaking all of the various "Westgard Rules." And we're going to see if you can identify which data points are outliers and which rules are being broken.

As the title suggests, this is third installment. Part One was just one control. Part Two was two controls. Now we're going to get about as complicated as most labs gets. We're going to look at rule violations with three controls.

The data

The following data set is going to have a low control with a mean of 47 and an SD of 3 and a middle control with a mean of 71 and an SD of 8 and a high control of 256 with an SD of 18. I'm not going to tell you which test or what units are involved. They're irrelevant to this exercise. We're also not going to look at the total allowable error or CV or bias. All of that becomes important after we know the basics of QC and charting.

To make it a little easier to understand this data, here are the Levey-Jennings charts for these first 29 values (notice for each control, not only is the raw value given, but the z-score is shown as well - that's really useful):

Values	Control 1	1-z	Control 2	2-z	Control 3	3 -z	RULE BROKEN?
Run 1	51.5	1.5	82.2	1.4	272.2	0.9
Run 2	38.9	-2.7	51	-2.5	288.4	1.8
Run 3	47.6	0.2	74.2	0.4	245.2	-0.6
Run 4	46	-0.33333	91.8	2.6	212.8	-2.4
Run 5	50.9	1.3	74.2	0.4	239.8	-0.9
Run 6	53.9	2.3	69	-0.25	211	-2.5
Run 7	52.7	1.9	58.2	-1.6	263.2	0.4
Run 8	49.1	0.7	83.3	1.54	324.4	3.8
Run 9	48.5	0.5	65.4	-0.7	286.6	1.7
Run 10	37	-3.33	90	2.375	230.8	-1.4
Run 11	47	0	63	-1	261.0	0.28
Run 12	54	2.333333	82.2	1.4	239.8	-0.9
Run 13	50.3	1.1	65.4	-0.7	259.6	0.2
Run 14	40	-2.33333	86.2	1.9	223.6	-1.8
Run 15	55.4	2.8	70.2	-0.1	254.2	-0.1
Run 16	49.4	0.8	88.6	2.2	290.2	1.9
Run 17	46.7	-0.1	87.8	2.1	261.4	0.3
Run 18	45.8	-0.4	78	0.875	256.0	0.0
Run 19	52.7	1.9	76.6	0.7	293.8	2.1
Run 20	51.2	1.4	65.4	-0.7	261.4	0.3
Run 21	49	0.667	86.2	1.9	304.6	2.7
Run 22	53	2	71	0	248.8	-0.4
Run 23	49.7	0.9	81.4	1.3	241.6	-0.8
Run 24	48.2	0.4	90.2	2.4	308.2	2.9
Run 25	46.4	-0.2	80.6	1.2	221.8	-1.9
Run 26	54.2	2.4	90	2.375	243.4	-0.7
Run 27	51.2	1.4	53	-2.25	266.8	0.6
Run 28	53.9	2.3	76.6	0.7	308.2	2.9
Run 29	44.3	-0.9	72.6	0.2	263.2	0.4

If the table is too hard to work with, here are the three Levey-Jennings charts (click on the graph to enlarge the figures):

Here is the second table of QC values for runs 30 through 59:

Values	Control 1	1-z	Control 2	2-z	Control 3	3-z	Rule Broken?
30	48.5	0.5	84.6	1.7	279.4	1.3
31	48.8	0.6	79.8	1.1	254.2	-0.1
32	46.7	-0.1	82.2	1.4	250.6	-0.3
33	46.1	-0.3	83.8	1.6	265	0.5
34	49.7	0.9	72.6	0.2	279.4	1.3
35	53.9	2.3	63.8	-0.9	286.6	1.7
36	52.1	1.7	60.6	-1.3	284.8	1.6
37	48.2	0.4	62.2	-1.1	245.8	-0.6
38	52.1	1.7	71	0	286.6	1.7
39	50.6	1.2	69.4	-0.2	261.4	0.3
40	50.3	1.1	64.6	-0.8	245.2	-0.6
41	44.3	-0.9	87.8	2.1	263.2	0.4
42	51.8	1.6	79.8	1.1	281.2	1.4
43	41.9	-1.7	64.6	-0.8	284.8	1.6
44	49.4	0.8	63.8	-0.9	261.4	0.3
45	48.8	0.6	75.8	0.6	241.6	-0.8
46	49.9	0.97	66.2	-0.6	263.2	0.4
47	52.4	1.8	71	0	293.8	2.1
48	52.7	1.9	82.2	1.4	248.8	-0.4
49	49.1	0.7	56.6	-1.8	223.6	-1.8
50	44	-1	66.2	-0.6	229	-1.5
51	48.8	0.6	82.2	1.4	247	-0.5
52	50.6	1.2	77.4	0.8	266.8	0.6
53	43.7	-1.1	78.2	0.9	290.2	1.9
54	47.6	0.2	75	0.5	281.2	1.4
55	45.5	-0.5	85.4	1.8	245.2	-0.6
56	44.3	-0.9	86.2	1.9	261.4	0.3
57	52.4	1.8	55.8	-1.9	221.8	-1.9
58	53.9	2.3	65.4	-0.7	277.6	1.2
59	45.2	-0.6	83.8	1.6	284.8	1.6

And now the Levey-Jennings charts for runs 3 through 59:

2017 break all the rules N=3 runs 30 through 59

Next, we're ready to see the table of data for runs 60 through 89:

Values	Control 1	z 1	Control 2	Z 2	Control 3	z 3	Rule Broken?
60	45.8	-0.4	74.2	0.4	277.6	1.2
61	50.6	1.2	64.6	-0.8	284.8	1.6
62	48	0.3	83	1.5	266.8	0.6
63	49	0.7	83	1.5	259.6	0.2
64	46	-0.3	67.1	-0.49	260.1	0.23
65	53	2.0	74.2	0.4	281.2	1.4
66	48	0.3	80.6	1.2	241.6	-0.8
67	45.8	-0.4	71.8	0.1	272.2	0.9
68	48.2	0.4	69.4	-0.2	288.4	1.8
69	50.9	1.3	81.4	1.3	250.6	-0.3
70	48.2	0.4	65.4	-0.7	277.6	1.2
71	51.2	1.4	70	-0.125	221.8	-1.9
72	49.1	0.7	69	-0.25	263.2	0.4
73	51.5	1.5	66	-0.625	247	-0.5
74	47.9	0.3	68.6	-0.3	254.2	-0.1
75	48.5	0.5	86.2	1.9	261.4	0.3
76	50.9	1.3	59.8	-1.4	266.8	0.6
77	46.7	-0.1	67.8	-0.4	252.4	-0.2
78	47.8	0.3	86.2	1.9	236.2	-1.1
79	45.5	-0.5	72.6	0.2	250.6	-0.3
80	52	1.7	81.4	1.3	259.6	0.2
81	52	1.7	74.2	0.4	243.4	-0.7
82	48.5	0.5	80.6	1.2	275.8	1.1
83	44.3	-0.9	79.8	1.1	281.2	1.4
84	49.4	0.8	75	0.5	252.4	-0.2
85	46.7	-0.1	76.6	0.7	272.2	0.9
86	52.7	1.9	72.6	0.2	281.2	1.4
87	45.2	-0.6	55.8	-1.9	275.8	1.1
88	48.8	0.6	79	1	256	0
89	46.4	-0.2	70.2	-0.1	236.2	-1.1

Now there are some Levey-Jennings charts for those runs 60 through 89:

Break All The Rules N3 runs 60 through 89

Now the data table for runs 90 through 119:

Values	Control 1	Z 1	Control 2	Z 2	Control 3	Z 3	Rules
90	49.7	0.9	82.2	1.4	275.8	1.1
91	47	0	68.6	-0.3	265	0.5
92	45.5	-0.5	60.6	-1.3	272.2	0.9
93	46.4	-0.2	79	1	283	1.5
94	51.5	1.5	59.8	-1.4	288.4	1.8
95	48.8	0.6	66.2	-0.6	259.6	0.2
96	44.9	-0.7	77.4	0.8	274	1
97	50.3	1.1	74.2	0.4	281.2	1.4
98	44.3	-0.9	65.4	-0.7	265	0.5
99	46.7	-0.1	63	-1	245.2	-0.6
100	50.6	1.2	74.2	0.4	268.6	0.7
101	49.4	0.8	85.4	1.8	279.4	1.3
102	47.6	0.2	80.6	1.2	288.4	1.8
103	46.7	-0.1	60.6	-1.3	234.4	-1.2
104	50	1	71	0	275.8	1.1
105	51.8	1.6	72.6	0.2	275.8	1.1
106	47.6	0.2	70.2	-0.1	250.6	-0.3
107	50	1	68.6	-0.3	261.4	0.3
108	49.1	0.7	81.4	1.3	230.8	-1.4
109	50.3	1.1	78.2	0.9	232.6	-1.3
110	48.8	0.6	69.4	-0.2	265	0.5
111	51.8	1.6	63	-1	256	0
112	50.6	1.2	82.2	1.4	252.4	-0.2
113	47.3	0.1	80.6	1.2	238	-1
114	48.8	0.6	76.6	0.7	230.8	-1.4
115	47.8	0.3	62.2	-1.1	239.8	-0.9
116	47	0	75	0.5	234.4	-1.2
117	47.6	0.2	81.4	1.3	275.8	1.1
118	46.7	-0.1	83.8	1.6	268.6	0.7
119	44.9	-0.7	72.6	0.2	236.2	-1.1

Break All The Rules N3 runs 90 through 120

Here's our final table of data, with values 120 through 150. We're just looking for the 12:x violation now:

Values	Control 1	Z 1	Control 2	Z 2	Control 3	Z 3	Rules Broken?
120	46.7	-0.1	62.2	-1.1	239.8	-0.9
121	47	0	75	0.5	234.4	-1.2
122	47.6	0.2	81.4	1.3	275.8	1.1
123	46.7	-0.1	83.8	1.6	268.6	0.7
124	44.9	-0.7	72.6	0.2	236.2	-1.1
125	50.9	1.3	79	1	263.2	0.4
126	52.7	1.9	76.6	0.7	230.8	-1.4
127	46.4	-0.2	80.6	1.2	232.6	-1.3
128	44	-1	75.8	0.6	270.4	0.8
129	51.2	1.4	86.2	1.9	268.6	0.7
130	48.5	0.5	80.6	1.2	252.4	-0.2
131	49.1	0.7	75.8	0.6	238	-1
132	51.8	1.6	75.8	0.6	248.8	-0.4
133	42.2	-1.6	58.2	-1.6	254.2	-0.1
134	43.4	-1.2	70.2	-0.1	265	0.5
135	50.3	1.1	69.4	-0.2	279.4	1.3
136	49.1	0.7	72.6	0.2	284.8	1.6
137	43.7	-1.1	70.2	-0.1	259.6	0.2
138	48.2	0.4	65.4	-0.7	274	1
139	42.8	-1.4	81.4	1.3	268.6	0.7
140	43.1	-1.3	78.2	0.9	277.6	1.2
141	48.8	0.6	69.4	-0.2	266.8	0.6
142	49.1	0.7	63	-1	290.2	1.9
143	46.4	-0.2	82.2	1.4	277.6	1.2
144	44	-1	75	0.5	266.8	0.6
145	45.8	-0.4	76.6	0.7	266.8	0.6
146	45.5	-0.5	59	-1.5	243.4	-0.7
147	51.2	1.4	81.4	1.3	266.8	0.6
148	48.5	0.5	78.2	0.9	277.6	1.2
149	49.1	0.7	73.4	0.3	266.8	0.6
150	50.3	1.1	75	0.5	270.4	0.8

Here are our last graphs of those runs:

2017 break all the rules N=3 runs 120 through 150

That should be enough data to keep you out of trouble. And the number of outliers you find in this dataset should be equivalent to a year's worth of outliers that you see normally in your own lab data.

Finally, I'm going to add the following conditions:

We are running THREE controls per run; assume each trio of values represents one run
We're NOT going to look for 2s violations. Let's break that habit right now. In any case, I didn't generate these data points randomly, so we should not expect to see a certain number of "natural false rejections" in the form of 2s outliers
We ARE going to try to find outliers of the following "Westgard Rules": 1:3s/2of3:2s/R:4s/3:1s/6:x (and later in the exercise, we switch to 9:x and12:x)
We're NOT going to sink into the swamp that is the repeating of controls. So don't assume anything you see is a repeated control - just stick with "in" or "out"

So, at the end of this, can you answer the following questions?

How many rule violations were there?
Which rules were violated and how many times?
What would be different if I told you that only the "Westgard Rules" 1:3s/2of3:2s/R:4s were necessary?
What would be different if I told you that only the 1:3s control rule was necessary?

These answers, and more, in a few weeks...

[Don't submit this exercise for grading. You'll get to compare your answers to the answer set in a few weeks. But feel free to share this with your colleagues and see if you can all reach the same conclusions and interpretations.]

Tools, Technologies and Training for Healthcare Laboratories

Basic QC Practices