Sigma Quality Performance Quality Goal Index Complete listing of applications, Sigma metrics, and QGI: Laboratory A Complete listing of applications, Sigma metrics and QGI: Laboratory B Summary of Laboratory Performance Problems Sigma Quality Improvement References The long-term goal of six sigma quality management is to achieve an error rate of 3.4 or less per million opportunities for all laboratory processes. In percent terms, that’s an error rate of less than 0.001%. This very demanding goal is achievable or within reach for many automated analytic processes, but implicitly requires knowledge of which processes fall short of this goal and why. Hence, a systematic approach to benchmarking sigma quality performance of analytic processes is fundamental to practicing six sigma quality management. In order to achieve six sigma quality, it is necessary to assess and document which analytical processes do and do not achieve six sigma quality performance and to understand why.
The long-term goal of six sigma quality management is to achieve an error rate of 3.4 or less per million opportunities for all laboratory processes. In percent terms, that’s an error rate of less than 0.001%. This very demanding goal is achievable or within reach for many automated analytic processes, but implicitly requires knowledge of which processes fall short of this goal and why. Hence, a systematic approach to benchmarking sigma quality performance of analytic processes is fundamental to practicing six sigma quality management. In order to achieve six sigma quality, it is necessary to assess and document which analytical processes do and do not achieve six sigma quality performance and to understand why.
As discussed elsewhere (1), quality performance of any analytical process in a laboratory can be described in terms of its sigma metric. Calculation of sigma performance is quite simple: Sigma = (TEa – Bias) / CV The harder part is making sure that data used in this calculation really reflects current analytical performance. In my region, we turned to our on-analyzer quality control databases for estimates of imprecision (CV). This data reflects actual analytical imprecision, if and only if, data integrity has been maintained. Data integrity can be assured only when procedures are in place and rigorously practiced to exclude erroneous quality control results due to procedural blunders and statistical outliers. This does not mean excluding out-of-control results; no statistically-valid data should be edited out. For estimates of test bias, our region makes use of Randox Interlaboratory Quality Assessment Scheme (RIAQS). This external quality assessment program, run out of Ireland, provides biweekly (every 2 weeks) challenges for a broad range of automated chemistry tests. There are two main attributes that make this program of particular value for assessing test bias. First, challenges are blind and second, they are at varying analyte concentrations. These two attributes, combined with its biweekly frequency and relatively large user base, make this database reflective, as close as possible, of “real-world” testing bias. Bias is calculated for each test application based on the average (running mean) bias of the last 10 values obtained over the preceding five month period, updated every two weeks. Sigma quality management forces one to think in terms of quality needs, not just performance capabilities and this is good since it focuses attention on what really matters. The next step is setting quality goals. This in itself can be a real eye-opener and a little daunting. There are multiple sources for quality goals but the proviso, of course, is objectivity in selecting these goals in deference to analytic capabilities. We selected quality goals for total error (TEa) based on either biological variation or proficiency testing target data accessible through this website (2, 3), fully recognizing that at least some of these goals may need customizing to specific clinical requirements as our experience with this approach develops. Selecting test specific goals, as uncomfortable or imperfect a process as this may be, it reinforces the fact that good clinical practice is implicitly linked to quality of laboratory testing. Once current and reliable data for imprecision and bias has been obtained and quality goals set, sigma quality performance is then calculated for each test application. These sigma metrics are date documented and used to benchmark existing sigma performance for inter-instrument and inter-laboratory comparisons within our region and for future historical reviews to assess performance changes. To illustrate this approach, the sigma metrics for 60 automated test applications over three analytic platforms at one laboratory (site A) in our region indicates that twenty eight (46%) of these applications fall short of meeting six sigma quality performance. Of these, twelve fail to meet minimum sigma quality performance with metrics less than three and another four just meet minimal acceptable performance with sigma metrics between three and four. At another laboratory (site B) in our regional group (seven sites in total), the original data collected indicates more than half (57%) of the automated test applications do not achieve six sigma quality performance.
As discussed elsewhere (1), quality performance of any analytical process in a laboratory can be described in terms of its sigma metric. Calculation of sigma performance is quite simple:
Sigma = (TEa – Bias) / CV
The harder part is making sure that data used in this calculation really reflects current analytical performance. In my region, we turned to our on-analyzer quality control databases for estimates of imprecision (CV). This data reflects actual analytical imprecision, if and only if, data integrity has been maintained. Data integrity can be assured only when procedures are in place and rigorously practiced to exclude erroneous quality control results due to procedural blunders and statistical outliers. This does not mean excluding out-of-control results; no statistically-valid data should be edited out.
For estimates of test bias, our region makes use of Randox Interlaboratory Quality Assessment Scheme (RIAQS). This external quality assessment program, run out of Ireland, provides biweekly (every 2 weeks) challenges for a broad range of automated chemistry tests. There are two main attributes that make this program of particular value for assessing test bias. First, challenges are blind and second, they are at varying analyte concentrations. These two attributes, combined with its biweekly frequency and relatively large user base, make this database reflective, as close as possible, of “real-world” testing bias. Bias is calculated for each test application based on the average (running mean) bias of the last 10 values obtained over the preceding five month period, updated every two weeks.
Sigma quality management forces one to think in terms of quality needs, not just performance capabilities and this is good since it focuses attention on what really matters. The next step is setting quality goals. This in itself can be a real eye-opener and a little daunting. There are multiple sources for quality goals but the proviso, of course, is objectivity in selecting these goals in deference to analytic capabilities. We selected quality goals for total error (TEa) based on either biological variation or proficiency testing target data accessible through this website (2, 3), fully recognizing that at least some of these goals may need customizing to specific clinical requirements as our experience with this approach develops. Selecting test specific goals, as uncomfortable or imperfect a process as this may be, it reinforces the fact that good clinical practice is implicitly linked to quality of laboratory testing.
Once current and reliable data for imprecision and bias has been obtained and quality goals set, sigma quality performance is then calculated for each test application. These sigma metrics are date documented and used to benchmark existing sigma performance for inter-instrument and inter-laboratory comparisons within our region and for future historical reviews to assess performance changes.
To illustrate this approach, the sigma metrics for 60 automated test applications over three analytic platforms at one laboratory (site A) in our region indicates that twenty eight (46%) of these applications fall short of meeting six sigma quality performance. Of these, twelve fail to meet minimum sigma quality performance with metrics less than three and another four just meet minimal acceptable performance with sigma metrics between three and four. At another laboratory (site B) in our regional group (seven sites in total), the original data collected indicates more than half (57%) of the automated test applications do not achieve six sigma quality performance.
It must be kept in mind that six sigma quality management is not only a tool for defining process performance but also a method for moving a process from its present error rate to a very low error rate. In order to achieve quality improvement of automated analytic tests where needed, it is important to understand the test-specific reasons for their quality shortcomings, be it either excessive imprecision, excessive bias or both. To facilitate this, performance data is assessed by calculating the quality goal index (QGI) by the following expression: QGI = Bias / 1.5 CV Derivation of this expression is based on mathematical reduction of the ratio (Bias/Accuracy Goal) / (CV/Precision Goal). The QGI ratio represents the relative extent to which both bias and precision meet their respective quality goals. The quality goals chosen for use in this expression are 1.5*TEa/6 for bias and TEa/6 for precision based on their widespread use in six sigma methodology literature. For those who feel that inclusion of a1.5 sigma shift in calculating the bias goal is not justified, then both quality goals can be calculated as TEa/6. The corresponding expression would then reduce to QGI = Bias/CV. Examination of the unreduced equation gives better insight into how this data-evaluating tool works. For example, QGI is higher for a test application when bias exceeds its accuracy goal and imprecision meets its precision goal and QGI is lower when bias meets its accuracy goal and imprecision exceeds its precision goal. The criteria we use for interpreting QGI when test applications fall short of six sigma quality is as follows: QGI Problem <0.8 Imprecision 0.8 - 1.2 Imprecision & Inaccuracy >1.2 Inaccuracy This quantifiable approach to problem assessment is computer programmable in Microsoft Excel using Visual Basic for Applications (VBA). Macro programming simplifies its application which is particularly helpful when dealing with larger numbers of tests at multiple lab sites. As an example of this approach, QGI indicates that of the 29 test applications that failed to meet six sigma quality performance at lab site A in our region, the main problem is excessive imprecision in 52%, with excessive inaccuracy occurring in 17%. At lab site B, the main problem is excessive inaccuracy in 73% with excessive imprecision being the problem in 20%. QGI provides easy insight into where improvement is required and can serve as a tool for focusing efforts on sigma quality improvement of automated analytic tests.
It must be kept in mind that six sigma quality management is not only a tool for defining process performance but also a method for moving a process from its present error rate to a very low error rate. In order to achieve quality improvement of automated analytic tests where needed, it is important to understand the test-specific reasons for their quality shortcomings, be it either excessive imprecision, excessive bias or both. To facilitate this, performance data is assessed by calculating the quality goal index (QGI) by the following expression:
QGI = Bias / 1.5 CV
Derivation of this expression is based on mathematical reduction of the ratio (Bias/Accuracy Goal) / (CV/Precision Goal). The QGI ratio represents the relative extent to which both bias and precision meet their respective quality goals. The quality goals chosen for use in this expression are 1.5*TEa/6 for bias and TEa/6 for precision based on their widespread use in six sigma methodology literature. For those who feel that inclusion of a1.5 sigma shift in calculating the bias goal is not justified, then both quality goals can be calculated as TEa/6. The corresponding expression would then reduce to QGI = Bias/CV.
Examination of the unreduced equation gives better insight into how this data-evaluating tool works. For example, QGI is higher for a test application when bias exceeds its accuracy goal and imprecision meets its precision goal and QGI is lower when bias meets its accuracy goal and imprecision exceeds its precision goal. The criteria we use for interpreting QGI when test applications fall short of six sigma quality is as follows:
This quantifiable approach to problem assessment is computer programmable in Microsoft Excel using Visual Basic for Applications (VBA). Macro programming simplifies its application which is particularly helpful when dealing with larger numbers of tests at multiple lab sites. As an example of this approach, QGI indicates that of the 29 test applications that failed to meet six sigma quality performance at lab site A in our region, the main problem is excessive imprecision in 52%, with excessive inaccuracy occurring in 17%. At lab site B, the main problem is excessive inaccuracy in 73% with excessive imprecision being the problem in 20%. QGI provides easy insight into where improvement is required and can serve as a tool for focusing efforts on sigma quality improvement of automated analytic tests.
APPLICATION INSTUMENT BIAS% CV% QG% SIGMA QGI PROBLEM Albumin PPE-P1 1.40 3.40 10 2.53 0.27 Imprecision Albumin MOD-2 4.28 1.04 10 5.50 2.74 Inaccuracy Alkaline Phosphatase PPE-P1 -14.47 6.50 30 2.39 1.48 Inaccuracy Alkaline Phosphatase MOD-2 -13.03 6.00 30 2.83 1.45 Inaccuracy ALT PPE-P1 9.17 2.20 32.1 10.42 2.78 None ALT MOD-2 8.59 2.00 32.1 11.76 2.86 None AST PPE-P1 3.89 2.30 15.2 4.92 1.13 Inaccuracy/Imprecision AST MOD-2 0.87 1.30 15.2 11.02 0.45 None Ammonia PPE-P1 0.00 3.86 20 5.18 0.00 Imprecision Ammonia MOD-2 0.00 6.20 20 3.23 0.00 Imprecision Total CO2 PPE-P1 0.22 1.62 10 6.04 0.09 None Total CO2 MOD-2 4.62 2.90 10 1.86 1.06 Inaccuracy/Imprecision Bilirubin, Direct PPE-P1 3.99 4.90 48.5 9.08 0.54 None Bilirubin, Direct MOD-2 -7.65 2.00 48.5 20.43 2.55 None Bilirubin, Total PPE-P2 1.24 1.89 9.5 4.37 0.44 Imprecision Bilirubin, Total MOD-2 -3.84 2.20 9.5 2.57 1.16 Inaccuracy/Imprecision Calcium PPE-P1 -3.03 1.77 7.6 2.58 1.14 Inaccuracy/Imprecision Calcium MOD-2 -2.13 1.51 7.6 3.62 0.94 Inaccuracy/Imprecision Chloride PPE-P1 -0.43 1.10 5 4.15 0.26 Imprecision Chloride MOD-2 -0.63 1.80 5 2.43 0.23 Imprecision Cholesterol PPE-P1 -2.62 1.35 9 4.73 1.29 Inaccuracy Cholesterol MOD-2 1.99 1.29 9 5.43 1.03 Inaccuracy/ Imprecision Creatine Kinase PPE-P1 -1.25 1.00 30 28.75 0.83 None Creatine Kinase MOD-2 -1.01 1.50 30 19.33 0.45 None Crearinine PPE-P1 1.58 2.00 15 6.71 0.53 None Creatinine MOD-2 -0.06 1.40 15 10.67 0.03 None GGT PPE-P1 -2.21 2.20 25 10.36 0.67 None GGT MOD-2 -3.29 2.60 25 8.35 0.84 None Glucose PPE-P1 -0.20 1.09 6.3 5.60 0.12 Imprecision Glucose MOD-2 -1.47 0.80 6.3 6.04 1.23 None HDL PPE-P2 4.97 3.20 30 7.82 1.04 None HDL MOD-2 9.90 2.70 30 7.44 2.44 None Hydroxybutrate PPE-P1 -3.41 6.87 25 3.14 0.33 Imprecision Iron PPE-P1 6.08 2.20 30.7 11.19 1.84 None Iron MOD-2 5.05 1.40 30.7 18.32 2.40 None LDH PPE-P2 1.44 1.30 11.4 7.66 0.74 None LDH MOD-2 -0.27 1.30 11.4 8.56 0.14 None Lipase PPE-P2 -3.06 3.50 29.1 7.44 0.58 None Lipase MOD-2 -6.83 4.10 29.1 5.43 1.11 Inaccuracy/Imprecision Lactate PPE-P2 0.00 1.68 30.4 18.10 0.00 None Lactate MOD-2 0.00 0.86 30.4 35.35 0.00 None Magnesium PPE-P2 2.31 2.20 25 10.31 0.70 None Magnesium MOD-2 -0.98 3.90 25 6.16 0.17 None Phosphate PPE-P2 -0.22 1.86 4.3 2.19 0.08 Imprecision Phosphate MOD-2 1.38 1.70 4.3 1.72 0.54 Imprecision Potassium PPE-P1 -0.08 1.15 5.8 4.97 0.05 Imprecision Potassium PPE-P2 -0.08 0.72 5.8 7.94 0.07 None Potassium MOD-2 -1.50 1.12 5.8 3.84 0.89 Inaccuracy/Imprecision Protein PPE-P2 -0.09 1.80 10 5.51 0.03 Imprecision Protein MOD-2 -1.09 1.05 10 8.49 0.69 None Sodium PPE-P1 1.09 1.01 2.4 1.30 0.72 Imprecision Sodium PPE-P2 1.09 0.89 2.4 1.47 0.82 Inaccuracy/Imprecision Sodium MOD-2 -0.53 0.87 2.4 2.15 0.41 Imprecision TIBC PPE-P2 -2.57 1.80 30 15.24 0.95 None Triglycerides PPE-P2 10.00 1.59 27.9 11.26 4.19 None Triglycerides MOD-2 -1.63 3.09 27.9 8.50 0.35 None Urea PPE-P2 1.09 0.74 15.7 19.74 0.98 None Urea MOD-2 0.04 1.01 15.7 15.50 0.03 None Uric Acid PPE-P2 6.57 0.70 11.9 7.61 6.26 None Uric Acid MOD-2 5.76 1.10 11.9 5.58 3.49 Inaccuracy
APPLICATION
INSTUMENT
BIAS%
CV%
QG%
SIGMA
QGI
PROBLEM
Albumin
PPE-P1
1.40
3.40
10
2.53
0.27
Imprecision
MOD-2
4.28
1.04
5.50
2.74
Inaccuracy
Alkaline Phosphatase
-14.47
6.50
30
2.39
1.48
-13.03
6.00
2.83
1.45
ALT
9.17
2.20
32.1
10.42
2.78
None
8.59
2.00
11.76
2.86
AST
3.89
2.30
15.2
4.92
1.13
Inaccuracy/Imprecision
0.87
1.30
11.02
0.45
Ammonia
0.00
3.86
20
5.18
6.20
3.23
Total CO2
0.22
1.62
6.04
0.09
4.62
2.90
1.86
1.06
Bilirubin, Direct
3.99
4.90
48.5
9.08
0.54
-7.65
20.43
2.55
Bilirubin, Total
PPE-P2
1.24
1.89
9.5
4.37
0.44
-3.84
2.57
1.16
Calcium
-3.03
1.77
7.6
2.58
1.14
-2.13
1.51
3.62
0.94
Chloride
-0.43
1.10
5
4.15
0.26
-0.63
1.80
2.43
0.23
Cholesterol
-2.62
1.35
9
4.73
1.29
1.99
5.43
1.03
Inaccuracy/ Imprecision
Creatine Kinase
-1.25
1.00
28.75
0.83
-1.01
1.50
19.33
Crearinine
1.58
15
6.71
0.53
Creatinine
-0.06
10.67
0.03
GGT
-2.21
25
10.36
0.67
-3.29
2.60
8.35
0.84
Glucose
-0.20
1.09
6.3
5.60
0.12
-1.47
0.80
1.23
HDL
4.97
3.20
7.82
9.90
2.70
7.44
2.44
Hydroxybutrate
-3.41
6.87
3.14
0.33
Iron
6.08
30.7
11.19
1.84
5.05
18.32
2.40
LDH
1.44
11.4
7.66
0.74
-0.27
8.56
0.14
Lipase
-3.06
3.50
29.1
0.58
-6.83
4.10
1.11
Lactate
1.68
30.4
18.10
0.86
35.35
Magnesium
2.31
10.31
0.70
-0.98
3.90
6.16
0.17
Phosphate
-0.22
4.3
2.19
0.08
1.38
1.70
1.72
Potassium
-0.08
1.15
5.8
0.05
0.72
7.94
0.07
-1.50
1.12
3.84
0.89
Protein
-0.09
5.51
-1.09
1.05
8.49
0.69
Sodium
1.01
2.4
1.47
0.82
-0.53
2.15
0.41
TIBC
-2.57
15.24
0.95
Triglycerides
10.00
1.59
27.9
11.26
4.19
-1.63
3.09
8.50
0.35
Urea
15.7
19.74
0.98
0.04
15.50
Uric Acid
6.57
11.9
7.61
6.26
5.76
5.58
3.49
APPLICATION INSTUMENT BIAS% CV% QG% SIGMA QGI PROBLEM Albumin PE-1 4.47 2.51 10.00 2.20 7.48 Inaccuracy Albumin PE-2 3.05 2.51 10.00 2.77 5.10 Inaccuracy Alk Phos PE-1 -13.84 2.25 30.00 7.18 20.76 None Alk Phos PE-2 -12.73 2.25 30.00 7.68 19.10 None ALT PE-1 1.37 3.05 32.10 10.08 2.79 None ALT PE-2 0.20 3.05 32.10 10.46 0.41 None AST PE-1 1.73 1.86 15.20 7.24 2.15 None AST PE-2 0.81 1.86 15.20 7.74 1.00 None Bicarbonate PE-1 -1.82 6.58 10.00 1.24 7.98 Inaccuracy Bicarbonate PE-2 -3.84 6.58 10.00 0.94 16.84 Inaccuracy Bilirubin, Direct PE-1 -20.63 4.65 44.50 5.13 63.95 Inaccuracy Bilirubin, Direct PE-2 -26.49 4.65 44.50 3.87 82.12 Inaccuracy Bilirubin, Total PE-1 0.80 3.22 9.50 2.70 1.72 Inaccuracy Bilirubin, Total PE-2 -1.37 3.22 9.50 2.52 2.94 Inaccuracy Calcium, Total PE-1 -3.18 2.42 7.60 1.83 5.13 Inaccuracy Calcium, Total PE-2 -2.23 2.42 7.60 2.22 3.60 Inaccuracy Chloride PE-1 -1.40 1.65 5.00 2.18 1.54 Inaccuracy Chloride PE-2 -0.89 1.65 5.00 2.49 0.98 Inaccuracy/Imprecision Cholesterol PE-1 1.42 1.82 9.00 4.16 1.72 Inaccuracy Cholesterol PE-2 -0.66 1.82 9.00 4.58 0.80 Inaccuracy/Imprecision CK, Total PE-1 -4.17 1.93 30.00 13.38 5.37 None CK, Total PE-2 -3.77 1.93 30.00 13.59 4.85 None Creatinine PE-1 -3.25 2.98 15.00 3.94 6.46 Inaccuracy Creatinine PE-2 -1.96 2.98 15.00 4.38 3.89 Inaccuracy GGT PE-1 -3.25 2.27 25.00 9.58 4.92 None GGT PE-2 -2.48 2.27 25.00 9.92 3.75 None Glucose PE-1 1.26 2.10 6.30 2.40 1.76 Inaccuracy Glucose PE-2 -0.22 2.10 6.30 2.90 0.31 Imprecision HDL-C PE-1 1.69 2.15 30.00 13.17 2.42 None HDL-C PE-2 3.45 2.15 30.00 12.35 4.95 None Hydroxybutrate PE-1 2.34 3.85 25.00 5.89 6.01 Inaccuracy Hydroxybutyrate PE-2 2.04 3.85 25.00 5.96 5.24 Inaccuracy Iron PE-1 0.45 2.58 30.70 11.72 0.77 None Iron PE-2 3.98 2.58 30.70 10.36 6.85 None LDH PE-1 -0.72 1.28 11.40 8.34 0.61 None LDH PE-2 0.28 1.28 11.40 8.69 0.24 None Lipase PE-1 0.54 4.58 29.10 6.24 1.65 None Lipase PE-2 -0.31 4.58 29.10 6.29 0.95 None Magnesium PE-1 0.62 2.86 25.00 8.52 1.18 None Magnesium PE-2 1.29 2.86 25.00 8.29 2.46 None Phosphate PE-1 -0.59 3.23 4.30 1.15 1.27 Inaccuracy Phosphate PE-2 1.06 3.23 4.30 1.00 2.28 Inaccuracy Potassium PE-1 -1.08 13.7 5.80 0.34 9.86 Inaccuracy Potassium PE-2 -0.71 13.7 5.80 0.37 6.48 Inaccuracy Protein, Total PE-1 -0.59 1.52 10.00 6.19 0.60 None Protein, Total PE-2 -0.14 1.52 10.00 6.49 0.14 None Sodium PE-1 -0.63 1.79 2.40 0.99 0.75 Imprecision Sodium PE-2 -0.17 1.79 2.40 1.25 0.20 Imprecision UIBC/TIBC PE-1 -10.00 8.70 30.00 2.30 58.00 Inaccuracy UIBC/TIBC PE-2 -3.24 8.70 30.00 3.08 18.79 Inaccuracy Triglycerides PE-1 4.21 2.27 27.90 10.44 6.37 None Triglycerides PE-2 -2.35 2.27 27.90 11.26 3.56 None Urea PE-1 0.00 2.80 15.70 5.61 0.00 Imprecision Urea PE-2 -0.91 2.80 15.70 5.28 1.70 Inaccuracy Uric Acid PE-1 5.93 1.62 11.90 3.69 6.40 Inaccuracy Uric Acid PE-2 5.31 1.62 11.90 4.07 5.73 Inaccuracy
PE-1
4.47
2.51
7.48
PE-2
3.05
2.77
5.10
Alk Phos
-13.84
2.25
30.00
7.18
20.76
-12.73
7.68
19.10
1.37
32.10
10.08
2.79
0.20
10.46
1.73
15.20
7.24
0.81
7.74
Bicarbonate
-1.82
6.58
7.98
16.84
-20.63
4.65
44.50
5.13
63.95
-26.49
3.87
82.12
3.22
9.50
-1.37
2.52
2.94
Calcium, Total
-3.18
2.42
7.60
1.83
-2.23
2.22
3.60
-1.40
1.65
5.00
2.18
1.54
-0.89
2.49
1.42
1.82
9.00
4.16
-0.66
4.58
CK, Total
-4.17
1.93
13.38
5.37
-3.77
13.59
4.85
-3.25
2.98
15.00
3.94
6.46
-1.96
4.38
2.27
25.00
9.58
-2.48
9.92
3.75
1.26
2.10
6.30
1.76
0.31
HDL-C
1.69
13.17
3.45
12.35
4.95
2.34
3.85
5.89
6.01
Hydroxybutyrate
2.04
5.96
5.24
30.70
11.72
0.77
3.98
6.85
-0.72
1.28
11.40
8.34
0.61
0.28
8.69
0.24
29.10
6.24
-0.31
6.29
0.62
8.52
1.18
8.29
2.46
-0.59
4.30
1.27
2.28
-1.08
13.7
5.80
0.34
9.86
-0.71
0.37
6.48
Protein, Total
1.52
6.19
0.60
-0.14
6.49
1.79
0.99
0.75
-0.17
1.25
UIBC/TIBC
-10.00
8.70
58.00
-3.24
3.08
18.79
4.21
27.90
10.44
6.37
-2.35
3.56
2.80
15.70
5.61
-0.91
5.28
5.93
11.90
3.69
6.40
5.31
4.07
5.73
SITE NUMBER OF PERFORMANCE PROBLEMS (PERCENT) NONE IMPRECISION INACCURRACY BOTH Lab A 32 (54%) 14 (23%) 5 (8%) 9 (15%) Lab B 24 (43%) 4 (7%) 26 (46%) 2 (4%)
SITE
NUMBER OF PERFORMANCE PROBLEMS (PERCENT)
NONE
IMPRECISION
INACCURRACY
BOTH
Lab A
32 (54%)
14 (23%)
5 (8%)
9 (15%)
Lab B
24 (43%)
4 (7%)
26 (46%)
2 (4%)
One cause of difficulty in reducing inaccuracy of automated tests is the variable error introduced when switching from one lot of calibrator to another. The manufacturer of the multi-test calibrator we use claims a variability limit of +/-5%. A tolerance of 10% is not adequate for achieving long-term accuracy for six sigma quality performance of automated analytic tests. Achieving and maintaining six sigma quality performance requires a rigorous approach to calibrator lot changes, one linked to a bias reference point or “anchor”. To accomplish this, the current biases of our “anchor tests” are incorporated into the decision making process during new calibrator lot evaluation. This concept is easiest to understand when described by the procedural steps involved as follows: TEST EXAMPLES Creatinine Urea 1 Obtain assigned value for new calibrator 331.0 15.1 2 Assay new calibrator over a one month period to establish assayed mean value 322.34 15.28 3 Subtract assayed mean value from calibrator assigned value to determine % difference from assigned value 2.62 -1.19 4 Obtain % bias from RIQAS 0.41 1.03 5 Calculate % combined error (% difference + % bias) 3.03 -0.17 6 Calculate accuracy goal (1.5TEa/6) 3.8 3.9 7 Determine ratio % bias / accuracy goal (B/AG) 0.11 0.26 8 Determine ratio % combined error / accuracy goal (CE/AG) 0.89 -0.04 9 Select either assigned value or assayed mean value for new calibrator setpoint depending on which ratio is numerically lower. If B/AG < CE/AG, use assayed mean value. If CE/AG < B/AG, use new calibrator assigned value 322.34 15.1 In the accompanying test examples, the assayed mean value is selected as calibrator setpoint for creatinine, whereas the assigned value is selected for urea. Each of these choices gives best calibrator cross-over accuracy on the RIQAS quality assessment program. Using this approach for twenty-three “anchor tests” at laboratory site A, it was determined that best cross-over accuracy was achieved by using the assigned values for new calibrator setpoints for 12 of these tests (Table 3). Assayed means provided better cross-over accuracy for the remaining 11 tests. In addition, the number of tests exceeding their six sigma accuracy goals decreased from five with the old lot to three with the new lot of calibrator, which is expected to improve further with successive lot changes. APPLICATION ASSIGNED VALUE ASSAYED MEAN %DIFFERENCE BIAS% CE% AG% B/AG CE/AG NEW SETPOINT Albumin 28.00 28.44 -1.57 1.77 0.20 2.5 0.71 0.08 28.0 Alkaline Phosphatase 227.00 217.50 4.19 -14.60 -10.41 7.5 -1.95 -1.39 227.0 ALT 113.00 122.39 -8.31 9.88 1.57 8.0 1.23 0.20 113.0 AST 111.00 115.50 -4.05 2.78 -1.27 3.8 0.73 -0.33 111.0 Bilirubin, Direct 40.50 37.56 7.26 3.03 10.29 11.1 0.27 0.92 37.56 Bilirubin, Total 84.00 73.11 12.96 -0.13 12.84 2.4 -0.05 5.41 73.11 Calcium 2.220 2.140 3.60 -3.12 0.48 1.9 -1.64 0.25 2.22 Choloesterol 3.990 3.960 0.75 -1.85 -1.09 2.3 -0.82 -0.49 3.99 Creatine Kinase 350.00 334.87 4.32 -1.44 2.88 7.6 -0.19 0.38 334.87 Creatinine 331.00 322.34 2.62 0.41 3.03 3.8 0.11 0.81 322.34 Iron 38.50 39.09 -1.53 5.51 3.97 7.7 0.72 0.52 38.5 GGT 93.60 91.94 1.77 -1.97 -0.20 6.3 -0.32 -0.03 93.6 Glucose 11.30 11.56 -2.30 -0.18 -2.48 1.6 -0.11 -1.57 11.56 Lactate 2.99 3.04 -1.67 0.00 -1.67 7.6 0.00 -0.22 3.04 LDH 237.00 239.56 -1.08 1.50 0.42 2.9 0.53 0.15 237.0 Lipase 72.50 74.50 -2.76 -3.46 -6.22 7.3 -0.48 -0.86 74.5 Magnesium 1.050 1.110 -5.71 2.32 -3.39 6.3 0.37 -0.54 1.11 Phosphorus 1.490 1.460 2.01 -0.51 1.50 1.1 -0.47 1.40 1.46 Salicylate 145.00 140.40 3.17 0.00 3.17 6.3 0.00 0.51 140.4 Triglycerides 1.540 1.550 -0.65 8.96 8.31 7.0 1.28 1.19 1.54 Total Protein 50.60 49.67 1.84 -0.76 1.08 2.5 -0.30 0.43 49.67 Uric Acid 290.00 293.99 -1.38 4.67 3.29 3.0 1.57 1.11 290.0 Urea 15.10 15.28 -1.19 1.03 -0.17 3.9 0.26 -0.04 15.1 Application of the above concepts based on six sigma principles is a “project-in-progress” in Winnipeg, Manitoba, Canada. It has already served to bring attention and interest in six sigma methodology in our region. Expectantly, it is a system to objectively assess how our automated analytic tests are doing, to determine where improvement is still needed and to measure where progress has been made. As such, it is fundamental to achieving six sigma quality performance.
One cause of difficulty in reducing inaccuracy of automated tests is the variable error introduced when switching from one lot of calibrator to another. The manufacturer of the multi-test calibrator we use claims a variability limit of +/-5%. A tolerance of 10% is not adequate for achieving long-term accuracy for six sigma quality performance of automated analytic tests.
Achieving and maintaining six sigma quality performance requires a rigorous approach to calibrator lot changes, one linked to a bias reference point or “anchor”. To accomplish this, the current biases of our “anchor tests” are incorporated into the decision making process during new calibrator lot evaluation. This concept is easiest to understand when described by the procedural steps involved as follows:
In the accompanying test examples, the assayed mean value is selected as calibrator setpoint for creatinine, whereas the assigned value is selected for urea. Each of these choices gives best calibrator cross-over accuracy on the RIQAS quality assessment program. Using this approach for twenty-three “anchor tests” at laboratory site A, it was determined that best cross-over accuracy was achieved by using the assigned values for new calibrator setpoints for 12 of these tests (Table 3). Assayed means provided better cross-over accuracy for the remaining 11 tests. In addition, the number of tests exceeding their six sigma accuracy goals decreased from five with the old lot to three with the new lot of calibrator, which is expected to improve further with successive lot changes.
ASSIGNED VALUE
ASSAYED MEAN
%DIFFERENCE
CE%
AG%
B/AG
CE/AG
NEW SETPOINT
28.00
28.44
-1.57
2.5
0.71
28.0
227.00
217.50
-14.60
-10.41
7.5
-1.95
-1.39
227.0
113.00
122.39
-8.31
9.88
1.57
8.0
113.0
111.00
115.50
-4.05
-1.27
3.8
0.73
-0.33
111.0
40.50
37.56
7.26
3.03
10.29
11.1
0.92
84.00
73.11
12.96
-0.13
12.84
-0.05
5.41
2.220
2.140
-3.12
0.48
1.9
-1.64
0.25
Choloesterol
3.990
3.960
-1.85
2.3
-0.82
-0.49
350.00
334.87
4.32
-1.44
2.88
-0.19
0.38
331.00
322.34
2.62
0.11
38.50
39.09
-1.53
3.97
7.7
0.52
38.5
93.60
91.94
-1.97
-0.32
-0.03
93.6
11.30
11.56
-2.30
-0.18
1.6
-0.11
2.99
3.04
-1.67
237.00
239.56
0.42
2.9
0.15
237.0
72.50
74.50
-2.76
-3.46
-6.22
7.3
-0.48
-0.86
74.5
1.050
1.110
-5.71
2.32
-3.39
-0.54
Phosphorus
1.490
1.460
2.01
-0.51
1.1
-0.47
1.46
Salicylate
145.00
140.40
3.17
0.51
140.4
1.540
1.550
-0.65
8.96
8.31
7.0
1.19
Total Protein
50.60
49.67
-0.76
1.08
-0.30
0.43
290.00
293.99
-1.38
4.67
3.29
3.0
290.0
15.10
15.28
-1.19
3.9
-0.04
15.1
Application of the above concepts based on six sigma principles is a “project-in-progress” in Winnipeg, Manitoba, Canada. It has already served to bring attention and interest in six sigma methodology in our region. Expectantly, it is a system to objectively assess how our automated analytic tests are doing, to determine where improvement is still needed and to measure where progress has been made. As such, it is fundamental to achieving six sigma quality performance.
1. www.westgard.com/essay36.htm 2. www.westgard.com/biodatabase1.htm 3. Westgard JO. Six Sigma Quality Design & Control, Westgard QC, Inc., 2001, Appendix 2, page 276
1. www.westgard.com/essay36.htm
2. www.westgard.com/biodatabase1.htm
3. Westgard JO. Six Sigma Quality Design & Control, Westgard QC, Inc., 2001, Appendix 2, page 276
Understanding Quality -- Jerry Ehrmeyer Approaches to Clinical Laboratory Utilization -- Art Eggert, Ph.D. Tips on managing the quality of immunoassays -- R. Neill Carey, Ph.D. Defect rates, quality and productivity -- Robert Burnett, Ph.D. What's New with CLIA'88, JCAHO & CAP -- Sharon Ehrmeyer, Ph.D. European Approaches to Analytical Goal-Setting -- Per Hyltoft Petersen QC - The Regulations -- Sharon Ehrmeyer, Ph.D. Total Quality Management for Medical Laboratories: A European point of view -- Dr. Pharm. J.C. Libeer, Ph.D., Ph.D. MV - The Regulations -- Sharon Ehrmeyer, Ph.D. Biological variation data for setting quality specifications -- Callum G. Fraser, Ph.D. QC Validation in Veterinary Laboratories -- Kathleen P. Freeman, DVM, MS, Ph.D. Report of the Norwegian EQA Validator Workshop -- Dr. Heidi Steensland Trueness and Uncertainty -- Dr Xavier Fuentes Arderiu Ph.D. Pharm.D. Good Laboratory Practice versus CLIA -- Janine Denis Cook, Ph.D. Biologic Variation and desirable specifications for quality control --Dr. Carmen Ricos CLS: Victims of our own Success?-- Diana Mass M.A., CLS(NCA) Biologic Variation: Principles & Practice --Callum Fraser Ph.D. Pre-,Post- & Analytical Errors -- David Plaut Biological Database Update -- Dr. Carmen Ricos et al Quality Requirements Update 2002 -- Sharon Ehrmeyer, Ph.D. Are "Scientific Statements" the Scientific Truth? by Callum G. Fraser, Ph.D.Biologic Variation and desirable specifications for quality control 2006 --Dr. Carmen Ricos A multisite validation that selected "Westgard Rules" is efficient and cost-effective -- David Plaut et al.