Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Essays
  4. Quality of Laboratory Testing
  5. Part IV: The Quality of Glucose Testing

Quality of Laboratory Testing

Part IV: The Quality of Glucose Testing

January 2005
with Sten Westgard, MS
The headline in the November 2004 issue of Clinical Laboratory News [1] reads “More Studies Support Tight Glycemic Control in Hospitals.” The article cites studies that recommend the use of a continuous insulin drip to keep a patient’s blood glucose in the range from 70 to 110 mg/dL for patients in the ICU and 80 to 120 mg/dL for other hospitalized patients (recommendations known as the Portland Protocol). A recent recommendation from Mayo Clinic sets the goal at less than 140 mg/dL. Such aggressive efforts at glycemic control require frequent blood glucose measurements, usually at least every 4 hours and sometimes as often as every 30 minutes. This requires rapid testing service from a central laboratory or Point-of-Care testing on the nursing units. It also requires very accurate testing to properly manage the insulin drip.

This aggressive treatment follows the trend of more aggressive diagnosis of diabetes. In 2002, the American Diabetes Association and the US Health and Human Services (ADA/HHS) recommended that fasting glucose be125 mg/dL or less and that a value of 126 mg/dL, if confirmed, indicates hyperglycemia [2]. More recently, there have been suggestions that values between 110 and 125 mg/dL be interpreted as pre-diabetic and that a lower cutoff for impaired fasting glucose be set at 100 mg/dL, not 110 mg/dL [3].

Obviously the quality of glucose testing today is critically important, which is why we consider glucose to be a touchstone test and a priority in our assessment of the quality of laboratory testing today [4]. We make this assessment using data from proficiency testing surveys that are required by CLIA and reported to CMS. Our assessment methodology is illustrated for another test, cholesterol, in the previous report in this series [5].

Materials and Methods

Sigma metrics are estimated for glucose testing based on proficiency testing data collected during 2004.

  • National requirement for the quality of glucose is defined as an allowable total error (TEa) of 6 mg/dL or 10.0%, whichever is larger. This is the CLIA criterion for acceptable performance in proficiency testing (PT).
  • Survey specimens were selected near the medically important decision concentration of 125 mg/dL.
  • PT data comes from 2004 surveys performed by the American Academy of Family Physicians (AAFP), Medical Laboratory Evaluation (MLE), American Association of Bioanalysts (AAB), American Proficiency Institute (API), College of American Pathologists (CAP), and New York State (NY).
  • National Test Quality (NTQ) observed for a single proficiency testing sample is estimated from the CLIA total allowable total error (TEa) divided by the group SD or CV, i.e., Sigma = TEa/CV. The average NTQ observed for multiple surveys is weighted for the number of laboratories participating in the survey.
  • Local Method Quality (LMQ) for a single proficiency testing sample is a weighted average of the Sigmas determined for each method subgroup without accounting for method bias, i.e., Sigma = TEa/CVmethsubgroup. The average LMQ observed for multiple surveys is weighted for the number of laboratories participating in each survey.
  • National Method Quality (NMQ) observed for a single proficiency testing sample is a weighted average of the Sigmas determined for each method subgroup taking bias into account, i.e., Sigma = (TEa – biasmethsubgroup)/CVmethsubgroup. The average NMQ observed for multiple surveys is weighted for the number of laboratories patricipating in each survey.
Further details on the methodology are discussed in an earlier essay.

Estimates of quality. Table 1 provides the estimates of Sigma performance from 5 different PT programs, as identified in Column 1. Column 2 shows the number of laboratories participating in each program and a total of over 10,722 laboratories for all 5 programs. AAFP is the smallest with 245 labs, followed by MLE with 628, AAB with 1665, API with 3038, and CAP with 5146 participants.

Table 1. Summary of glucose quality (weighted averages) from 5 national PT survey programs
Sigma Quality Performance Metrics
PT Program Labs Group Mean Nat. Test Loc. Method Nat. Method Datasheet







AAFP 2004A 245 134.0 1.91 3.16 2.64 1
MLE M3 628 106.1 1.75 2.99 2.13 2
AAB 2nd 2004 1665 106.4 2.22 3.20 2.60 3
API 2nd 2004 3038 106.6 2.42 3.24 2.70 4
CAP LP-01 2004 5146 149.6 3.70 4.88 4.14 5







Group summary 10722 120.5 2.95 4.00 3.34







NY Benchmark 412 125 3.55 4.94 4.26 6


Column 3 shows the group means for each of the survey samples from the different PT programs. We selected the sample closest to a concentration of 125 mg/dL, which is critical for diagnostic and treatment practices today. We observed that most survey samples tend to have a high concentration and often only 1 out of the 5 samples in a PT Event actually test the critical performance range.

Column 4 shows the estimates of National Test Quality for each survey group. The weighted average of the estimates is only 2.95 Sigma, which indicates, on average, glucose test quality is at the minimum level that is acceptable for routine production processes.

Column 5 gives a more optimistic estimate of Local Method Quality based on method subgroups without inclusion of bias between methods. Even so, the numbers range from 2.99 to 3.24 for half the labs, with the CAP figure of 4.88 boosting the weighted average to 4.00. Remember that this estimate of quality assumes that local reference ranges and medical cutoff points would be used to compensate for the bias between methods. If bias is not compensated in this way, then column 6 provides a better estimate of quality with respect to standardized national treatment guidelines. Those figures are lower, ranging from 2.13 to 2.70 for the labs in the smaller survey programs. Again, the larger labs in the CAP survey show better performance at 4.14 Sigma and brings the overall weighted average to 3.34 Sigma.

As a benchmark for comparison, we again include survey data from the NY program, which we believe sets the most demanding regulatory standards. These estimates are better, from 3.55 to 4.26 for NTQ and NMQ and 4.94 Sigma for LMQ.


Variability of estimates. Table 2 provides a more complete assessment for CAP testing results for 2 Events including 10 samples. The overall average sigmas are 4.05 for NTQ, 4.92 for LMQ, and 4.40 for NMQ. Note, however, that average concentration over the 10 specimens is over 200 mg/dL and that the estimates of performance are best at high concentrations. The two lowest samples, C-03 at 98.8 mg/dL and C-07 at 74.1 mg/dL, show considerably lower Sigmas - 2.70 and 2.63 for NTQ, 3.12 and 3.00 for NMQ. Sample C-06, which is probably representative of the most important clinical applications, shows Sigmas of 3.70 for NTQ and 4.14 for NMQ. Thus it is likely that method bias exerts some influence on the national quality of glucose tests. The estimates of LMQ show the most uniformity over the 10 samples, which is also explainable by the fact that method bias is not considered in these estimates.

Table 2. Summary of glucose quality (weighted averages) from 10 CAP 2004 Specimens



Sigma Quality Performance Metrics
PT Specimen Number Labs Group Mean Nat. Test Loc. Method Nat. Method
C-01 5035 303.9 4.35 5.22 4.82
C-02 5028 279.1 4.55 5.23 4.87
C-03 5051 98.8 2.70 4.23 3.21
C-04 5031 228.4 4.55 5.18 4.88
C-05 5028 176.4 4.17 4.82 4.38
C-06 5146 149.6 3.70 4.88 4.14
C-07 5151 74.1 2.63 3.95 3.00
C-08 5152 202.4 4.55 5.19 4.86
C-09 5140 228.0 4.76 5.27 5.01
C-10 5133 303.9 4.55 5.21 4.87






Avg Evt1 5,034.60 217.32 4.06 4.93 4.43
SD 9.61 82.40 0.78 0.43 0.71






Avg Evt2 5,144.40 191.60 4.04 4.90 4.38
SD 7.96 86.06 0.88 0.55 0.84






AVG ALL 5,089.50 204.46 4.05 4.92 4.40
SD 58.46 80.58 0.78 0.47 0.73
MEDIAN 5,092.00 215.20 4.45 5.19 4.84
MINIMUM 5028 74.10 2.63 3.95 3.00
MAXIMUM 5152 303.90 4.76 5.27 5.01

Discussion

As observed earlier for cholesterol, the quality of glucose tests appears to depend on the size of laboratory and sophistication of the methods and analysts involved. The laboratories participating in the AAFP, MLE, AAB, and API surveys show Sigmas of 1.75 to 2.42 for NQT and 2.12 to 2.70 for NMQ. Even the most optimistic estimate, LMQ, shows only 2.99 to 3.24 Sigma quality, which is borderline for a routine production process in any other industry. The laboratories participating in the CAP survey clearly achieve a higher level of quality, from 3.70 to 4.14 Sigma for NTQ and NMQ and 4.88 for LMQ. The overall weighted performance suggests 2.95 to 3.34 Sigma for NTQ and NMQ, which is far from 6-Sigma world class quality.

As shown in the figure below, these estimates of Test and Method Quality challenge the adequacy of current CLIA regulations for the minimum QC of 2 controls per day. Methods with 4 Sigma performance need to be controlled with 4 controls per run. Methods with 3 Sigma performance, or less cannot be controlled to achieve the national quality requirement specified by the CLIA PT criterion for glucose.

Conclusion

Strike two!

References

  1. Downer K. More studies support right glycemic contron in hospitals: What is the ideal target level and protocol? Clin Lab News 2004;30 (No. 11), pp 1-3.
  2. Sainato D. A new attack on the diabetes epidemic. Clin Lab News 2002;28(June):1-5.
  3. Lowering the threshold for impaired fasting glucose. Clin Lab Strategies 2004;9(Jan), p3.
  4. Westgard JO, Westgard S. The Quality of Laboratory Testing. Part II. Touchstone Test Methodology.
  5. Westgard JO, Westgard S. The Quality of Laboratory Testing. Part III. Cholesterol.

James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.