Tools, Technologies and Training for Healthcare Laboratories

Quality of Blood Glucose Meters, 2011

In January 2011, the journal, Diabetes Care, published a study of blood glucose meters: Suboptimal Performance of Blood Glucose Meters in an Antenatal Diabetes Clinic. In addition to highlighting some of the findings, we add Sigma-metrics calculations and Method Decision charts.

Quality of Blood Glucose Meters, 2011

February 2011
Sten Westgard, MS

[Note: This QC application is an extension of the lesson From Method Validation to Six Sigma: Translating Method Performance Claims into Sigma Metrics. This article assumes that you have read that lesson first, and that you are also familiar with the concepts of QC Design, Method Validation, and Six Sigma. If you aren't, follow the link provided.]

This application looks at a study recently published in the journal Diabetes Care:

Suboptimal Performance of Blood Glucose Meters in an Antenatal Diabetes Clinic, Nimalie Jacinthat Perera, MD, Dennis Koon-See Yue, MD, Lynda Molyneaux, RN,  Stephen Morris Twigg, MD, Maria Ines Constantino, BSC, Glynis Pauline Ross, MD, Marg McGill, MSC,  Diabetes Care, online, January 7, 2011.

Managing diabetes in pregnancy is important to prevent any adverse fetal outcomes. There are, however, no goals for the accuracy of glucose meters used for such purposes. Instead, the general goals for accuracy are used.

The Precision and Comparison Data

"Imprecision was evaluated by replicate analysis of glucose measurement on the duplicate meters using plasma samples )n=20) with mean glucose concentrations 5.6 mmol/L and low and mid control solutions for each meter (n=20). Accuracy was evaluated using these meter readings against the plasma glucose measured in the laboratory."

The laboratory method used was a "Hexokinase/G6PD/ADP-NADP/NADPH reference method (Modular-PPE, Rochdiagnostics, Indianapolis, IN)"

The table provided in the study gives mean bias%, as well as mean Total Error calculated as %bias + 1.96*CV%. We can use that and work backwards to calculate the CV.

Imprecision and Bias estimates:

Method mean Bias%
mean Total Error% CV% derived
from Total Error
A 8.99%
14.87% 3.0%
B 13.08%
26.21% 6.7%
C 9.04%
15.65% 3.37%
D
15.76%
32.14% 8.36%
E 6.37%
17.07%% 5.46%
F 6.10%
12.29%% 3.16%

You can already see a variety of bias and imprecision between instruments.

What's the quality required for a glucose meter?

While it's useful to see the imprecision and bias estimates from the study, it's hard to assess them in the absence of context. Is a Total Error of 32% a problem? Is a 13% bias a problem?

CLIA states a proficiency testing criteria of +/- 10%  for glucose central laboratory methods, but there is no official point-of-care quality requirement in the US. The most commonly used standard for performance for glucose meters is the Clarke Error Grid. The usual goal of a Clarke Error grid is to assure that 95% of the results stay within roughly 20% of the "true value" of the test.

If we use a 20% goal, that gives us a better idea of what level of bias is acceptable.

Calculate Sigma-metrics

Another approach that can provide a unified view or summary perspective is to calculate Sigma-metrics.

As detailed in other essays and books, the Sigma-metric Equation is

Sigma-metric = (TEa - bias) / CV

where TEa is the Total Allowable Error, or quality requirement

For example, with method A, we can use the imprecision estimate of 3.0%, as well as the mean bias estimate of 8.99%. We use the quality requirement of 20.0% from the Clarke Error Grid and we have everything we need to calculate the Sigma-metric of the method:

Sigma-metric A = (20 - 8.99) / 3.0

= 11.01/3.0 = 3.67 Sigma

If we summarize all the different levels of performance and sample types, here's what we get:

Method mean Bias%
CV%
Sigma
A 8.99%
3.0%
3.67
B 13.08%
6.7%
1.03
C 9.04%
3.37%
3.25
D
15.76%
8.36%
0.51
E 6.37%
5.46%
2.50
F 6.10%
3.16%
4.40

If this table is too long and abstract, you may find it easier to perform a visual assessment

Method Decision Charts

Using the same data, we can create a Method Decision Chart (MEDX).

MEDx-2011-GlucoseMeters

The Sigma-metrics and MEDx chart may not seem encouraging. But if the usual goal is 95% of results within 20%, that's really aiming for a 5% defect rate, or around 3.1 Sigma. Given that benchmark, three of these methods meet that goal. With our central lab methods, we aim for a more demanding quality requirement, usually 10%, and we accept 3 Sigma only as a minimum performance.

What QC would be appropriate?

Using Operating Specification (OPSpecs) charts, we can further characterize what control limits and numbers of control measurements (N) would be appropriate for these glucose meters:

OPspecs20-2011-BGM

For even the best glucose meter in this study, a set of "Westgard Rules" (2 controls, looking over 2 runs) is most appropriate for proper detection of medically significant errors. Of course, for most glucose meters, the methods themselves are waived, thus all the users need to do is follow the manufacturer directions, which is unlikely to be as robust as these recommendations. If lesser QC procedures are employed, what happens is that error detection is lower, and any problem that occurs takes longer to detect.

Conclusion

From the paper's conclusion:

"This study shows that the current glucose meters are not optimally accurate when compared with plasma glucose measurement..."

"This study identifies a significant and underemphasized clinical problem in the management of DP. Awareness of the potentital inaccuracies of glucose readings is important in advising DP patients on diet and insulin adjustments. This study demonstrates that some of the meter systems appear to have technology-producing results closer to ADA goals, which were thought to be unrealistic or unattainable when they were proposed....Although the ADA goals for performance should remain as a target for manufacturers to improve their performance, in our opinion meters with an analytical error (bias) <10%, total error < 15%, and no hematocrit interference would be minimum for monitoring and managing DP."

Following on the heels of the authors' conclusion, it's worth noting that if Method Decision Charts and OPSpecs charts were set for a 15% goal, none of the meters hit 3 Sigma performance. Since the FDA held a meeting on glucose meter quality in 2010 and the loudest comment was that the allowable error should be tightened from 20% to 15%, it's important to recognize that a future quality requirement may be imposed that these meters cannot fulfill.

Suboptimal Performance of Blood Glucose Meters in an Antenatal Diabetes Clinic, Nimalie Jacinthat Perera, MD, Dennis Koon-See Yue, MD, Lynda Molyneaux, RN,  Stephen Morris Twigg, MD, Maria Ines Constantino, BSC, Glynis Pauline Ross, MD, Marg McGill, MSC,  Diabetes Care, online, January 7, 2011.