Tools, Technologies and Training for Healthcare Laboratories

Quality of Blood Glucose Meters, An Update

Back in 2010, an study reviewed 27 different blood glucose meters to see if they met the quality requirements of the Clarke Error Grid and different editions of the ISO 15197 standard. In 2012, this review was udpated to include 43 different blood glucose meters. We review the data to see if new meters mean better meters.

 

 

September 2013
Sten Westgard, MS

Back in 2010, a comprehensive study of a multitude of Blood Glucose Meters (BGMs) was conducted:

System Accuracy Evaluation of 27 Blood Glucose Monitoring Systems According to DIN EN ISO 15197, Guido Freckmann, MD, Annette Baumstark, PhD, Nina Jendrike, MD, Eva Zechomack, MD, Serge Kocher, PhD, Jacques Tshiananga, MPH, Frank Heister, PhD, and Cornelia Haug, MD, Diabetes Technology and Therapeutics, 2010;12;3:221-231.

Not only did this study evaluate BGMs on the basis of whether they met current quality standards, but they recorded the data in such a way that we can evaluate whether any of these BGMs will meet possible future quality standards. In that study, it found that only 9 BGMs would meet the suggested tighter standards for ISO 15197.

In 2009-2011, the study authors continued to evaluate BGMs as new models were introduced into the marketplace:

System Accuracy Evaluation of 43 Blood Glucose Monitoring Systems for Self-Monitoring of Blood Glucose according to DIN EN ISO 15197. Guido Freckmann, MD, Christina Schmid, PhD, Annette Baumstrack, PhD, Stefan Pleus MS, Manuela Link ME, and Cornelia Haug, MD, Journal of Diabetes Science and Technology. September 2012:6(5):1060-1075.

The study

Freckmann et al prepared their evaluation study according to the methodology described in DIN EN ISO 15197-2003. It was conducted in 2009-2011 at the Institute for Diabetes-Technology in Ulm, Germany.

"Reference measurements were performed with the following two different methods for all BG systems: glucose oxidase (GOx) (YSI 2300 STAT Plus glucose analyzer, YSI Life Sciences, Yellow Springs OH; measurements were performed at the study site) and hexokinase (HK) [Nitachi 917 (from January 2009 to August 2010)/cobas 6000 c501 (since August 2010), Roche Diagnostics GmbH, Mannheim, Germany,; measurements were performed at a Deutsche Akkreditierungsstelle-accredited calibration laboratory of Roche Diagnostics GmbH].

"The accuracy of the GOx method was verified measuring NERL Glucose Standards (Thermo Fisher Scientific, East Providence RI), verified against National Institute of Standards and Technology (NIST) (Gaithersburg, MD) reference material. The accuracy of the HK method was verified measuring NIST standard reference Material 965a (from January 2009 to February 2011) or 965b (since February 2011)."

In other words, this comparison of accuracy was done with methods that were checked against true reference materials and standards. This helps establish traceability and gives more confidence in the findings.

The patient results were carefully partitioned so that results covered a broad range of the test method:

"Native capillary blood samples werre used at BG concentraions of 50 to 400 mg/dL.... At least 100 fresh capillary blood samples from 100 subjects were collected."

The assessment criteria

The study assessed whether or not results fell within 15, 10, and 5 mg/dL at concentrations less than 75 mg/dL, and whether or not results fell with 20%, 15%, 10%, or 5% at concentrations above 75 mg/dL. That is in accord with the current version of the ISO 15197 standard. However, a draft version of the next edition of ISO 15197 will contain different standards.

The difference between the ISO 15197 standards involves the cutoff between a unit and a percentage-based quality requirement. The current version looks at goals above and below 75 mg/dL, while the new edition will possibly look at performance above and below 100 mg/dL.

The current ISO 15197 standard sets the goal of 95% of results should fall within 15 mg/dL of the true value < 75 mg/dL, and within 20% of the true value for values  > 75 mg/dL. [International Organization for Standardization. ISO 15197 in vitro diagnostic test systems—Requirements for blood-glucose monitoring systems for self-testing in managing diabetes mellitus. 2003.]

The possible update to ISO 15197 may require 95% of results to fall within 15 mg/dL of the true value < 100 mg/dL, and within 15% of the true value for values  > 100 mg/dL. Obviously, this is a tighter standard to meet. But there are other studies and standards that support the tightening of the current standard.

The American Diabetes Association, in 1987, recommended that glucose meter results should fall within 15% of reference systems. [Consensus Development Panel: Consensus statement on self-monitoring of blood glucose. Diabetes Care 1987; 10:95-99.] In 1996, the ADA reduced the goal to 5%, but frankly no one has expected meters to meet this standard.

Finally, Karon, Klee and Boyd used simulation studies based on a databse of patient data results to model the appropriate standard. Their study reached the conclusion that 95% of the glucose meter results should fall within 15% of the true value. [Glucose Meter Performance Criteria for Tight Glycemic Control Estimated by Simulation Modeling, Brad S Karon, James C. Boyd, and George G. Klee, Clinical Chemistry 2010;56:7; 1091-1097] This recommendation is similar to the consensus that appeared to be emerging from the FDA public hearing on blood glucose meters a few years ago.

The assessment

The recent study was able to assess whether or not 34 of the 43 BGMs met the standards of the current and proposed ISO 15197 standard. 9 BGMs were not able to be completely assessed due to an oxygen dependency specified on the manufacturer's labeling.

Below is a summary table of the 34 BGMs and whether or not they met the criteria of the different quality standards.

Instrument 20% and +/- 15 mg/dL
(current ISO 15197)
15% and +/- 15 mg/dL
(proposed new ISO 15197 standard)
Accu-Chek Active Yes Yes
Accu-Chek Aviva Yes Yes
Accu-chek Aviva Nano Yes Yes
Accu-Chek Compact Plus Yes Yes
Accu-Chek Go Yes Yes
Accu-Chek Mobile
(maltose dependent)
Yes Yes
Accu-Chek Mobile
(maltose independent)
Yes Yes
Accu-Chek Performa
(maltose dependent)
Yes Yes
Accu-Chek Performa
(maltose independent)
Yes Yes
Accu-Chek Performa Nano Yes Yes
Bayer Contour usb Yes No
Beurer GL 32 Yes No
Beurer GL 40 Yes No
Biocheck TD-4225 No No
Instrument 20% and +/- 15 mg/dL
(current ISO 15197)
15% and +/- 15 mg/dL
(proposed new ISO 15197 standard)
FreeStyle Freedom Lite Yes Yes
FreeStyle Lite Yes Yes
Futura Monometer No No
GlucoCheck Classic Yes No
GlucoCheck XL Yes No
Glucohexal II No No
GlucoRx (TD-4230) No No
GlucoSmart Swing Yes No
GlucoTel Yes No
Gluco-Test Plus+ TD-4230 Yes Yes
iDia Yes No
IME-DC Fidelity No No
iXell Yes No
iXell OLED Yes Yes
Instrument 20% and +/- 15 mg/dL
(current ISO 15197)
15% and +/- 15 mg/dL
(proposed new ISO 15197 standard)
microdot+ Yes Yes
OneTouch Verio Yes Yes
OneTouch Verio Pro Yes No
Pura Yes Yes
SeniorLine GM210 No No
Wellion CALLA Light No No

For 9 BGMs, a more limited assessment was performed, with the following results:

Instrument 20% and +/- 15 mg/dL
(current ISO 15197)
15% and +/- 15 mg/dL
(proposed new ISO 15197 standard)
BGStar Yes Yes
Element Yes No
GlucoCheck Comfort Yes Yes
iBGStar Yes Yes
Omnitest 3 Yes No
OneTouch VITA Yes Yes
SmartLAB genie No No
SmartLAB global No No
WaveSense Jazz Yes Yes

 

Conclusion

The study authors offered this conclusion:

"Our study shows that systems with a CE mark do not necessarily exhibit equal quality and therefore should not be used interchangably without further considerations such as evaluation of measurement accuracy."

Of the 34 completely assessed systems, 7 (20.5%) did not fulfill the minimum accuracy requirements of the ISO 15197:2003 standard. 16 (47%) of those systems will not fulfill the proposed tighter quality requirements for the proposed ISO 15197 guideline. Of the 9 incompletely assessed BMs, 2 of 9 didn't meet the current ISO 15197 guideline, and 4 of 9 didn't meet the proposed tighter ISO 15197 guideline.

If we combine the assessments, we see that 11 (20%) cannot meet current quality requirements, and 18 (41%) BGMs that cannot meet future quality requirements. Given that the previous study found 40% of BGMs couldn't meet the current quality requirements and 66% of BGMs couldn't meet future quality requirements, we are seeing gradual improvements in the performance of BGMs on the marketplace. However, the pace is too slow and manufacturers should be producing much better meters.

The striking finding of this study is the same as the first study: many of the BGMs on the market do not provide enough accuracy. Furthermore, the CE label is not a guarantee of quality for BGMs now or in the future. If we are trying to improve the accuracy and usefulness of BGMs, it appears we will need serious improvements over the current generation of instrumentation.