Tools, Technologies and Training for Healthcare Laboratories

Four Glucose Methods

Tight Glycemic Control is one of the latest trends in laboratory medicine - one that requires fast, frequent, and precise testing. A 2008 paper studied four different glucose methods, one POC, 2 blood gas, and one central laboratory hexokinase method. Can you guess which methods performed the best, and what methods were good enough to support TGC?

February 2009
Sten Westgard, MS

[Note: This QC application is an extension of the lesson From Method Validation to Six Sigma: Translating Method Performance Claims into Sigma Metrics. This article assumes that you have read that lesson first, and that you are also familiar with the concepts of QC Design, Method Validation, and Six Sigma. If you aren't, follow the link provided.]

This application looks at a recent paper, "Critical evaluation of connectivity-based point of care testing systems of glucose in a hospital environment" by Katelijne M.J.Flore, Tom Fiers and Joris R. Delanghe, Clin Chem Lab Med 2008; 46(12):1763-1768.

Glucose monitoring has been profoundly impacted by point of care devices. More recently, the concept of "tight glycemic control" has attempted to leverage the power of taking frequent glucose measurements. But if tight glycemic control is to be successful and effective, it must involve fast turn-around time combined with highly precise and accurate testing.

Flore, Fiers, and Delanghe's study took a comprehensive look at a glucometer as well as two blood-gas instruments, comparing them to a central lab method, where the hexokinase method principle was used as a reference. They went beyond the usual method validation protocols, using larger datasets and performing additional studies to determine where components of variation were occuring at the the point of care and their magnitude.

Flore, Fiers and Delanghe calculated the total error of the methods and compared them against allowable total error. In this application, we're going to use Sigma-metrics as the technique of method evaluation. At the end, we'll compare our results with Delanghe to see if the two approaches agree.

For this particular application, we're going to blind the brand names of the instruments. Rather than focus on a "gotcha" of a specific instrument, let's just look at the comparisons between a glucometer, a blood gas, and a central lab method.

We'll call the glucometer POC, the two blood gas instruments BG1 and BG2, and the central lab hexokinase method CLHX. If you're really curious about the identity of these methods, simply click through to read the entire paper.

The Precision and Comparison data

The precision studies for the blood-gas and central lab methods are quite robust. They are based on an entire year of daily quality control results. The number of measurements are based on hundreds of observations, not just the small sample that you typically find in an EP5 or EP15 study. This kind of data gives us reliable estimates of imprecision. Bias was also calculated as the difference between the target values and the observed values of these control results.

Instrument n Level (mmol/L) CV% Bias%
POC 281 3.1 8.7 3.1
est. 5.6 est. 8.0 3.1
295 17.2 5.7 3.2
BG1 154 2.6 2.7 6.5
161 5.6 3.0 0.9
172 11.4 2.7 1.9
BG2 161 2.4 3.2 6.1
100 5.6 2.2 0.9
198 13.6 2.1 0.7
CLHX 5.3 5.3 1.7 2.1
14.5 14.5 1.9 2.7

When given multiple levels of QC data, we have to narrow our focus to a critical decision level. Recently, the ADA lowered the cutoff for fasting plasma glucose to 5.6 mmol/L. Patients whose results breach this lower cutoff are considered to have Impaired Fasting Glucose (IFG) and may be more vulnerable to diabetes and associated health problems.

This cutoff is particularly attractive since both BG1 and BG2 have their mid-level control run at that level, while the lower level of the CLHX is very close by at 5.3 mmol/L. It gets more tricky with the POC device, since the closest observation is made at a level of 3.1 mmol/L and it's the worst measure of imprecision for all of the instruments. We're going to extrapolate the imprecision for the POC device at 5.6 mmol/L, making a slightly lower estimate of 8.0%.

Determine the quality requirement at the critical decision level

Finding the quality requirement for glucose, one of the most common laboratory measurements, should be easy. But it's not. For glucometers, performance is usually measured using Clarke's error grid, which provides multiple zones of interpretation that vary over the analytical range. The shorthand quality requirement often used with the Clarkes error grid is 20%. Coincidentally, ISO 15197 also sets a goal of 20% for measurements over 4.2 mmol/L. In contrast, the CLIA proficiency testing criterion for glucose is 10%. Of course, CLIAs goal was set with central laboratory measurements in mind, while the error grid and ISO 15197 attempts to take into account the greater variation present in whole blood point of care measurements. As if to split the difference, the German Rilibak rules have placed their goal for glucose interlaboratory comparisons at 15%.

So there is a range of 10-20% allowable errors in glucose measurements. In this case, we're going to calculate the metrics for each quality requirement.

Calculate Sigma metrics

Now we have all the pieces in place.

Remember the equation for Sigma metric is (TEa - bias) / CV:

For a 20% quality requirement, with the POC method, (20.0 - 3.1) / 8.0 = 2.11

The metrics are displayed along the right columns.

Instrument Level (mmol/L) CV% Bias % Sigma

TEa=20%

Sigma

TEa=15%

Sigma

TEa=10%

POC 5.6 est. 8.0 3.1 2.11 1.49 0.86
BG1 5.6 3.0 0.9 6.37 4.70 3.03
BG2 5.6 2.2 0.9 8.68 6.41 4.14
CLHX 5.3 1.7 2.1 10.53 7.59 4.65

A pretty wide range of performance here. It seems like the further away from point of care, the higher the quality of the measurement. The central laboratory method does behave like a reference method, achieving world class quality at 20% and 15% error goals, and approaching 5 Sigma when the tighter CLIA goal is used.

The POC method, however, does not do well by any standard. In other industries, processes below 3 Sigma are not considered stable enough for routine implementation. According to this data, the POC is not providing enough quality.

Summary of Performance by Sigma-metrics chart

Here's a Method Decision chart, using Six Sigma metrics lines to delineate the performance of the methods. This is the same data from the table above presented graphically. We'll present the chart for the 20% error goal

qcapp55figure2

This makes clear the large gap in performance between the POC and the other methods. However, while non-POC methods are "world class" by the 20% requirement, if we hold them to the 10% requirement, the picture isn't quite as positive:

Method Decision chart, 10% requirement

Clearly, how you judge these methods is heavily dependent on the standard you apply.

QC Design using OPSpecs chart

Not only can we use tools to graphically depict the performance of the method - we can also use those tools to help determine the best QC procedure to use with that method. Using EZ Rules 3, we can express the method performance on an OPSpecs (Operating Specifications) chart :

qcapp55figure3

With OPSpecs charts, the lines above the operating point mean that those QC procedures are acceptable for use with that method. For example, with the CLHX method, the 2.5s rule with 2 controls or 4 controls would provide sufficient error detection. Multirule combinations with 4 control measurements would also work for CLHX and BG2. BG1, unfortunately, is beyond the ability of the presented control procedures. POC is off the charts. Neither BG1 nor POC can be adequately controlled to a 10% allowable error. Bias and imprecision are simply too large in these methods - aggressive QC can't make up for that variation.

Conclusions

It's rare to find a study that covers so much ground and compares so many methods objectively.

In this case, the authors of the paper concluded that the variation in the POC method was too large:

"In conclusion, total analytical variation of POCT glucose systems in a hospital environment is larger than expected. Clinicians should be aware that the variability of glucose measurements obtained by blood gas instruments is significantly lower than the variability obtained by capillary blood with a handheld glucometer.... Furthermore, the high CV values obtained by both blood gas instruments and glucose hydrogenase based POCT systems implies that low glucose concentrations need to be confirmed with a hexokinase based method in the central laboratory."

Sigma-metrics confirm that conclusion. There is a large gulf of performance between the POC method and the other methods. The study goes further to determine all the sources and magnitudes of variation. But there is also an important difference between the blood gas methods and the central laboratory method, when a tighter quality requirement is applied. The hexokinase method is a superior choice for glucose measurements.