Quality Requirements and Standards

A new study from Lenters-Westra, Slingerland et al was published in early 2011. The title says it all: One in Five Laboratories Using Various Hemoglobin A1c Methods Do Not Meet the Criteria for Optimal Diabetes Care Management. What were the findings and what does this mean for HbA1c methods?

April 2011
Sten Westgard, MS

It seems that 2011 is destined to become the year of the HbA1c method. There have been significant studies published in just the first few months of the year, and it looks like there will be a lively debate for the rest of the year.

In the journal Diabetes Technology and Therapeutics, Erna Lenters-Westra and Robbert Slingerland and colleagues have published another impressive study with a dramatic title:

One in Five Laboratories Using Various Hemoglobin A1c Methods Do Not Meet the Criteria for Optimal Diabetes Care Management, Erna Lenters-Westra, Cas Weykamp, Roger K. Schindhelm, Carla Siebelder, Henk J. Bilo, and Robbert Slingerland, Diabetes Technology and Therapeutics, Vol. 13, No.4, 2011.

This study has an interesting focus. While imprecision was being estimated for multiple methods from Bio-Rad, Arkray/Menarini, Roche, Tosoh, and others, the main focus was actually on the Reference Change Value (RCV). In other words, the study evaluated whether HbA1c methods could register a critical difference  between two serial patient test results that was equal or smaller to what is considered a clinically significant change in test results. As it turns out, there are some methods with RCVs larger than the generally accepted clinically significant change in HbA1c results. Thus, while a clinician may believe a change in test results on those methods is clinically important, it may in fact only be the "noise" of the analytical method. Given the amount of testing devoted to monitoring diabetes, this poses a considerable challenge to the proper treatment of the disease.

The study data

This study was the result of a large trial involving External Quality Assurance (EQA) programs in Belgium (Stichting Kwaliteitsbewaking Medische Laboratoria, SKML) and the Netherlands (WIV). The results of these EQA programs, plus those of a ring survey with two pooled fresh whole blood samples,

"were used to assess the individual laboratory performance of various HbA1c laboratory methods...The design of the Dutch SKML and the Belgian WIV scheme is based on 24 lyophilized interconnected samples. The samples are sent annually to all participating laboratories and stored at -20 C or below. Each sample is requested to be analyzed every fortnight, and the results are to be submitted to the website of SKML / WIV. The 24 samples were in fact 12 samples in duplicate. The duplicates were blinded to prevent any influence on results."

Already, you can hear the objections. It's nice that they have a long-term assessment of imprecision, but lyophilized samples require a reconstitution step, which can inject more variability into the assessment. Lenters-Westra and Slingerland already thought of that:

"To avoid discussions about commutability of lyophilized samples for certain methods with respect to systematic error (bias), we used two pooled fresh whole blood samples that were sent halfway through the inverval of the SKLM/WIV scheme (in May) to the laboratories, similar to the College of American Pathologists' (CAP) survey. The values were assigned with five International Federation of Clinical Chemistry (IFCC) secondary Reference Measurement procedures on two days in duplicate."

Imprecision was estimated using these samples, and that CV was used in the calculation of Reference Change Value, or RCV. The RCV is also known as the critical difference, and it represents the smallest change between two serial test results on the same patient that is considered significantly different at a probability of 95%. You can see longer discussions of RCV on the website here. But to be brief, we'll restate the equation here:

RCV (%) = Z * 2^½ * (CV_A² + CV_W²)^½

The key point here is that the RCV is driven by two factors, the imprecision of the analytical method (CV_A ) and the within-subject biologic variation (CV_W).

Another interesting feature of this paper is that the authors assert that the within-subject biologic variation for HbA1c is 1%. This is in contrast to the findings of the Ricos database, which estimate that the within-subject biologic variation for HbA1c is 3.4%. The authors note that they have unpublished data which supports this estimation, and we look forward to a future publication of those findings.

The results

Using the calculated RCV value (at an assumed 1% CV_w), here are the calculated critical differences for the mean performance of these methods.

The n listed with each method indicates the number of laboratories using that method. The key here is to note (in green) the methods that have RCVs smaller than 0.5. These are methods that can support a clinically significant change of 0.5% HbA1c or smaller. The methods in blue have RCVs larger than that change.

In other words, if clinicians assume a change of 0.5% HbA1c in a patient is significant, they need to be using the methods highlighted in green. If not, the clinicians need to adjust their interpretations so that they account for the larger critical differences.

Note again that this is assuming the within-subject biologic variation is 1%. If the 3.4% figure from Ricos is used, most of the methods have RCVs larger that 0.5% [the paper does include these calculations]. So we need that additional data on what the proper within-subject biologic variation is for HbA1c, if we are to make the right decisions based on this study.

But recall that this graph is the mean performance of these methods. So some labs with a given method do better than what the graph shows (a smaller RCV), some labs do worse (a larger RCV). In fact, the study authors tracked the individual labs, regardless of method, and determined how many were able to achieve the desired RCVs:

"...based on the calculated RCVs, almost 22% of HbA1c methods are not able to distinguish an HbA1c result of 59 mmol/mol (7.5%-DCCT) from a previous HbA1c result of 53 mmol/mol (7.0% DCCT). This may have a profound impact on the management of patients with diabetes if changes in medication are made due to changes in serial HbA1c measurements."

The whole study is worth reading. We look forward to seeing more results.