POC INR: don't rely on the "r"?

Posted by Sten Westgard, MS

A recent Clinical Laboratory Strategies article:Anchoring POC Quality in Clinical Decision-Making and the related study: Novel analysis of clinically relevant diagnostic errors in point-of-care devices, KM Shermock, MB Streiff, BL Pinto, P Kraus, an dPJ Pronovost,  (J Thromb Haemost 2011;9:1769-1775) have an interesting observation about the use of the correlation coefficient to accept method performance.

They looked at Hemochron POC devices, analyzing 1518 paired INRs. The correlation between the POC and laboratory measurements ranged between 0.84 and 0.91.

The authors stated, "Traditional, quarterly, quality assurance studies emphasize correlation analysis." So this study has good news, right?

Not exactly.

While "Traditional quality assurance showed that the...devices were acceptable for continued use in each quarterly analysis," a more detailed look told otherwise:

"The Hemochron devices systematically inflated INR values <3 and deflated values >4, biasing results towards the target INR range. Consequently, the Hemochron devices lead to a different clinical decision than the clinical laboratory measure in 31% of cases....When the reference INR was low, the Hemochron devices would not result in appropriate dose increases in 52% of cases..., placing those patients at risk for a significant adverse drug event."

The correlation coefficient, long used as the easy statistic to determine acceptability, has proven (yet again) that it cannot be used for this purpose. Medical acceptability can be ascertained when the proper method validation studies have been performed, errors have been estimated, and decision criteria have been established to judge acceptability.

It may be useful to remember a paper from back in 1973, when Method Validation was first introduced to the laboratory:

"The correlation coefficient (r) is sensitive only to random error, but is not easily interpreted. Values for r, t, and F are not useful in making decisions on the acceptability of performance. These decisions should be judgments on the errors that are tolerable. Statistical tests can be applied in a manner that provides specific estimates of these errors."
Use and Interpretation of Common Statistical Tests in Method-Comparison Studies, James O. Westgard and Marian R. Hunt, Clin Chem., Jan 1973; 19: 49 - 57.