Quality Standards
Eight of Nine assays from a Roche Cobas c702 cannot meet pU goals
A continuing investigation into assay capability to meet new performance specifications for permissible measurement uncertainty (pU). Is the Roche c702 able to hit these targets?
8 of 9 assays from a Roche c702 analyzer cannot meet 2021 pU% goals
Sten Westgard, MS
January 2022
- 6 of 8 assays of a Hitachi 7600 cannot meet pU goals.
- 8 of 9 assays of a Cobas c702 cannot meet pU goals.
- 6 of 8 assays of 2 ARCHITECT c16000s cannot meet pU goals.
- 6 of 7 assays of an AU 680 cannot meet pU goals
- MU pU goals, can anything hit them?
The 2021 CCLM article by Braga and Panteghini https://pubmed.ncbi.nlm.nih.gov/33725754/ remains a seminal publication: providing a defined set of performance specifications for measurement uncertainty.
A 2022 article about the analytical performance of a Roche c702 analyzer provides another convenient opportunity to test whether today's instruments can meet the new performance specifications:
Evaluation of the clinical chemistry tests analytical performance with Sigma Metric by using different quality specifications - Comparison of analyser actual performance with manufacturer data. Murat Keles, https://www.biochemia-medica.com/en/journal/32/1/10.11613/BM.2022.010703
While the study evaluated 21 analytes, remember that Braga and Panteghini only published 13 performance specifications. Between those two lists, there were only 9 analytes in common. So we will evaluate if those 9 assays can meet the MU APS.
The Braga and Panteghini pU and the performance
The imprecision was estimated from quarterly IQC data. Bias was estimated by summary of a year of EQA participation, an external quality assessment program, over the same time period.
| Measurand | Milan Model | APS for standard MU, % | APS for desirable lab CV% | Level 1 CV | Level 2 CV | |Bias| |
| total bilirubin | Biological Variation (2nd best) | 10.5% | 5.25% | 3.64% | 2.96% | 2.45% |
| creatinine | Biological Variation (2nd best) | 2.2% | 1.1% | 3.87% | 2.86% | 0.18% |
| glucose | outcome-based (best) | 2.00% | 1.00% | 2.41% | 2.37% | 0.36% |
| sodium | Biological Variation (2nd best) | 0.27% | 0.14% | 1.65% | 1.55% | 0.14% |
| potassium | Biological Variation (2nd best) | 1.96% | 0.98% | 1.73% | 1.86% | 0.01% |
| chloride | Biological Variation (2nd best) | 0.49% | 0.25% | 1.72% | 1.75% | 0.09% |
| total calcium | Biological Variation (2nd best) | 0.91% | 0.46% | 3.37% | 3.1% | 0.81% |
| urea | Biological Variation (2nd best) | 7.05% | 3.03% | 3.4% | 3.16% | 0.35% |
| alanine aminotransferase | Biological Variation (2nd best) | 4.65% | 2.38% | 3.09% | 2.94% | 0.60% |
Does pU Pass or Fail?
When the performance specification is applied to the imprecision measured on this instrument, what is the verdict? Note that the MU and pU are specifications that mostly ignore bias. Measurement Uncertainty can't be combined across all the levels if bias exists. So typically the approaches assume (1) either bias is so small (unspecified) that it can be ignored or (2) the bias varies over the long term, so it can be incorporated as just like another imprecision, or (3) the bias must be eliminated before any of the measurementuncertainty approaches can be applied.
| Measurand | Milan Model | APS for standard MU, % | APS for desirable lab CV% | Level 1 CV | Level 2 CV | level 1 pU? | level 2 pU? |
| total bilirubin | Biological Variation (2nd best) | 10.5% | 5.25% | 3.64% | 2.96% | PASSES | PASSES |
| creatinine | Biological Variation (2nd best) | 2.2% | 1.1% | 3.87% | 2.86% | FAILS | FAILS |
| glucose | outcome-based (best) | 2.00% | 1.00% | 2.41% | 2.37% | FAILS | FAILS |
| sodium | Biological Variation (2nd best) | 0.27% | 0.14% | 1.65% | 1.55% | FAILS | FAILS |
| potassium | Biological Variation (2nd best) | 1.96% | 0.98% | 1.73% | 1.86% | FAILS | FAILS |
| chloride | Biological Variation (2nd best) | 0.49% | 0.25% | 1.72% | 1.75% | FAILS | FAILS |
| total calcium | Biological Variation (2nd best) | 0.91% | 0.46% | 3.37% | 3.1% | FAILS | FAILS |
| urea | Biological Variation (2nd best) | 7.05% | 3.03% | 3.4% | 3.16% | FAILS | FAILS |
| alanine aminotransferase | Biological Variation (2nd best) | 4.65% | 2.38% | 3.09% | 2.94% | FAILS | FAILS |
Are Roche assays failing by design?
The additional wrinkle on this paper is that the authors also determined what the CV performance was supposed to be - based on manufacturer data in reagent package inserts. So, even if the instrument is acting in its most optimal way, can it hit the pU goals?
| Measurand | Milan Model | APS for standard MU, % | APS for desirable lab CV% | Level 1 CV | Level 2 CV | level 1 pU? | level 2 pU? |
| total bilirubin | Biological Variation (2nd best) | 10.5% | 5.25% | 2.1% | 0.8% | FAILS | PASSES |
| creatinine | Biological Variation (2nd best) | 2.2% | 1.1% | 2.2% | 1.7% | FAILS | FAILS |
| glucose | outcome-based (best) | 2.00% | 1.00% | 1.3% | 1.1% | FAILS | FAILS |
| sodium | Biological Variation (2nd best) | 0.27% | 0.14% | 0.8% | 0.5% | FAILS | FAILS |
| potassium | Biological Variation (2nd best) | 1.96% | 0.98% | 0.9% | 0.5% | PASSES | PASSES |
| chloride | Biological Variation (2nd best) | 0.49% | 0.25% | 0.8% | 0.6% | FAILS | FAILS |
| total calcium | Biological Variation (2nd best) | 0.91% | 0.46% | 0.8% | 0.9% | FAILS | FAILS |
| urea | Biological Variation (2nd best) | 7.05% | 3.03% | 1.2% | 1.1% | PASSES | PASSES |
| alanine aminotransferase | Biological Variation (2nd best) | 4.65% | 2.38% | 1.4% | 1.0% | PASSES | PASSES |
So there are only three to four assays, according to Roche claims, that can possible fulfill the pU% goals. And then, in practice, only one of them succeeds. This points out that there are some powerful disconnects between the new maximum permissible measurement uncertainty performance specifications and the current engineering of today's diagnostic instrumentation. The pU goals are laudable, but they don't look practically achievable. If we're to try to achieve analytical improvement, we have to mix the carrot with the stick. Highlight and approve of the best of what's being done today, while strongly encouraging even more improvement in the future.