Quality Standards
13 of 15 assays on a Roche Cobas 8000 cannot meet preferred perimissible uncertainty performance specifications
A recent study of Roche Cobas 8000 chemistry assays in Turkey looks at their measurement uncertainty. We compare the Roche mu against the EFLM MAU and the preferred permissible uncertainties proposed by the leading authorities on mu.
13 of 15 assays on a Roche Cobas 8000 in Turkey cannot meet preferred measurement uncertainty goals, over a period of 2 years
February 2023
Sten Westgard, MS
Two papers from Turkey in the same month! While laboratories elsewhere in the world seem to shy away from publishing their uncertainty, the labs in Turkey are doubling down. Now that we have the goals for measurement uncertainty, more and more papers are holding today's instruments to the new standards.
The most recent paper to do so is
Comparison of Two Clinical Chemistry Analyzers by Total Analytical Error and Measurement Uncertainty
Muhammed E. Duz, Elif Menekse, Burak Y. Avci, Alper Gumus. Clin. Lab. 2023;69:161-171
This study is quite thorough in its calculation of measurement uncertainty, but it is not a perfect calculation. In this case, manufacturer controls are used, without any traceability information, and the uncertainty of calibration is not included in the calculation of the full laboratory uncertainty. However, there is certainly quantity even if the quality is lacking: "...ClinChem Multi control (Roche Diagnostics Cor-poration, Indianapolis, Indiana, USA) with low and high levels were used and studied daily throughout the years 2019 and 2020....The % CV was calculated separately from each year’s 12-month quality control data as normal and pathological levels" Since the preferred estimate of uncertainty is supposed to come from 6 months or more of performance, this study has double the data, and we'll make two benchmarks, one for each year.
"A six-stage measurement uncertainty calculation model defined in the Nordtest manual was used to calculate the MU. Using the formula below, the expanded uncertainty of measurement (U%) was calculated according to the Nordtest guideline (k: 1.96 for 95% coverage probability). U% = k x √¯(imprecision2 + RMSbias2). RMSbias (Root mean square bias) were calculated using 12 bias results from different months for each parameter in each year as given....RMSbias = √¯[(Bias12 + Bias22 + Bias32 + …)/n]" While this is not the currently preferred way to calculate measurement uncertainty, it does reflect a very rigorous approach. It should be noted this model includes Bias observed as another uncertainty.
The sources for these measurement uncertainty performance specifications can be found in these references:
- EFLM database, accessed 11/26/2022. https://biologicalvariation.eu/meta_calculations
- Performance specifications for measurement uncertainty of common biochemical measurands according to Milan models, Federica Braga and Mauro Panteghini, CCLM 2021; 59(8):1362-1368.
- Definition and application of performance specifications for measurement uncertainty of 23 common laboratory tests: linking theory to daily practice. Braga F, Pasqualetti S, Borrillo F, Capoferri A, Chibireva M, Rovegno L, Panteghini M, CCLM 2022 https://doi.org/10.1515/cclm-2022-0806
- Redesigning the surveillance of in vitro diagnostic medical devices and of medical laboratory performance by quality control in the traceability era. Mauro Panteghini. CCLM 2022. https://www.degruyter.com/document/doi/10.1515/cclm-2022-1257/html
We will focus on tehse 15 biochemistry assays in the paper. The 2022 papers from Panteghini et al provide the definitive, preferred specifications for permissible uncertainy (pU, or as well will refer to them, ppU). The EFLM database provides specifications for MAU, minimum and desirable, for all 15 analytes.
Roche Cobas 8000 |
Level | 2019 MU | 2020 MU |
EFLM Desirable, Minimum MAU |
Preferred pU (ppU) |
Final verdict |
Albumin | 1 | 10.28% | 7.24% | 2.5% (des) 3.8% (min) |
1.25% (des) |
Fails MAU and ppU 2019, 2020 |
2 | 8.19% | 7.35% | Fails MAU and ppU 2019, 2020 |
|||
ALT | 1 | 15.71% |
12.07% |
10.1% (des) 15.1% (min) |
4.65%(des) 6.98%(min) |
Fails ppU 2019, 2020 Fails MAU 2019, passes min MAU 2020 |
2 | 14.73% | 12.88% | Fails ppU 2019, 2020 Passes min MAU 2019, 2020 |
|||
ALP | 1 | 16.95% | 18.75% | 6.6% (des) 9.9% (min) |
2.65% (des) 3.98% (min) |
Fails MAU and ppU 2019, 2020 |
2 | 15.62% | 19.3% | Fails MAU and ppU 2019, 2020 |
|||
AST | 1 | 9.7% | 7.8% | 9.6% (des) 14.4% (min) |
4.75% (des) 7.13% (min) |
passes min MAU, fails ppU 2019, 2020 |
2 | 9.24% | 7.51% | passes min MAU, fails ppU 2019, 2020 | |||
Chloride | 1 | 5.54% | 6.61% | 1.7% (des) 1.1% (min) |
0.49% (des) 0.74% (min) |
Fails MAU and ppU 2019, 2020 |
2 | 6.08% | 6.17% | Fails MAU and ppU 2019, 2020 | |||
Cholesterol, Total | 1 | 9.93% | 8.89% | 5.7% (des) 7.9% (min) |
3% (des) 7% (min) |
Fails MAU and ppU 2019, 2020 |
2 | 10.37% | 8.71% | Fails MAU and ppU 2019, 2020 | |||
Creatinine | 1 | 22.36% | 13.07% | 4.5% (des) 6.8% (min) |
2.2% (des) 3.3% (min) |
Fails MAU and ppU 2019, 2020 |
2 | 20.94% | 12.52% | Fails MAU and ppU 2019, 2020 | |||
Glucose | 1 | 9.95% | 5.62% | 5% (des) 7.5% (min) |
2% (des) 3% (min) |
Fails MAU and ppU 2019, 2020 |
2 | 9.28% | 5.66% | Fails MAU and ppU 2019, 2020 | |||
LDH | 1 | 11.3% | 7.4% | 5.2% (des) 7.8% (min) |
2.6% (des) 3.9% (min) |
Fails MAU 2019, ppU 2019, 2020 passes min MAU 2020 |
2 | 11.44% | 7.07% | Fails MAU 2019, ppU 2019, 2020 passes min MAU 2020 |
|||
HDL-Cholesterol | 1 | 14.07% | 8.38% | 5.8% (des) 8.7% (min) |
2.84% (des) 4.26% (min) |
Fails MAU 2019, ppU 2019, 2020 passes min MAU 2020 |
2 | 14.98% | 8.19% | Fails MAU 2019, ppU 2019, 2020 passes min MAU 2020 |
|||
Potassium | 1 | 8.36% | 4.45% | 4.1% (des) 6.1% (min) |
1.96% (des) 2.94% (min) |
Fails MAU 2019, ppU 2019, 2020 passes min MAU 2020 |
2 | 8.49% | 4.28% | Fails MAU 2019, ppU 2019, 2020 passes min MAU 2020 |
|||
Protein, Total | 1 | 9.58% | 6.61% | 2.6% (des) 3.9% (min) |
1.3% (des) 1.95% (min) |
Fails MAU and PpU 2019, 2020 |
2 | 9.25% | 6.31% | Fails MAU and PpU 2019, 2020 |
|||
Sodium | 1 | 3.89% | 4.95% | 0.5% (des) 0.8% (min) |
0.27% | Fails MAU and PpU 2019, 2020 |
2 | 4.55% | 4.78% | Fails MAU and PpU 2019, 2020 | |||
Triglycerides | 1 | 15.21% | 9.1% | 20% (des) 30% (min) |
9.9% (des) 14.9% (min) |
Passes des MAU 2019, 2020 Passes ppU 2019, passes min ppU 2020 |
2 | 14.79% | 9.73% | Passes des MAU 2019, 2020 Passes min ppU 2019, passes des ppU 2020 |
|||
Urea Nitrogen | 1 | 9.97% | 7.99% | 13.9% (des) 20.9% (min) |
9.9% (des) 14.9% (min) |
Passes des MAU 2019, 2020 Passes min ppU 2019, passes des ppU 2020 |
2 | 10.07% | 8.01% | Passes des MAU 2019, 2020 Passes min ppU 2019, passes des ppU 2020 |
If we judge these assays by the ppU goals alone, we have 15 assays, and for 13 assays, there is complete failure at both levels for two years. A 86% failure rate for the Roche Cobas 6000 is not encouraging.
As we have seen with other Roche mu analyses, and other mu analyses of other instruments, these new measurement uncertainty performance standards are harsh, perhaps impractically harsh, beyond the capability of even the most advanced instrument as a Cobas 8000. That should raise concerns about the promulgation and mandatory implementation of these new goals throughout the world.