Tools, Technologies and Training for Healthcare Laboratories

Roche cobas 6000 c501 in Saudi Arabia, multimode analysis

We look at the impact of new CLIA and EFLM goals on the assessment of a Roche cobas 6000 c501 instrument in Saudi Arabia.

Roche cobas 6000 c501 in Saudi Arabia, multimode analysis

February 2023
Sten Westgard, MS

See the other analyses in this series:

This Roche cobas 6000 c501 study comes from Saudi Arabia, possibly from King Abdulaziz Specialist Hospital, although the paper only specifies it as the "Department of Laboratory and Blood Bank of an accredited teriary care hospital":

Evaluation of Measurement Performance of Routine Chemistry Analytes by Application of Sigma Metrics, Method Evaluation Decision (MEDx) Charts and Quality Goal Index (QGI). Awan TM, Al Baiomy AA, Malik MH, Al-Ruwaili FK, Abdelrahman R, and Al-Johani AZ, J Biomed Res Environ Sci, https://www.jelsciences.com/articles/jbres1665.pdf

"This prospective study was carried out over a period of 6 months from Jan to June 2022....[T]wo levels of QC material are run i.e. normal L1 and pathological L2 from Roche Germany, daily in the morning shift and on 'as and when required' basis during evening and the night shifts...."

"Monthly %Bias for an analyte is estimated by the RIQAS-EQAS program based on the difference between the monthly EQA result obtains by the laboratory's method and the peer group mean for comparison calculated from results of all the participating laboratories from all over the world....Average of these monthly %Deviations for a period of 6 months was taken to estimate the Average %Bias."

The use of manufacturer controls are a weakness of this study, as they are less likely to provide an honest, independent assessment of the performance of the instrument. But we'll go ahead and use them, because, as you will see, even the most optimistic controls don't provide a rosy picture.

There are two levels that will be analyzed here, for 24 parameters when possible. The study covered the Direct Bilirubin analyte, but since neither the EFLM nor the CLIA 1992 or 2024 have specific performance criteria for this analyte, we will omit it here in this study. So for 12 parameters there will be 24 data points to plot on the Normalized Method Decision Chart. Note that, since there are 2 observed CVs but only one observed bias, the plotted points may visually appear as bands. Also, when data points are plotted at the edge of the graph, this indicates the imprecision has exceeded the allowable specification significantly. Also, those data points at the edge will be plotted with slightly different biases in order to show both points.

The TEa goals applied can be found on our Consolidated Chemistry Performance Specifications page.

TEST % Bias %CV
Albumin 2.48 2.46
Albumin 2.48 1.96
Alk Phos 1.34 2.75
Alk Phos 1.34 2.39
ALT 5.38 2.92
ALT 5.38 2.67
Amylase 1.70 1.96
Amylase 1.70 2.23
AST 2.93 2.13
AST 2.93 2.18
Total Bilirubin 2.90 4.75
Total Bilirubin 2.90 5.03
Calcium 0.85 1.78
Calcium 0.85 1.76
Chloride 0.18 2.31
Chloride 0.18 2.28
Cholesterol 0.60 2.14
Cholesterol 0.60 2.02
CK 1.17 2.14
CK 1.17 2.32
Creatinine 0.78 3.05
Creatinine 0.78 2.81
GGT 2.48 1.86
GGT 2.48 1.80
Glucose 1.23 2.01
Glucose 1.23 1.85
D-HDL 0.27 1.78
D-HDL 0.27 1.89
Iron 3.28 4.29
Iron 3.28 3.36
LDH 3.80 4.82
LDH 3.80 5.07
Magnesium 0.82 2.15
Magnesium 0.82 2.19
Phosphorous 1.78 3.18
Phosphorous 1.78 2.61
Potassium K 0.30 2.29
Potassium K 0.30 2.12
Protein, Total 3.05 2.15
Protein, Total 3.05 2.27
Sodium 0.38 2.00
Sodium 0.38 1.98
Triglycerides 4.65 4.05
Triglycerides 4.65 3.91
Uric Acid 1.43 2.39
Uric Acid 1.43 2.51
Urea Nitrogen 0.09 2.77
Urea Nitrogen 0.09 2.64

Sigma-metrics according to EFLM-derived DESIRABLE performance specifications

 2023 cobas c501 Saudi Arabia EFLM des goals

The EFLM desirable specifications used to be the de facto global standard, but have fallen out of favor due to their toughness.

The Roche cobas 6000 c501 has 43% of the performance below 3 Sigma, and fully half of the performance is at 3 sigma or lower.

Only 8 of the 48 points are in the Six Sigma zone.

Not surprising then, that EFLM recommended lowering the standards.

Sigma-metrics according to EuBIVAS-derived MINIMUM performance specifications

 2023 cobas c501 Saudi Arabia EFLM min goals
There is an improvement, now a majority of the performance is in the Six Sigma zone, and only about a one-third of that performance is at 3 Sigma or lower.

Here's one of the most interesting new aspects of CLIA's 1992 and new 2024 goals. Are they more or less demanding than EFLM goals?

Sigma-metrics according to CLIA 1992 performance specifications

2023 cobas c501 Saudi Arabia CLIA 1992 goals

Now, some assays are not covered by CLIA 1992, so not all assays are able to be plotted. But those that are, there is one-third that is in the Six Sigma zone, and one-third that are 3 Sigma or lower. Note this: CLIA 1992 is less forgiving than EFLM minimum specifications.

Sigma-metrics according to CLIA 2024 performance specifications

2023 cobas c501 Saudi Arabia CLIA 2024 goals 

The CLIA 2024 goals are more stringent than CLIA 1992, obviously. But less than one-quarter of the performance falls into the Six Sigma zone, and a majority of performance is at 3 Sigma and lower. CLIA 2024 is more demanding than the EFLM minimum goals.

Conclusion

Even with possibly the most favorable assessment (from their own controls), the Roche Cobas 6000 c501 is not able to meet a significant number of the the desirable EFLM goals and the CLIA 2024 goals. While EFLM minimum specifications are the most forgiving goals for judging this instrument, they do not give the instrument an easy grade. This is another example showing that the new CLIA 2024 goals are challenging to even the most advanced instruments on the market.

Bonus Analysis: How are these assays judged when MU is compared to MAU, pU specifications?

While the original publication didn't intend to assess uncertainty, since imprecision was measured, the simple estimation of uncertainty can be made (take 2x the imprecision). The imprecision was measured over 6 months, which is a match for the stated requirements of estimating mu. We're not going to add in the bias estimate as another variance, most likely that would make all the uncertainties unacceptable. But perhaps these optimistic estimates are able to meet these MU specifications. Is MU more forgiving than the EFLM and CLIA goals?

Roche Cobas 6000 c501
Level % simple
Expanded Uncertainty
EFLM minimum MAU Panteghini's pU (PpU)
Final verdict
Albumin 1 4.92% 3.8%

1.25%(des)
1.88%(min

Fails MAU and PpU
2 3.92% Fails MAU and PpU
ALT 1

5.84%

15.1% 4.65%(des)
6.98%(min)
Passes min PpU,
passes min MAU 
2  5.34% Passes min PpU,
passes min MAU 
Alk Phos 1

5.5%

9.9% 2.65%(des)
3.98%(min)
Fails PpU,
passes min MAU 
2  4.78% Fails PpU,
passes min MAU 
Amylase 1 3.92% 9.9% 3.15% (des)
4.73% (min) 
Passes min PpU,
passes min MAU
2 4.46% Passes min PpU,
passes min MAU
AST 1 4.26% 14.4% 4.75% (des)
7.13% (min)
Passes des PpU,
passes min MAU
2 4.36% Passes des PpU,
passes min MAU
Bilirubin, Total 1 9.5%  30% 10.3% (des)
15.7% (min)
Passes des Ppu
passes min MAU
 2 10.06% Passes des Ppu
passes min MAU
Calcium 1 3.56%  2.7%  0.91% (des)
1.36% (min)
Fails MAU and PpU
 2 3.52% Fails MAU and PpU
Chloride  1 4.62%  1.7%  0.49% (des)
0.74% (min)
 Fails MAU and PpU
 2  4.56%  Fails MAU and PpU
Cholesterol 1 4.28% 7.95% 3% (des)
7% (min)
Passes min Ppu
passes min MAU
2 4.04% Passes min Ppu
passes min MAU
CK 1 4.28% 22.5%  7.25% (des)
10.9% (min)
Passes min MAU and Ppu
2  4.64% Passes min MAU and Ppu
Creatinine 1 6.1%  6.8%  2.2% (des)
3.3% (min)
Fails Ppu
passes min MAU
2  5.62% Fails Ppu
passes min MAU
GGT 1 3.72% 13.6%  4.45% (des)
6.68% (min)
Passes min MAU and Ppu
2 3.6% Passes min MAU and Ppu
Glucose 1 4.02%  7.5% 2% (des)
3% (min)
Fails Ppu
passes min MAU
2 3.7% Fails Ppu
passes min MAU
HDL Cholesterol 1 3.56% 8.7% 2.84% (des)
4.26% (min)
Passes min MAU and Ppu
2 3.78% Passes min MAU and Ppu
Iron 1 8.58% 31.0%  none stated Passes min MAU
2 6.72% Passes min MAU
LDH 1 9.64% 7.8%  2.6% (des)
3.9% (min)
Fails MAU and PpU
2  10.14% Fails MAU and PpU
Magnesium 1 4.3% 4.3% 1.44% (des)
2.16% (min)
Fails PpU
Passes min MAU
 
2 4.38% Fails MAU and PpU
Phosphorous 1 6.36% 11.7% none stated Passes MAU
2  5.22% Passes MAU
Potassium 1 4.58% 6.1% 1.96% (des)
2.94% (min) 
Fails PpU
Passes min MAU
2 4.24% Fails PpU
Passes min MAU
Sodium 1 4.0% 0.8% none stated Fails MAU
2 3.96% Fails MAU
Triglycerides 1 8.1% 30% 9.9% (des)
14.9% (min)
Passes min MAU and Ppu
2 7.82% Passes min MAU and Ppu
Uric acid 1 4.78% 12.5% 4.16% (des)
6.24% (min)
Passes min MAU
passes min Ppu
2 5.02% Passes min MAU
passes min Ppu
Urea Nitrogen 1 5.54% 20.9% 7.05% (des)
10.6% (min)
Passes min MAU
passes min Ppu
2 5.28% Passes min MAU
passes min Ppu

If we judge these assays by the various mu performance specifications set forth by EFLM and the work of Panteghini et al, there are far more victories than when these parameters are judged by the TEa benchmarks. This is actually one of the better results for Roche that we've seen recently. MAU and pU are far more forgiving than the use of TEa to set performance specifications.