Sigma Metric Analysis
Multimode analysis of a Roche Cobas c501 chemistry analyzer in Turkey
Now that EFLM has gone to minimum, and CLIA has issued tighter goals for 2024, it's time to take another look at a major instrument system. Ricos 2014, EFLM minimum and desirable, CLIA 1992 and 2024. How does a c501 stack up?
Multimode analysis of an Roche Cobas c501 chemistry analyzer in Turkey
August 2022
Sten Westgard, MS
With goals a-changing, it's time to re-examine the latest performance from instruments like the c501. This analysis will use the latest desirable and minimum performance specifications from the EFLM, as well as Ricos 2014, and CLIA 1992 and CLIA 2024 goals.
See the other analyses in this series:
- Multimode Analysis of a DxC 700
- Multimode Analysis of an Abbott Alinity
- Multimode Analysis of a Siemens Atellica
- Multimode Analysis of a Siemens Attelica in Romania
- Multimode Analysis of a Siemens Attelica in Spain
- Multimode Analysis of a Siemens ADVIA 2120i
- Multimode Analysis of a Roche c501 in Turkey
- Multimode Analysis of a MicroLab RX-50 in India
- Multimode Analysis of Roche cobas 6000 immunoassays in Turkey
- Multimode Analysis of Sysmex XN 350 in India
- Multimode Analysis of Mindray 7500 in China
The Roche Cobas c501 data comes from Karacabey State Hospital in Bursa, Turkey:
The study was conducted with data from March through August of 2020. The controls used were the Roche Diagnostics manufacturer-supplied controls, which is a flaw in the study, and thus these Sigma metrics may be inappropriately optimistic and high. "Bias was calculated using the monthly EQA reports (Oneworld System Accuracy, Canada) from March to August."
The raw data of imprecision and bias
There are 12 test methods and 24 control levels for our analysis.
| TEST | % Bias | CV |
| Albumin | 1.32 | 2.33 |
| Albumin | 1.32 | 1.86 |
| Alk Phos | 5.0 | 3.26 |
| Alk Phos | 5.0 | 3.76 |
| ALT | 0.1 | 2.46 |
| ALT | 0.1 | 2.57 |
| AST | 0.7 | 1.6 |
| AST | 0.7 | 1.6 |
| Chloride | 0.74 | 2.1 |
| Chloride | 0.74 | 1.6 |
| Cholesterol | 2.37 | 1.9 |
| Cholesterol | 2.37 | 1.4 |
| Creatinine | 1.3 | 2.6 |
| Creatinine | 1.3 | 2.3 |
| Glucose | 1.3 | 1.6 |
| Glucose | 1.3 | 1.7 |
| LDH | 1.61 | 1.42 |
| LDH | 1.61 | 1.37 |
| Protein, Total | 0.56 | 1.35 |
| Protein, Total | 0.56 | 1.22 |
| Triglycerides | 2.09 | 2.47 |
| Triglycerides | 2.09 | 1.57 |
| Urea Nitrogen | 0.25 | 1.53 |
| Urea Nitrogen | 0.25 | 1.65 |
Our analysis will now use the performance specifications from latest (2022) EFLM, both minimum and desirable as derived from the EuBIVAS biological variation database, as well as the desirable goals from the Ricos 2014 performance specifications; and finally the 1992 and newly announced 2024 CLIA goals.
Sigma-metrics according to EuBIVAS-derived DESIRABLE performance specifications
The EFLM desirable specifications used to be the global standard, but have fallen out of favor due to their toughness and demanding nature. The EFLM database used to calculate and display the desirable performance specifications. If those goals are used, more than 40% of the assay performance is located at 3 Sigma and below, while only one third of performance is at 6 Sigma. Given that these are optimistic controls, that's not a good start.
What happens if we lower the EFLM standards?.
Sigma-metrics according to EuBIVAS-derived MINIMUM performance specifications
There is a big improvement, but there are still assays that aren't hitting the mark. The number of levels performing at 6 Sigma rises to over 45%, and the number of assays performing below 3 Sigma drops significantly to around 17%. Even the minimum specifications for chloride and albumin are still quite demanding.
Next, let's look at the "original" version of the biological variation database. The last version of the database was updated in 2014 by Ricos et al, so we refer to these performance specifications as Ricos 2014 goals.
Sigma-metrics according to Ricos 2014 desirable performance specifications
The original Ricos goals harder on this cobas c501 than the EFLM minimum goals. Almost 40% of the values are below 3 Sigma, but the values above 6 Sigma have risen to almost 42%.
What happens if we move away from biologically-derived goals toward the CLIA goals, including those newly announced to take effect in 2024?
Sigma-metrics according to CLIA 2024 performance specifications
Interesting to see here that CLIA 2024 is a bit tougher and a bit easier than Ricos 2014 goals. Less than 30% of the performance benchmarks in at 6 Sigma, but less than 10% are considered below 3 Sigma.
Finally, let's grade these assays using the oldest set of goals, the original CLIA specifications from 1992
Sigma-metrics according to CLIA 1992 performance specifications
If you judge the c501with the oldest specifications, also the most popular ones in the USA, you finally get 50% of the performance at 6 Sigma. And now only about 8% of the assays below 3 Sigma.
Conclusion
If we're honest, this instrument doesn't look spectacular by any standard. The EFLM desirable goals are still the most difficult to achieve. Applying the CLIA goals from provide the kindest assessment. The new 2024 CLIA goals are pretty demanding.
The optimal set of goals is not just from one source. We have to recognize that goals from outside the EFLM database are necessary to provide a reasonable assessment of analytical quality.