Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. QC Applications
  4. Sigma Metric Analysis
  5. Sight OLO in UK, Multimode Analysis

Sigma Metric Analysis

Sight OLO in UK, Multimode Analysis

Continuing in our analysis of hematology systems, we look at the impact of new CLIA and EFLM goals on the assessment of a Sight OLO in the United Kingdom.

Sight OLO in United Kingdom, multimode analysis

July 2023
Sten Westgard, MS

See the other analyses in this series:

This Sight OLO data comes from St. Thomas's Hospital in London, United Kingdom:

Performance analysis of the compact haematology analyser Sight OLO, Rui Leite, Sue Woodcock, Sally BRady, Charlotte Bigsby, Kay Webber, Yvonne Daniel, Int J Lab Hematol. 2022. DOI:10.1111/ijlh.13956

"Reproducibility was assessed using the three-level OLO Control Kit. Each level of QC was tested on OLO in single rns, three times a day, for 20 days. Test runs were executed by a single operator."

"One hundred and thirteen venous whole blood samples collected in K2 ethylenediaminetetraacetic acid (EDTA) tubes were analysed with both OLO and Unicel DxH 800 analysers to assess the agreement of results between the two methods....To assess method comparability Passing-Bablock regression was applied to each measurand. Slope and intercept coordinates were assessed to verify result comparability."

The use of manufacturer controls are a weakness of this study, as they are less likely to provide an honest, independent assessment of the performance of the instrument. But on the hematology side, it's far less common to use a 3rd party control.

There are three levels that will be analyzed here. So for 13 parameters there will be 39 data points to plot on the Normalized Method Decision Chart. Note that while the EFLM biological variation database contains performance specifications for MCV, MCHC, and all the differential paraments, the CLIA 1992 and CLIA 2024 goals do not. Thus, the graphs for CLIA will only cover 5 parameters and 15 data points.

The TEa goals applied can be found on our Consolidated Hematology Performance Specifications page.

Parameter Level Slope Y-intercept Bias% CV%
Hemoglobin 75.18 1.05 -9.03 7.011 1.60
  134.53 1.05 -9.03 1.712 1.50
  173.03 1.05 -9.03 0.219 1.40
Hematocrit 22.73 1.08 -3.71 8.322 1.90
  41.13 1.08 -3.71 1.020 1.60
  54.75 1.08 -3.71 1.224 1.60
MCV 80.2 1.02 -2.16 0.693 1.20
  85.9 1.02 -2.16 0.515 0.80
  92.76 1.02 -2.16 0.329 0.90
MCH 26.53 0.96 0.96 0.381 1.10
  28.11 0.96 0.96 0.585 0.90
  29.31 0.96 0.96 0.725 1.20
MCHC 315.9 0.77 73.85 0.378 1.00
  327.27 0.77 73.85 0.435 0.90
  330.77 0.77 73.85 0.673 1.40
Platelets -Impedance 85.68 0.95 14.1 11.457 6.60
  188.83 0.95 14.1 2.467 5.50
  467.3 0.95 14.1 1.983 7.30
RBC 2.83 1.06 -0.308 4.883 2.00
  4.79 1.06 -0.308 0.430 1.60
  5.9 1.06 -0.308 0.780 0.70
WBC 2.95 1.01 -0.052 0.763 4.30
  8.73 1.01 -0.052 0.404 3.70
  23.65 1.01 -0.052 0.780 3.10
Lymphocytes 0.69 1.05 -0.016 2.681 10.50
  2.25 1.05 -0.016 4.289 5.80
  6.52 1.05 -0.016 4.755 5.10
Monocytes 0.22 0.99 -0.006 3.727 18.70
  0.64 0.99 -0.006 1.937 12.60
  1.35 0.99 -0.006 1.444 10.00
Eosinophils 0.31 1.1 0.017 15.484 16.80
  0.57 1.1 0.017 12.982 14.70
  1.16 1.1 0.017 11.466 9.30
Basophils 0.31 0.71 -0.003 29.968 15.00
  0.65 0.71 -0.003 29.462 12.50
  1.4 0.71 -0.003 29.214 9.90
Neutrophils 1.42 0.98 -0.024 3.690 6.50
  4.61 0.98 -0.024 2.521 4.20
  13.21 0.98 -0.024 2.182 3.80

Sigma-metrics according to EFLM-derived DESIRABLE performance specifications

 2023 OLO EFLM des NMEDX 1

The EFLM desirable specifications used to be the de facto global standard, but have fallen out of favor due to their toughness.

The Sight OLO has over eighty percent of the performance below 3 Sigma. The main driving factor is imprecision that is too high relative to the performance specifications.

Not surprising then, that EFLM recommended lowering the standards.

Sigma-metrics according to EuBIVAS-derived MINIMUM performance specifications

2023 OLO EFLM min NMEDX2 

There isn't that much of an improvement, with still two-thirds of performance below  3 Sigma, and less than one-third of that performance below 3 Sigma.

Here's one of the most interesting new aspects of CLIA's new 2024 goals. Are they more or less demanding than EFLM goals?

Sigma-metrics according to CLIA 2024 performance specifications

2023 OLO CLIA 2024 NMEDX4 

The CLIA 2024 goals are a bit easier on the OLO, with only 40% below 3 Sigma.

Sigma-metrics according to CLIA 1992 performance specifications

2023 OLO CLIA 1992 NMEDX3

If we apply the CLIA 1992 goals, it's even more favorable. The most number of 6 Sigma analytes occur when these performance specifications are used. But no set of goals gives the OLO a clean bill of health.

Conclusion

Even with possibly the most favorable assessment (from their own controls), the Sight OLO has a lot of difficulty hitting any set of targets. With the most recent goals (CLIA 2024) that have been imposed, it's possible to explain away the failures by noting that the tighter goals are an over-correction, an attempt to make up for decades of stagnation with the CLIA PT goals. It's possible to argue that a comparison of the OLO against the DxH may not be the best choice, if OLO is truly a different methodology, but the main problems dragging down the Sigma metrics are imprecision, not bias. It's also possible to look at the EFLM biological goals (either minimum or desirable) and note that hardly any platform on the market has been able to successfully hit them with consistency. Indeed, there's growing evidence that the current set of goals for hematology, from both CLIA and EFLM, are not in synch with today's analyzers. But if the OLO can't hit the goals from 30 years ago, that's not an encouraging sign for an instrument introduced in the 2020's.