As discussed elsewhere (1), quality performance of any analytical process in a laboratory can be described in terms of its sigma metric. Calculation of sigma performance is quite simple:
Sigma = (TEa – Bias) / CV
The harder part is making sure that data used in this calculation really reflects current analytical performance. In my region, we turned to our on-analyzer quality control databases for estimates of imprecision (CV). This data reflects actual analytical imprecision, if and only if, data integrity has been maintained. Data integrity can be assured only when procedures are in place and rigorously practiced to exclude erroneous quality control results due to procedural blunders and statistical outliers. This does not mean excluding out-of-control results; no statistically-valid data should be edited out.
For estimates of test bias, our region makes use of Randox Interlaboratory Quality Assessment Scheme (RIAQS). This external quality assessment program, run out of Ireland, provides biweekly (every 2 weeks) challenges for a broad range of automated chemistry tests. There are two main attributes that make this program of particular value for assessing test bias. First, challenges are blind and second, they are at varying analyte concentrations. These two attributes, combined with its biweekly frequency and relatively large user base, make this database reflective, as close as possible, of “real-world” testing bias. Bias is calculated for each test application based on the average (running mean) bias of the last 10 values obtained over the preceding five month period, updated every two weeks.
Sigma quality management forces one to think in terms of quality needs, not just performance capabilities and this is good since it focuses attention on what really matters. The next step is setting quality goals. This in itself can be a real eye-opener and a little daunting. There are multiple sources for quality goals but the proviso, of course, is objectivity in selecting these goals in deference to analytic capabilities. We selected quality goals for total error (TEa) based on either biological variation or proficiency testing target data accessible through this website (2, 3), fully recognizing that at least some of these goals may need customizing to specific clinical requirements as our experience with this approach develops. Selecting test specific goals, as uncomfortable or imperfect a process as this may be, it reinforces the fact that good clinical practice is implicitly linked to quality of laboratory testing.
Once current and reliable data for imprecision and bias has been obtained and quality goals set, sigma quality performance is then calculated for each test application. These sigma metrics are date documented and used to benchmark existing sigma performance for inter-instrument and inter-laboratory comparisons within our region and for future historical reviews to assess performance changes.
To illustrate this approach, the sigma metrics for 60 automated test applications over three analytic platforms at one laboratory (site A) in our region indicates that twenty eight (46%) of these applications fall short of meeting six sigma quality performance. Of these, twelve fail to meet minimum sigma quality performance with metrics less than three and another four just meet minimal acceptable performance with sigma metrics between three and four. At another laboratory (site B) in our regional group (seven sites in total), the original data collected indicates more than half (57%) of the automated test applications do not achieve six sigma quality performance.
It must be kept in mind that six sigma quality management is not only a tool for defining process performance but also a method for moving a process from its present error rate to a very low error rate. In order to achieve quality improvement of automated analytic tests where needed, it is important to understand the test-specific reasons for their quality shortcomings, be it either excessive imprecision, excessive bias or both. To facilitate this, performance data is assessed by calculating the quality goal index (QGI) by the following expression:
Derivation of this expression is based on mathematical reduction of the ratio (Bias/Accuracy Goal) / (CV/Precision Goal). The QGI ratio represents the relative extent to which both bias and precision meet their respective quality goals. The quality goals chosen for use in this expression are 1.5*TEa/6 for bias and TEa/6 for precision based on their widespread use in six sigma methodology literature. For those who feel that inclusion of a1.5 sigma shift in calculating the bias goal is not justified, then both quality goals can be calculated as TEa/6. The corresponding expression would then reduce to QGI = Bias/CV.
Examination of the unreduced equation gives better insight into how this data-evaluating tool works. For example, QGI is higher for a test application when bias exceeds its accuracy goal and imprecision meets its precision goal and QGI is lower when bias meets its accuracy goal and imprecision exceeds its precision goal. The criteria we use for interpreting QGI when test applications fall short of six sigma quality is as follows:
| QGI |
Problem |
| <0.8 |
Imprecision |
| 0.8 - 1.2 |
Imprecision & Inaccuracy |
| >1.2 |
Inaccuracy |
This quantifiable approach to problem assessment is computer programmable in Microsoft Excel using Visual Basic for Applications (VBA). Macro programming simplifies its application which is particularly helpful when dealing with larger numbers of tests at multiple lab sites. As an example of this approach, QGI indicates that of the 29 test applications that failed to meet six sigma quality performance at lab site A in our region, the main problem is excessive imprecision in 52%, with excessive inaccuracy occurring in 17%. At lab site B, the main problem is excessive inaccuracy in 73% with excessive imprecision being the problem in 20%. QGI provides easy insight into where improvement is required and can serve as a tool for focusing efforts on sigma quality improvement of automated analytic tests.