QP-10: Automated Chemistry Applications
|
|
Quality planning applications for clinical chemistry tests are perhaps the easiest to get started with because criteria for acceptable performance have been defined by CLIA for many of the routine chemistry tests and estimates of method imprecision and inaccuracy are generally available in the laboratory. It is also helpful that there are documented applications available in the literature. Finally, the high level of automation in chemistry analyzers has improved the performance of chemistry tests, particularly imprecision, thus careful selection of QC procedures can often reduce operating costs! Saving time and money should provide strong motivation for initiating your quality planning applications in the clinical chemistry area.
CLIA defines criteria for acceptable performance for 27 different tests as part of the proficiency testing guidelines. These criteria are in the form of an allowable total error and are presented as a statement of a target value plus and minus a certain amount. This amount is given in three different forms:
- A percentage, e.g., the acceptable performance for cholesterol is stated as "target value +/- 10%.
- A concentration, e.g., target value +/- 0.5 mmol/L for potassium.
- A interval based on the observed SD of the survey group, e.g., target value +/- 3SD for blood gas pO2. Laboratories can calculate these quality requirements on the basis of group SDs observed in peer review QC programs or previous proficiency testing events.
For some tests, it is necessary to define the medical decision level of interest to interpret the quality requirement, e.g., for glucose, acceptable performance is stated to be target value +/- 6 mg/dL or 10%, whichever is greater.
- At a medical decision level of 50 mg/dL, the quality requirement would be 6 mg/dL or 12%.
- At a medical decision level of 120 mg/dL, the quality requirement would be 10% or 12 mg/dL.
Most clinical chemistry methods are classified as moderately complex via CLIA-88 guidelines, which require that the imprecision and inaccuracy of the method be validated through replication and comparison of methods experiments, respectively. Therefore, initial estimates of method imprecision and inaccuracy should be available in all laboratories. Ongoing estimates of method imprecision can be obtained from monthly or cumulative QC data and periodic estimates of method inaccuracy can be made from the difference between the laboratory mean and the group mean in inter-laboratory peer comparison programs, external quality assessment programs, and/or proficiency testing surveys.
Some initial examples were presented in lesson 8 to illustrate the selection of QC procedures for glucose, cholesterol and calcium. Real-world applications for these tests have been described in the literature for a multitest chemistry analyzer [1] , where the selection of QC procedures gave the following results:
- For 14 of 18 tests, a 1 3.5s rule with N=2 was selected for sodium, potassium, glucose, urea nitrogen, creatinine, phosphorus, uric acid, cholesterol, total protein, total bilirubin, GGT, ALP, AST, and LD;
- A single-rule, 12.5s with N=2, was selected for albumin;
- A multirule QC procedure was found useful for Cl and CO2 tests;
- A special QC strategy was implemented for calcium by averaging the results of duplicate measurements.
This study was published in 1990 and therefore predates the 1992 publication of the CLIA-88 criteria for acceptable performance in proficiency testing surveys. Thus, the quality requirements were not exactly the same as those defined by CLIA. Note also that this study makes use of a different quality-planning tool - the critical-error graph - which preceded the OPSpecs chart that was introduced in 1991 [2,3].
Related to this study, we also documented the cost-savings that can be expected from optimized, individualized QC designs [4]. We had initially used a multirule QC procedure for all the tests on this analyzer because that's what was accepted practice on the earlier generation analyzer that was being replaced. Old QC practices are often carried over to new generation analyzers because we assume the new system will have similar performance characteristics. However, improvements in precision and stability may permit more cost-effective QC procedures to be implemented, as demonstrated in this application. The initial changes in QC procedures provided a cost-savings of about $1,450 per month or $17,400 per year, which provides a total savings of $87,000 over the expected 5-year lifetime of the instrument. Later changes in the length of the analytical run increased the savings to $3,000 per month or $36,000 per year, which provides a total savings of $180,000 over the 5-year lifetime of the instrument.
The method has an observed standard deviation of 2.7 mg/dL or 1.35% at a decision level of 200 mg/dL. Bias is assumed to be 0.0% because method validation studies show small systematic differences between comparative systems within the laboratory and ongoing comparison studies are used to maintain near-zero biases. Two control materials are to be analyzed each run to fulfill the CLIA requirement. TEa is defined as 20 mg/dL which is 10% at a medical decision level of 200 mg/dL (note the units used in the paper are actually mg/L rather than mg/dL, therefore the values are all higher by a factor of 10).
- The normalized operating point would have an x-coordinate of 13.5% [(1.35/10.0)100]and a y-coordinate of 0.0%.
- When plotted on the normalized OPSpecs chart for 90% AQA and N=2, the validity of the 13.5s rule can be seen from the accompanying figure. Any other QC procedure on the chart could also be used, but the advantage of using 3.5s control limits is that the false rejections would be essentially zero.
- The Total QC strategy can depend on statistical QC to detect problems because at least 90% detection is expected.
The observed standard deviation is 1.2 mg/dL or 1.1% at a decision level of 110 mg/dL. Bias is assumed to be zero. TEa was defined as 8.0 mg/dL, which would be 7.2% at a decision level of 110 mg/dL. This is somewhat more demanding than the CLIA requirement of 6 mg/dL or 10% (whichever is greater), which would be 10% at a decision level of 110 mg/dL. Two control materials are to be analyzed.
- The normalized operating point would have an x-coordinate of 15.3% [(1.1/7.2)100] and a y-coordinate of 0.0%.
- When plotted on the normalized OPSpecs chart for 90% AQA and N=2, the results are the same as for the cholesterol example above - a 13.5s control rule will provide adequate error detection with essentially zero false rejections (see accompanying figure).
- The Total QC strategy is to depend on statistical QC to detect problems.
The method SD was 1.04 mmol/L which is 1.04% at a decision level of 100 mmol/L. Bias is assumed to be zero. The CLIA requirement of 4.0 mmol/L corresponds to 4.0% at a decision level of 100 mmol/L. Two control materials are to be analyzed.
- The normalized operating point has an x-coordinate of 26% [(1.04/4.0)100] and a y-coordinate of 0%.
- When plotted on a normalized OPSpecs chart for N=2 and 90%AQA, there is no appropriate QC procedure.
- When plotted on the chart for N=2 and 50%AQA, either a 12.5s single-rule or a multirule with 13s/22s/R4s will provide at least 50% error detection (see accompanying figure). Addition of a 41s rule to the multirule procedure (13s/22s/R4s/41s) would be useful to maximize error detection across two runs.
- The Total QC strategy should provide more emphasis on preventive maintenance and specific instrument checks when only moderate error detection can be achieved within the first run. For this chloride test, frequent maintenance and cleaning of the chloride electrode is needed to prevent problems from occurring.
The observed SD was 0.168 mg/dL, which is 1.68% at a decision level of 10.0 mg/dL. TEa was defined as 0.5 mg/dL, which would be 5% at a medical decision level of 10.0 mg/dL (which is much more demanding than the CLIA requirement of 1.0 mg/dL or 10% at a decision level of 10.0 mg/dL).
- The normalized operating point would have an x-coordinate of 33.6% [(1.68/5.0)100] and a y-coordinate of 0.0%.
- When plotted on the N=2 with 90% AQA or N=2 with 50%AQA, no QC procedure that can be selected. A maximum QC procedure could be selected, possibly a multirule procedure that includes a 41s rule to look-back at the control data in the previous run. However, the error detection for a single rule will be low to moderate, which is not very satisfactory.
- The Total QC strategy should emphasize improving method performance and preventing problems from occurring. If duplicate measurements were made to reduce the imprecision of the method (which would change the y-coordinate to 33.6%/1.414 or 23.8%), then a multirule solution could be found on the N=2 50%AQA chart (gives essentially the same operating point as shown on the previous N=2 and 50%AQA chart for chloride). Further improvement to give an x-coordinate of 20% is needed to find a solution on the N=2 90%AQA chart, as shown in the accompanying figure. This strategy of improving method performance by making replicate measurements was actually implemented in the laboratory by installing two calcium tests on the analyzer. Note, however, if the CLIA quality requirement of 10% were used, then the original operating point would have had a normalized x-coordinate of 16.8%, and the same QC procedure could have been selected as shown in the cholesterol and glucose examples above.
There are lessons to be learned from quality-planning applications in all areas of the laboratory. Routine chemistry applications may be simpler than others, but are still a source of important insights and advice. You're more likely to solve complicated quality-planning problems if you have a good grasp of the solutions to simpler applications.
- Manage quality in a quantitative manner to improve productivity and reduce costs. Cost-savings can be demonstrated when QC procedures are designed or selected on the basis of the quality required for the test and the imprecision and inaccuracy observed for the method. These savings will be particularly significant if your current laboratory QC practice is to utilize 2 SD control limits, which cause a high level of false rejections - about 10% when N is 2, 14% when N is 3, and 18% when N is 4. If multirule QC procedures are being used with 4th and 5th generation automation analyzers, it may be possible to simplify QC, reduce N, improve productivity, and reduce costs, as demonstrated for the example applications here [which have been documented in the scientific literature, references 1 and 4].
- Begin with proficiency testing criteria as the quality requirements. For most routine chemistry tests, proficiency testing criteria will have been defined in the form of allowable total errors. Although these requirements represent the minimum level of quality that must be achieved by the laboratory, their availability will allow you to get started right away. Begin your quality-planning applications using these criteria, then repeat the QC planning process when you obtain better information about the quality requirements for your patient applications.
- Use the best estimates of method performance available at the time. Laboratories always have current QC data that can be used to estimate the SD of a method, but may not have current comparison of methods data to estimate the bias a method. There is nothing wrong with making an initial assumption that bias is zero in order to get started with your quality-planning applications. Use the best estimates of method performance that are available at the moment. You can always repeat the planning process when you have better information. Information about bias may be obtained from monthly peer-comparison programs or from periodic proficiency testing surveys.
- Individualize the QC procedure for each test on a multitest system. Different tests have different quality requirements and different analytical performance, therefore, there will be different QC procedures that are appropriate for different tests on a multitest system.
- Keep N constant and changes the rules to adjust QC performance. It will often be necessary to keep the number of control measurements the same for each test on a multitest analyzer and then vary the control rules to optimize QC performance for different tests. Usually only 3 to 5 different QC designs will be needed to handle all the tests, as illustrated by the chemistry multitest application where 14 of the 18 tests could be controlled using 3.5s limits, 1 with 2.5s limits, 2 others with multirule procedures, and 1 required a special QC procedure.
- Adjust the TQC strategy to deal with tests with poor performance. It is not unusual to encounter a few tests for which error detection is less than ideal, therefore efforts are needed to improve analytical performance and/or to prevent problems from occurring. Sources of errors and causes of instability need to be identified and preventive maintenance schedules adjusted to prevent problems from occurring with these tests.
- Implement individualized QC designs via computer. With automated systems, it is generally necessary to computerize QC to keep up with the production of test results. Flexible QC software is needed, particularly the ability to select different control rules for different tests that are performed on the same analytical system. Test-specific QC is an important feature in computer software to help you manage and operate your automated systems in the most cost-effective way.
