Final5
CLIA Rule. Part III: QC - Quality or Compliance?
Final5
CLIA Rule. Part III: The main focus of the CLIA Final Rule is quality systems for non-waived testing. This section of the rules has been called "quality assurance" in earlier versions. Most of the rules still deal with "control procedures" and the ones that are most difficult to understand are still about quality control.
After waiting for ten years to get the Final Rule, laboratories still need to wait to find out how the final words will be interpreted. The problem is that the Final Rule introduces new terminology and phrases that need further explanation to understand their intended meanings. The final clarifications are expected to appear in the State Operations Manual within the next 6 to 12 months.
Meanwhile, what should you do? Actually, you don't have to do anything until you see those clarifications. Laboratories will have at least one year to come into compliance with the Final Rule, according to Judy Yost, Director of the Division of Laboratory Services, Centers for Medicare and Medicaid Services [Clin Lab News, March 2003, "Moving to One Quality Control Standard for Non-Waived Testing"]. On the other hand, you can be proactive and determine what should be done to assure and improve the quality of testing services in your laboratory, then assess how those plans and actions will fit with the final rules and regulations.
In short, you have a choice between doing what's necessary to be in compliance with the quality regulations and doing what's right to assure the quality of your testing processes. Both are allowable under the CLIA Final Rule. It seems obvious that it would be better for our patients if we focused on assuring quality, reducing medical errors, and improving patient safety, but it is equally obvious that it is both easier and cheaper to settle for compliance.
Section 493.1250 describes the basic requirements for analytic systems, which include the following:
- Monitor and evaluate the overall quality of the analytical systems and correct identified problems;
- Have a procedure manual;
- Perform testing following the manufacturer's instructions;
- Define criteria for reagent storage, etc.
- Verify or establish performance specifications;
- Determine calibration and control procedures;
- Perform maintenance and function checks;
- Verify calibration every 6 months (if applicable);
- Document remedial actions;
- Maintain QC records for 2 years.
The most difficult of these requirements is the one to "monitor and evaluate the overall quality of the analytical systems and correct identified problems." To do so requires the laboratory to determine appropriate control procedures, implement them properly, and respond to out-of-control situations by identifying problems and eliminating their sources. The CLIA Final Rule describes the laboratory's responsibilities for doing so and also identifies minimum requirements to be in compliance with the regulation.
Section 493.1256 is the Standard for Control Procedures and provides the details for control procedures. Parts a through c2 describe a laboratory's responsibilities for assuring the quality of patient test results.
(a) "The laboratory is responsible for having control procedures that monitor the accuracy and precision of the complete analytical process."
"Complete analytical process" - a new phrase that appears in the Final Rule - implies several steps that include sample introduction and manipulation, reagent additions, generation of the analytical reaction that is measured, measurement of the analyte via some type of sensor, and conversion of the sensor response to an analytical result. The obvious approach for monitoring precision and accuracy is to use liquid controls that go through the complete analytical process just like a real patient sample. Routine calculation of the internal QC data will provide ongoing estimates of method precision (SD, CV) and indications of any systematic changes or drifts (stability of mean or changes over time).
Given that liquid controls are needed to monitor the complete analytical process, there should be concern that Electronic QC (EQC) is not adequate. Here's where the State Inspectors Manual will become important. EQC had been given a temporary allowance in the past. When the new version is available, I would expect to see some guidance about using liquid controls along with EQC.
(b) "The laboratory must establish the number, type, and frequency of testing control materials using, if applicable, the performance specifications verified or established by the laboratory…"
Here's where the fun begins. It is generally accepted as good laboratory practice to analyze two or three different control materials whose mean values are close to concentrations important for the interpretation of the test results. The type of material will usually depend on what is available commercially. Whether assayed or unassayed, a laboratory should establish its own mean and SD (or CV) from measurements carried out under routine operating conditions. The act of establishing your own means and SDs brings in the performance specifications observed in your own laboratory.
The total number of control measurements needed for QC purposes depends on the quality required for the test, the precision and accuracy observed for the method, and the statistical control rules that are implemented. Here's where "QC planning" is needed! Quantitative methodology and quality-planning tools are available on this website:
- "Six Sigma Quality Design and Control Processes", at http://www.westgard.com/lesson67.htm
- "Normalized OPSpecs Charts", at http://www.westgard.com/lesson9.htm
- "Normalized Operating Point Calculator" at http://www.westgard.com/normcalc.htm
The frequency of testing control materials should be related to the stability of the process and its susceptibility to problems. It makes sense to analyze control materials more frequently if a method is unstable. One practical indication of stability is the frequency of calibration. Whenever a method is calibrated, it also makes sense to run controls to check performance. Susceptibility is often related to the level of automation of the analytical procedure. Manual methods are usually much more susceptible to problems than automated methods; early generations of automation are more susceptible to problems than later generations of automation. However, there is no quantitative methodology that tells you the frequency for running controls. It is a judgment that should be based on manufacturer's recommendations for stability and your expectation of susceptibility, based on your experience with the methods in your laboratory, the skills of your analysts, and the factors and variables that change in your laboratory.
(c 1) "The control procedures must detect immediate errors that occur due to test system failure, adverse environmental conditions, and operator performance…"
Here's where you have some new and serious responsibility! The intent of these regulations is not just for laboratories to run controls, but to be able to "detect immediate errors." This phrase "detect immediate errors" has never appeared in earlier versions of CLIA, therefore it will need some clarification and explanation. Until there is some official clarification about this, here's what I think it should mean:
- Error does not mean all errors, but those that are medically important. All tests are in error to some degree because all methods have some imprecision or random error. It is important to be able to detect errors of a size that would cause test results to be misinterpreted and therefore cause patients to be mis-diagnosed or mis-treated. You can calculate the size of medically important errors if you define the quality required for a test and assess the precision and accuracy available from a method.
- Immediate is not a type of error! It means that laboratories should detect important errors immediately. To do so requires that the controls be analyzed at an appropriate frequency, the proper number of control measurements be collected, and the appropriate control rules be used to interpret the data. Patient data algorithms can also be utilized to cross check between tests and to measure the "average of normals" (Aon) to monitor stability.
The most obvious implication is that analyzing controls once a day or once a shift may not be good practice, even though CLIA may specify those minimums for compliance. If you have an instrument that is reporting data continuously throughout an eight hour shift, your ability to "detect immediate errors" depends on analyzing controls throughout that shift, not just at the beginning or at the end.
An appropriate strategy for dealing with this requirement is to utilize what we call "multistage QC". Begin with a "startup design" that provides high detection of medically important errors, then switch to a "monitor design" for periodic verification during the on-going run and reporting of results. A third stage can be employed using patient data, such as Aon algorithms to monitor stability and determine when controls should be analyzed again to validate method performance.
A useful analogy is the QC done on an airplane. There certainly is a "startup" design that is employed while the plane is on the ground. It's very detailed and thorough and is aimed at catching any little problem before you're up and flying. Once in the air, there is a "monitor design" that follows critical factors and variables to ensure the flight remains safe. We should and can do that in the laboratory!
- See "Sage advice about new approaches to quality management" at http://www.westgard.com/essay30.htm
(c 2) "Monitor over time the accuracy and precision of test performance that may be influenced by changes in test system performance, environmental conditions, and variance in operator performance."
The emphasis here is "monitor over time" in contrast to the "detect immediate errors" in the previous statement. The obvious strategy here is to participate in a peer-comparison program that gives you information on the performance of your method over time, as well as the comparative performance of other laboratories using the same control materials. You utilize the same control data being collected to "detect immediate errors", but analyze that data for monthly periods.
Changes in the precision of your methods will show up in the monthly and cumulative summaries of your method SDs and CVs. Any changes with the accuracy of your methods will show up in the monthly and cumulative comparison of your mean values with those of the peer group. If there are any problems with the control materials themselves, they will show up in the peer comparison data.
The general requirements in parts a-c provide good guidance for assuring the quality of laboratory test results, but CLIA continues to provide minimum requirements for compliance in parts d-h. These minimum requirements each represent a good practice, but the totality of the minimum requirements does not necessarily provide adequate control of quality.
Here are some of the minimum requirements:
- Run two controls for quantitative tests each day when patient specimens are assayed or examined (note that this general requirement may be modified by specific requirements in specialty areas as described in sections 493.1261 through 493.1278, e.g., blood gas analysis requires 1 control every 8 hours, hematology manual cell counts require 1 control every 8 hours, non-manual coagulation requires 2 levels of control every 8 hours, but manual coagulation tests require 2 levels of control each time specimens are analyzed, etc.);
- Test controls after a complete change of reagents, preventive maintenance, or replacement of a critical part or component;
- Rotate controls among all operators performing the test;
- Test control materials the same as patient materials;
- Establish or verify criteria for acceptability for all control materials;
- Determine the statistical parameters (mean and SD) over time through concurrent testing for all unassayed control materials;
- Make sure controls meet criteria before reporting patient test results;
- Document all control procedures performed;
I agree that these minimum requirements are absolutely essential, but I disagree that they are sufficient. Compliance does not equate to quality! Quality relates to excellence, which means doing more than the minimums needed to get by!
The danger in the regulations is that laboratories can be in compliance with the minimum requirements, yet not assure the quality of patient test results. For example, compliance with these minimums does not assure that the laboratory will utilize the proper number of control materials and the proper frequency of testing to detect immediate errors. Instead, the laboratory can just analyze two control materials each day for any method, even if the method is running continuously for twenty-four hours and producing thousands of patient test results.
In the past, the CLIA minimums became maximums for many laboratories because of the compliance mentality of managers and administrators. Under the new Final Rule, we have an opportunity to put quality back into quality control. Concern for patient safety should make quality a higher priority than compliance!
