QP-7: Formulating a Total Quality
Control Strategy
|
 |
Optimal management of analytical quality depends on individualizing
the QC procedures for each test and method in the laboratory.
The quality-planning process provides the methodology for (a)
setting method performance specifications that are appropriate
for the analytical or clinical quality required for a test and
(b) selecting statistical QC procedures appropriate for the actual
imprecision and inaccuracy observed for methods in routine operation
in a laboratory. Ideally, the QC procedure should provide at least
a 90% chance of rejecting an analytical run having medically important
errors. At the same time, the QC procedure should have less than
a 5% chance of falsely rejecting a run that contains only the
random errors due to the inherent (stable) imprecision of the
method. And, for practicality and low cost, the QC procedure should
require only 2 to 6 control measurements per run - the lower the
better.
Review of CLIA regulations
US government regulations (CLIA-88) define a set of standards
for quality control that include method performance specifications,
statistical quality control, preventive maintenance, instrument
function checks, and method performance tests [1,2]. These CLIA
rules may be viewed as separate requirements for individual components
of Quality Control (QC), or as a requirement for developing a
Total Quality Control (TQC) strategy that incorporates these components
in a manner appropriate for controlling individual testing processes.
The latter view would seem to be more desirable for assuring the
quality of laboratory testing because of the need to individualize
the QC designs for the many different analytical methods for performing
those tests.
The responsibility for establishing a TQC strategy initially
belongs to the manufacturers of medical testing systems, devices,
or kits. When a manufacturer's QC instructions have by cleared
by FDA as meeting CLIA requirements for quality control, the CLIA
rules require that a laboratory does the following:
"demonstrate that, prior to reporting patient test results,
it can obtain the performance specifications for accuracy, precision,
and reportable range of patient test results, comparable to those
established by the manufacturer [2, p. 5230, par. 493.1213(b)(1)],
"perform maintenance as defined by the manufacturer and
with at least the frequency specified by the manufacturer,"
[2, p. 5231, par. 493.1215(a)(i)],
"perform function checks as defined by the manufacturer
and with at least the frequency specified by the manufacturer"
[2, p. 5231, par. 493.1215(b)(i)],
"follow the manufacturer's instructions for calibration
and calibration verification procedures using calibration materials
specified by the manufacturer" [2, p. 5231, par. 493.1217(a)],
and
"follow the manufacturer's instructions for control procedures"
[2, p. 5232, par. 493.1218(a)].
For a test method whose QC instructions have not been cleared
by the FDA, the laboratory itself assumes responsibility for formulating
an appropriate strategy for quality control that includes these
same components. Note that as of the year 2000, a QC clearance
process has NOT yet been implemented in accordance with the CLIA
regulations, thus the laboratory is primarily responsible for
selecting appropriate QC procedures and for implementing appropriate
TQC strategies.
General TQC Guidelines
The starting point for formulating a TQC strategy is the quality-planning
process and the error detection available from the selected statistical
QC procedure. Testing processes will be classified into one of
three categories: high error detection when a QC procedure can
be selected from an OPSpecs chart with 90% AQA; moderate error
detection when a QC procedure is selected from an OPSpecs chart
with 50% AQA; low error detection when 50% AQA is not obtainable,
in which case a maximum QC procedure should be defined as the
default selection. The recommendations for maximum QC selections
here are to use a multirule procedure such as 13s/22s/R4s/41s/8x
with the maximum of N=4 (for 2 control materials) and 13s/2of32s/R4s/31s/6x
with N=6 (for 3 control materials).
The general TQC strategies for
these three classes are shown in the accompanying table, where
SQC refers to statistical QC, Other QC includes preventive maintenance
(PM), instrument function checks (FC), performance validation
tests (PV), and patient data quality control (PD). QI means quality
improvement and refers primarily to improving the precision, accuracy,
and stability of the measurement procedure.
- HI-Ped strategy. When SQC is able to provide high
error detection, then the TQC strategy is to depend primarily
on SQC and perform the minimum requirements for other QC components.
- MOD-Ped strategy. When SQC provides moderate error
detection, the TQC strategy is to balance the emphasis on SQC,
Other QC, and QI.
- LOW-Ped strategy. When SQC provides low error detection,
the TQC strategy cannot rely on SQC, but must emphasize Other
QC and QI.
Step-by-step guidelines
The flowchart shows a more detailed
process for developing a TQC strategy for an individual method.
When SQC provides high error detection (90% AQA), the emphasis
is on minimizing the costs of statistical and non-statistical
QC. When SQC provides moderate error detection (at least 50% AQA),
then the emphasis is on maximizing statistical and non-statistical
components, as well as improving measurement performance. If SQC
provide low error detection (less than 50% AQA), the efforts also
include optimizing QC for process stability, improving the skills
of the analysts, and adding patient data QC. In all cases, the
final step is to document the QC system.
HI-Ped Strategy
- Minimize the cost of statistical QC. Use as few control
measurements as needed. Reduce N to a minimum of 2 whenever possible.
Use single-rule rather than multi-rule procedures. Use control
limits as wide as 3.5s when possible to minimize false rejections.
Aim for 1% or lower false rejection. Increase run length to maximize
test yield, i.e., the ratio of patient samples to control and
calibration samples.
- Minimize cost of non-statistical QC. Recognize the
limitations of control materials and their matrices when minimizing
non-statistical QC. Weigh the clinical needs and risks carefully.
Then identify the minimum frequency of system function checks,
performance validation tests, and preventive maintenance, as
required by regulations, manufacturer's instructions, and good
laboratory practice.
- Document the TQC strategy. The last step for any of
the TQC strategies is documentation, including the QC acceptability
criteria (control rules, N) for assessing the control status
of an analytical run. Document the expected rejection characteristics
of the statistical QC procedure. Establish the schedule for performing
non-statistical QC checks and document the procedures for performing
those checks.
MOD-Ped and LO-Ped Strategies
- Maximize error detection. Increase N from a minimum
of 2 up to at least 4 control measurements per run when using
two control materials and increase N from 3 to 6 when using three
control materials. Increase run length as a way of increasing
N, being careful to satisfy the turnaround time requirements
for the test. Narrow the control limits and tolerate higher false
rejections, up to 6-7% for the N=6 procedures. Change from single-rule
to multirule QC procedures. Use look-back rules to effectively
increase N by inspecting control data from the previous run.
Implement multi-stage QC procedures that have startup designs
with higher N's and/or more sensitive control rules to maximize
error detection, then switch to a monitor design having lower
N and/or less sensitive control rules that minimize false rejections.
Switch back and forth between the startup and monitor designs
as necessary.
- Maximize non-statistical QC. Perform preventive maintenance,
calibration, instrument checks, and performance verification
tests that are required by CLIA, recommended by the manufacturer,
and appropriate for the susceptibility of the method and the
clinical application of the test. Adopt a more aggressive schedule
to minimize problems.
- Improve method performance. Reduce analytical bias
by selecting appropriate standards, calibrating properly, and
by selecting proper comparison groups in proficiency testing
surveys. Reduce imprecision by identifying and minimizing the
major component of variance, standardizing operator techniques,
and mechanizing manual steps in the process. Reduce frequency
of errors by identifying and eliminating sources of problems,
increasing the preventive maintenance schedule, increasing function
checks and performance verification tests, reducing operator
variables, and increasing operating training and expertise. When
necessary, change measurement procedures or analytic systems
to obtain better accuracy, precision, and stability.
Additional steps for LO-Ped strategy
- Optimize QC for process stability. Document the frequency
of errors by careful study of the analytical process. In general,
a 50% error detection rate will be satisfactory for stable processes
that have <2% frequency of errors and even a 25% detection
rate may be sufficient for extremely stable processes that have
<1% frequency of errors.
- Deploy skilled analysts. Assign highly skilled analysts
to testing processes that are problematic and difficult to control.
Provide thorough in-service training. Increase technical skills
and experience. Improve problem-solving capabilities. Improve
statistical skills for method validation and quality control.
Reduce the rotation schedule for personnel to maintain operator
experience and continuity.
- Add patient data QC. Perform between system comparisons
on patient samples. Check patient data with consistency algorithms,
such as delta checks, anion gap, etc. Utilize population statistics,
such as mean of normals, Bull's algorithm, etc. Perform clinical
correlations to check test results with patient diagnosis and
condition.
Cost of quality control vs cost of quality
improvement
The costs of quality control are
ongoing and never-ending, month after month, year after year.
The accompanying figure illustrates the relative costs for different
TQC strategies.
- HI-Ped TQC costs. When 90% detection of medically
important errors can be achieved with statistical QC, cost of
ongoing quality management is lowest. The better the analytical
performance, the lower number of control measurements needed,
the wider the control limits, the lower the false rejection rate,
and the less non-statistical QC is needed. This demonstrates
Deming's principle that improved quality leads to lower costs.
- MOD-Ped TQC costs. When moderate error detection is
achieved - greater than 50% detection of medically important
errors, all the costs go up, including the cost of statistical
QC. There is a need for more control measurements, more preventive
maintenance, more frequent instrument checks, etc., which means
more time and effort by laboratory personnel.
- LO-Ped TQC costs. When error detection is low - less
than 50% detection of medically important errors, costs become
even greater. There is a need for more training, more operator
experience, and more review and correlation of patient data.
At some point, the costs for MOD-Ped TQC and LO-Ped TQC should
justify efforts to improve the performance of an analytical method.
Begin your quality improvement efforts by reducing the inaccuracy
or bias of the method. Next try to reduce the imprecision or CV
of the method. Try to gain enough improvement to change the TQC
strategy, moving from LO-Ped to MOD-Ped to HI-Ped strategies.
If this is not successful, consider replacing the method with
one that has better analytical performance.
REFERENCES
- Health Care Financing Administration (HCFA) and Public Health
Service (PHS), US Dept of Health and Human Services (HHS). Medicare,
Medicaid and CLIA Programs. Regulations implementing the Clinical
Laboratory Improvement Amendments of 1988 (CLIA) and Clinical
Laboratory Improvement Act program fee collection. Fed Regist
1993;58:5215-37.
- U.S. Dept. of Health and Human Services. Medicare, Medicaid,
and CLIA Programs: regulations implementing the Clinical Laboratory
Improvement Amendments of 1988 (CLIA). Final Rule. Fed Regist
1992;57:7002-186.
