QP 3: Complying with Regulations,
Standards, and Practice Guidelines
Implementation of a quality planning process should be a high
priority in laboratories today to assure or guarantee the quality
of laboratory tests and services. Guidelines for quality planning
can be found in government regulations, accreditation standards,
and national practice standards. The widespread application of
quality planning is recommended in the latest inspection manual
from the Joint Commission for Accreditation of Healthcare Organizations
(JCAHO). A focus on analytical quality, the assessment of method
performance, and the selection of appropriate quality control
procedures is provided by the Clinical Laboratory Improvement
Amendments (CLIA-88). A general planning methodology for quality
control is outlined by the National Committee for Clinical Laboratory
Standards (NCCLS). Together, these guidelines provide the basis
for implementing a quality planning process in laboratories. Here's
a summary of the standards, rules, and recommendations that bear
on quality planning.
JCAHO Guidelines for Improving Organizational
Effectiveness
Improving Organizational Performance, or IOP, is JCAHO's latest
terminology and represents an evolution of their TQM philosophy
and concepts. The 2000-2001 Comprehensive Accreditation Manual
for Pathology and Clinical Laboratory Services [1] states that
the goal of IOP is "for the laboratory to design processes
well and systematically monitor, analyze, and improve its services
that affect patient health outcomes." The following are identified
as essential management activities:
- Designing processes;
- Monitoring performance through data collection;
- Analyzing current performance; and
- Improving and sustaining improved performance.
Review of specific standards
More guidance is provided by specific standards. In the list
that follows, I have also identified how these standards fit into
the total quality management framework described earlier, which
is composed of quality laboratory practices (QLP), quality control
(QC), quality assessment (QA), quality improvement (QI), quality
planning (QP), and quality goals (QG). This additional information
will be helpful for assessing the relationship between IOP and
TQM.
- PI.1 The leaders establish a planned, systematic, organization-wide
approach to process design and performance measurement, analysis,
and improvement. [QP to establish QLP]
- PI.1.1 The activities are planned in a collaborative and
interdisciplinary manner. [QP across functions or departments
to establish QLP]
- PI.1.2 New or modified processes are designed well. [QP or
re-planning to establish QLP]
- PI.2.1 Performance expectations are established for new and
modified processes. [QG or requirements for QLP]
- PI.2.2 The performance of new and modified processes is measured.
[QA of QLP]
- PI.3 Data are collected to monitor the stability of existing
processes, identify opportunities for improvement, identify changes
that will lead to improvement, and sustain improvement. [QC and
QA of QLP]
- PI.3.1 The organization collects data to monitor its performance.
[QA]
- PI.3.1.1 The organization collects data to monitor the performance
of processes that involve risks or may result in sentinel events.
[QA of QLP]
- PI.3.1.2 The organization collects data to monitor performance
in areas targeted for further study. [QA]
- PI.3.1.3 The organization collects data to monitor improvements
in performance. [Q A]
- PI.4 Data are systematically aggregated and analyzed on an
ongoing basis. [QA]
- PI.4.1 Appropriate statistical techniques are used to analyze
and display data. [QA and QI]
- PI.4.2 The organization compares it performance over time
and with other sources of information. [QA and QI]
- PI.4.3 Undersirable patterns or trends in performance and
sentinel events are intensively analyzed. [QA and QI]
- PI.4.4 The organization identifies changes that will lead
to improved performance and reduce the risk of sentinel events.
[QI and QP or re-planning]
- PI.5 Improved performance is achieved and sustained. [QP,
QLP, QC, QA, QI]
IOP vs TQM
From the review of these standards
and comparison to our TQM framework, the relationship between
IOP and TQM is described in the accompanying figure, where theemphasis
on each component is shown by the darkness of each Q. Quality
Planning is strongly recommended by the JCAHO standards on the
"design" of processes. Quality Laboratory Processes
is pervasive throughout the standards and JCAHO's frequent reference
to "processes." Quality Control and Quality Assessment
provide the measure and monitor aspects that are mentioned in
a large number of the standards. Quality Improvement is the focus
of the whole IOP approach and is specifically mentioned in several
standards. Quality goals are found in the standard that recommends
establishing performance expectations for new and modified processes.
In summary, JCAHO provides a broad recommendation for quality
planning that applies to all laboratory processes. Given that
the fundamental laboratory process is to perform tests, carefully
planned testing processes should be the heart of quality laboratory
processes. Performing tests is the main event in the laboratory.
Other activities are important in support of testing and patient
services, but none of them willl matter if the laboratory can't
produce correct test results.
CLIA Rules and Regulations
The strongest guidelines for analytical quality management
are found in the CLIA-88 rules and regulations [2]. CLIA provides
a broad focus on quality management of the "total testing
process", which include the pre-analytical, analytical, and
post-analytical phases. Rules for the analytical phase provide
very detailed guidelines for method validation and quality control,
i.e., for establishing valid testing processes and assuring the
quality of the test results on an ongoing basis. CLIA also establishes
specific criteria for acceptable performance in proficiency testing
surveys, thereby defining the minimum quality standards that must
be achieved by US laboratories.
Review of specific rules
Subpart K of the CLIA rules begins with a statement that "the
laboratory must establish and follow written QC procedures for
monitoring and evaluating the quality of the analytical testing
process of each method to assure the accuracy and reliability
of patient test results and reports." This is followed by
detailed rules that can be ascribed to different components in
our TQM framework, as follows:
- Quality Goals are defined by the proficiency testing criteria
for acceptable performance which are given for immunology [493.927],
routine chemistry [493.931], endocrinology [493.933], toxicology
[493.937], and hematology [493.941].
- Quality Laboratory Processes are defined through the facilities
[493.1204], test methods, equipment, instrumentation, reagents,
materials, and supplies [493.1205], and the procedure manual
[493.1211], as well as detailed personnel standards [Subpart
M].
- Quality Control is described by control procedures [493.1218],
remedial actions [493.1219], and quality control records [493.1221].
- Quality Assessment requires the establishment and verification
of method performance specifications [493.1213], calibration
verification procedures [493.1217], and participation in external
proficiency testing [Subpart H].
- Quality Improvement is not mentioned.
- Quality Planning for analytical methods is implied through
the requirement for verification of QC.
- "For each method that is developed in-house, is a modification
of the manufacturer's test procedure, or is a method that
has not been cleared by FDA as meeting the CLIA requirements
for general quality control, the laboratory must evaluate
the instrument and reagent stability and operator variance in
determining the number, type, and frequency of testing calibration
or control materials and establish criteria for acceptability
used to monitor test performance during a run of patient specimen(s)."
[page 7166, 493.1218(b), emphasis added]
Given that no methods have been cleared by FDA as meeting
the CLIA requirements for general quality control, all QC procedures
need to be carefully planned or designed to account for the variability
observed for the instrument, reagents, and operators in the laboratory.
CLIA rules vs TQM
The accompanying figure summarizes
the relationship of the CLIA rules to the TQM framework. CLIA
provides a strong emphasis on Quality Laboratory Processes, particularly
analytical testing processes, through the detailed rules that
govern facilities, equipment and materials, procedure manual,
and personnel. The rules for Quality Control and Quality Assessment
are also much more detailed, as applied to analytical methods.
There is no mention of Quality Improvement, but Quality Planning
for analytical processes is implied in the rules for establishing
QC procedures. Specific Quality Goals are defined for approximately
80 different tests that include some from immunology, hematology,
coagulation, chemistry, endocrinology, and toxicology.
NCCLS QC Practice Guidelines
Given that JCAHO provides a strong recommendation for Quality
Planning and that CLIA specifies Quality Goals and requires that
QC procedures account for the laboratory's own variability, laboratories
must carefully plan their analytical testing processes. NCCLS
provides the methodology for accomplishing this in its 1999 QC
practice guideline [3] which describes a step-by-step process
for selecting a statistical QC procedures on the basis of the
quality required by a test and the performance observed for a
method.
Review of NCCLS QC planning methodology
The NCCLS methodology appears in
section 5 of the C24-A2 document. The steps recommended are shown
in the figure and explained below:
- Define the quality requirement. A quality requirement
may be defined in terms of an allowable total analytical error,
such as often provided by proficiency testing criteria for acceptable
performance. The allowable analytical error is the magnitude
of analytical error that if exceeded would cause a test result
to be of unacceptable quality [4]. It encompasses both random
and systematic errors, i.e., both method imprecision and bias.
There also are recommendations for medically important changes
in test results that similarly include both method imprecision
and bias, as well as subject biological variation [5]. Biologic
variation itself provides another basis for defining the allowable
imprecision and allowable bias for a test [6]. Clinical treatment
models can also be a source of information about the analytical
quality required to assure that test results are medically useful
[7].
- Determine method performance. The performance characteristics
of an analytical process that are critical for the proper planning
of QC procedures are imprecision and bias. Estimates of these
parameters should represent the stable performance of an analytical
process. In addition to imprecision and bias, it would be useful
to have information about unstable performance, such as the expected
type, magnitude, and frequency of analytical errors, but this
information is not generally available.
- Identify candidate statistical QC strategies. A quality
control strategy is defined by the control materials used, the
number of control samples analyzed, the location of these control
samples in an analytical run, the quality control rules applied
to the control sample measurements, and the time when the quality
control rules are evaluated. The appropriateness of the QC strategy
depends on the quality required, as well as the expected instability
of the analytical method (e.g., type, magnitude, and frequency
of errors). Several QC strategies may be defined and evaluated.
- Predict QC performance. The performance of a quality
control strategy can be predicted from probability calculations
or from computer simulation studies. The most direct indicator
of the performance of a quality control procedure is the expected
number of unacceptable patient test results that are produced
(or reported) when an out-of-control error condition exists [8].
This will depend on the type and magnitude of the out-of-control
error condition, when the error condition occurs and how long
it lasts, which in turn depends on how frequently quality control
testing occurs and the probability that the quality control rules
detect the error condition. These predictions generally assume
the shape of the error distribution is gaussian, which may not
account for some periodic and irregular effects observed with
real laboratory systems, therefore the complexity of the prediction
model needs to match the complexity of the potential error sources
of the method and system.
- Set goals for QC performance. Quality control performance
goals set desirable targets for quality control performance.
The goal will depend on the chosen quality control performance
measure. Thus, one goal could be specified as a maximum allowable
number of unacceptable results due to an out-of-control error
condition, or a maximum allowable probability or reporting unacceptable
results (maximum defect rate), or a minimum acceptable probability
of detecting an out-of-control error condition. Another goal
could specify a maximum allowable probability of false rejections.
- Select appropriate QC. When more than one quality
control strategy meets the quality control performance goals,
other characteristics such as cost and ease of implementation
can be used to select the best approach. Practical approaches
for selecting appropriate QC procedures have been described based
on power function graphs, critical-error graphs, and charts of
operating specifications [9]. Illustrative applications of QC
planning are available in the literature to provide guidance
in selecting appropriate QC strategies [10, 11].
Other guidelines
The College of American Pathologists (CAP) also provides detailed
guidelines that focus on laboratory testing processes. CAP has
"deemed status," meaning the CAP guidelines are at least
as stringent as the CLIA rules. Likewise, the Commission of Office
Laboratory Accreditation (COLA) has deemed status and it's guidelines
must comply with the CLIA rules. Therefore, the CLIA rules, JACHO
IOP standards, and NCCLS QC planning process provide the a comprehensive
synthesis of guidelines for managing quality in healthcare laboratories.
These guidelines are also seen to be consistent with the principles
from TQM.
REFERENCES
- 2000-2001 Comprehensive Accreditation Manual for Pathology
and Clinical Laboratory Services. Joint Commission on Accreditation
of Healthcare Organizations, Oakbrook Terrace, IL, 1999.
- U.S. Department of Health and Human Services. Medicare, Medicaid
and CLIA programs; Regulations implementing the Clinical Laboratory
Improvement Amendments of 1988 (CLIA). Final rule. Fed Regist
1992;57:7002-186.
- C24-A2. Statistical Quality Control for Quantitative Measurements:
Principles and Definitions; Approved Guideline - Second Edition.
National Committee for Clinical Laboratory Standards, Wayne,
PA, 1999.
- Burnett RW, Westgard JO. Selection of measurement and control
procedures to satisfy HCFA requirements and provide cost-effective
operation. Arch Pathol Lab Med 1992;116:777-782.
- Skendzel LP, Barnett RN, Platt R. Medically useful criteria
for analytical performance of laboratory tests. Am J Clin Pathol
1985;83:200-205.
- Fraser CG, Hyltoft Petersen P, Ricos C, Haeckel R. Proposed
quality specifications for the imprecision and inaccuracy of
analytical systems for clinical chemistry. Eur J Clin Chem Clin
Biochem 1992;30:311-317.
- Hyltoft Petersen P, deVerdier C-H, Groth T, Aronsson T. Clinically
based quality goals; a NORDKEM project. Eur J Haematol. 1990;45(Suppl
53):6-8.
- Parvin CA, Gronowski AM. Effect of analytical run length
on quality-control (QC) performance and the QC planning process.
Clin Chem 1997;43:2149-2154.
- Westgard JO. Error budgets for quality management: Practical
tools for planning and assuring the analytical quality of laboratory
testing processes. Clin Lab Manag Review 1996;10:377-403.
- Koch DD, Oryall JJ, Quam EF, Feldbruegge DH, Dowd DE, Barry
PL, Westgard JO. Selection of medically useful QC procedures
for individual tests on a multi-test analytical system. Clin
Chem 1990;36:230-233.
- Mugan K, Carlson IH, Westgard JO. Planning QC procedures
for immunoassays. J Clin Immunoassay 1994;17:216-222.
