METHOD VALIDATION - THE MANAGEMENT OF QUALITY
THE NEED FOR STANDARD PROCESSES AND STANDARDS OF QUALITY
James O. Westgard, PhD
METHOD VALIDATION - Quality must be assured, not assumed! As illustrated in myths of quality, ideas about the current state of quality in healthcare and laboratory testing may be influenced by mistaken yarns, theories, and hypotheses, i.e., myths that are not supported by fact or data. Quality doesn't happen by itself! Quality must be achieved by work processes that are carefully planned, properly operated, optimally controlled, appropriately measured, and continuously improved, i.e., by proper management of quality. This lesson emphasizes the need for standard laboratory processes to provide consistent quality, as well as standards of quality to guide the management of those processes.
Standard has many meanings in the laboratory. In analytical terms, a standard solution provides a known value for calibration of a testing process and represents purity, truth, and correctness. In management terms, a standard process provides a consistent way of doing things, e.g., a standard testing process provides a well-defined protocol for performing a laboratory test. Likewise, a standard method validation process provides a regular and systematic way for evaluating the performance of a laboratory testing process. [Note that the idea of "process" can be applied to any repeated activity, whether physical or mental; both physical work processes and management decision processes need to be standardized or systematized to assure consistent and reliable results.]
Another important term is standard of quality which is a criterion or statement that describes the acceptable level of something. For an analytical test, we need to know how quickly a test result needs to be reported, as well as how close the result must be to the true or correct value. A standard for turnaround time is more easily understood than a standard for truth or correctness. For example, it is obvious that turnaround time should be stated in units of time, usually minutes. These units are understood by both the party requesting the test and the party providing the service. The party ordering the test defines the requirement on the basis of the medical service being provided. Both parties can measure whether the observed performance satisfies the requirement.
Analytical quality is more difficult because it involves technical concepts such as imprecision and inaccuracy, which are not always understood by laboratorians and are certainly even less well understood by the physicians who order the tests or the patients who are the ultimate consumers of the test results. Customers and consumers of laboratory services can not easily define the analytical quality that is required (at least not in the analytical terms desired by the laboratory), nor can they measure or assess analytical quality. The laboratory, therefore, must take full responsibility for managing the analytical quality of its services
Quality management should be a
standard laboratory process. Such a process can be structured
as shown in the accompanying figure [1]. Beginning at the top
and proceeding clockwise, QP refers to quality planning, QLP stands
for quality laboratory processes, QC stands for quality control,
QA for quality assessment, and QI for quality improvement. In
the center, QS stands for quality standards, i.e., standards for
the quality required by the test or service being provided.
This framework provides a quality management process that functions like a feedback loop. QP plans the best way to get the work done, e.g., the selection and evaluation of the analytical methods, equipment, reagents, and procedures used to perform laboratory tests; QLP establishes standard work processes to utilize the policies, procedures, protocols, and personnel of the laboratory; QC provides quantitative measures of process performance using statistical process control techniques; QA provides broader measures of how well the work is getting done, e.g., effectiveness of specimen acquisition procedures, turnaround time for laboratory services, appropriate formats for reporting results, etc. When problems are detected, QI provides a problem solving mechanism to determine root causes, which can then be eliminated through QP, in that case actually re-planning the testing processes and implementing new and better ways of doing the work (i.e., changes in QLP). Through this framework, continuous improvement is built into the management process by cycling through the Q's. Customer focus is achieved by centering these Q's on "quality standards" that represent the laboratory's goals, objectives, and customer requirements - essential information for the objective and quantitative management of the laboratory.
The initial focus of the laboratory is on the establishment of standard operating procedures or processes by which the work gets done. In particular, we are concerned with the selection, validation, and implementation of analytical methods that will provide consistently reliable analytical quality. Those activities are part of QP and lead to the establishment of quality testing processes that are part of QLP. QC is applied after appropriate testing processes have been established. Likewise, QA and QI are part of the ongoing monitoring and improvement of standard operating processes.
Traditional practices for laboratory quality management have always started with establishing standard methods for performing laboratory tests (QLP), then emphasized statistical quality control to monitor analytical performance (QC), and later expanded and broadened the measures of quality to include turnaround time, etc. (QA). More recently, practices of quality improvement (QI) have been introduced as part of Total Quality Management (TQM) or Continuous Quality improvement (CQI) [2,3]. What is usually lacking in laboratories are:
If these quality requirements and the plans for achieving them have not been spelled out, then quality isn't really being managed. Instead, what happens is what happens. It's like the cartoon where Lucky Eddy and Hagar the Horrible are standing at the bar. Hagar asks Lucky Eddy what he'll have, and Lucky Eddy replies "My usual." Hagar asks "What's your usual?" and Lucky Eddy answers "It's what I usually have." In healthcare laboratories, in spite of supposedly well-established management practices of QLP, QC, QA, TQM, and CQI, what happens with quality is usually what has happened before. To actually manage quality, laboratories must define the quality that is required and implement systematic processes to validate method performance, select appropriate statistical QC procedures, and monitor process performance in quantitative and objective ways.
Our objective here is not to have you become an expert in quality standards, but to help you start learning about quality standards and make sure you are aware of different sources of information that are conveniently available. There is no "one and only" way to define the quality needed for a laboratory test, even though there are fierce arguments for or against certain types of quality requirements [see Quality requirements: The debate heats up]. Different types of quality standards are needed to manage quality at different places in the process, such as clinical outcome criteria that reflect medically important changes in test results, analytical outcome criteria that describe the allowable total analytical error in test results, and analytical operating specifications that describe the allowable imprecision, allowable bias, and the QC needed to detect medically important errors in the testing process.
The accompanying figure shows the
relationships between these different types of quality standards.
Starting at the top left of the figure, standard treatment guidelines
(clinical pathways, clinical practice guidelines, etc.) can be
used to define medically important changes and establish clinical
outcome criteria in the form of decision intervals (Dint).
Such clinical criteria can be converted to laboratory operating
specifications for imprecision (smeas), inaccuracy
(biasmeas), and QC (control rules, N) by a clinical
quality-planning model [4] that accounts for preanalytical factors,
such as within-individual biologic. Biologic goals based on within-subject
biologic variability should set a boundary condition on these
operating specifications, defining the most demanding condition
for stable performance that would be required to monitor changes
in individual subjects. The right side of the figure shows how
proficiency testing criteria define analytical outcome criteria
in the form of allowable total errors (TEa), which
can likewise be translated to operating specifications (smeas,
biasmeas, control rules, N) via an analytical quality-planning
model [5]. Note that the allowable total error can also be set
on the basis of total biologic goals [6], therefore the extensive
information that is available on biologic variation can also be
useful in this situation.
In establishing a routine testing process, the performance of the analytical method should be evaluated by a standard method validation process (i.e., a carefully planned set of experimental studies) and judged by comparison to defined standards of quality, i.e., statements of the clinical or analytical quality needed for the test. The lessons on Method Validation will teach you a standard method validation process, i.e., the physical steps needed to obtain reliable data. You also need to learn how to analyze that data and how to apply available quality standards to judge the acceptability of a method's performance, i.e., the mental steps necessary to make a reliable decision on the basis of the experimental data collected. Elsewhere on this website, you can find extensive materials on a quality-planning process for the selection of statistical QC procedures, as well as training materials on Basic QC Practices.
