Dr. Sharon Ehrmeyer is back again! You may remember her article on regulations in our Basic QC Practices series. In this article, she explains which regulations and standards and agencies are involved with Method Validation. A must-read reference.
Please note: Consult our CLIA Final Rules section for the latest on regulations for Method Validation.
The Clinical Laboratory Improvement Amendments of 1988 (CLIA'88) include specific regulations for laboratory quality control, which contain some specific recommendations for method validation [1]. Because of delays in implementing the regulations [2,3], it is difficult to understand which standards are in effect and which ones have been delayed. Therefore, in this discussion, the intent of the regulations is described as "recommendations" rather than "requirements."
Until the CLIA regulations are fully implemented, it will be important to consider the requirements of accreditation organizations with deemed status such as the Joint Commission for Accreditation of Healthcare Organizations (JCAHO) [4], the College of American Pathologists (CAP) [5], and the Commission on Office Laboratory Accreditation (COLA)[6]. There are also state health laboratory organizations in Washington, Oregon, and New York, that are approved by the government (have exempt status) and impose specific requirements [7]. And, it is always important to keep in mind those practices that would be expected as part of professional responsibility and good laboratory practice.
CLIA provides a broad framework of quality control. Subpart K begins with the statement:
"The laboratory must establish and follow written QC procedures for monitoring and evaluating the quality of the analytical testing process of each method to assure the accuracy and reliability of patient test results and reports."
The regulations then proceed to define a broad set of standards for quality control, which include sections on each of the following:
The section on the "establishment and verification of method performance specifications" provides the recommendations for method validation studies.
The recommendations that must be followed depend on the "complexity" of the test. The categories are waived, moderate complexity, and high complexity. Information about classification of specific tests is available from the CDC web-site at the address: http://www.hcfa.gov/medicaid/clia/cliahome.htm. Each of these categories has different regulatory requirements for personnel, quality control, quality assurance, proficiency testing, etc.
Waived tests. The minimum requirement for waived testing is to follow the manufacturers' directions. There are no recommendations for method validation.
Unmodified moderate and high complexity tests. Most tests performed in laboratories today fall in these categories, thus most method validation studies should follow these recomendations. Section §493.1213(b)(1) identifies what is intended when CLIA is finalized:
"…a laboratory that introduces a new procedure for patient testing using a moderate or high complexity method (instrument, kit, or test system) cleared by the FDA as meeting the CLIA requirements for general quality control, must demonstrate that prior to reporting patient test results, it can obtain the performance specifications for accuracy, precision, and reportable range of patient test results, comparable to those established by the manufacturer. The laboratory must also verify that the manufacturer's reference range is appropriate for the laboratory's patient population."
This would generally mean performing four different experiments:
Modified moderate or high complexity tests. Section §493.1213(b)(2) identifies what is intended when CLIA is fully implemented:
This would mean performing seven different experiments:
Documentation. Section §493.1213(c) requires that "the laboratory must have documentation of the verification or establishment of all applicable test performance specifications.
Additional requirements for periodic validation of method performance. Ongoing (at least semi-annual) assessment of accuracy is required under Subpart P - Quality Assurance. The need for ongoing method comparison studies is given in §493.1709 Standard; Comparison of test results", which states:
"If a laboratory performs the same test using different methodologies or instruments, or performs the same test at multiple testing sites, the laboratory must have a system that twice a year evaluates and defines the relationship between test results using different methodologies, instruments, or testing sites.
"In addition, if a laboratory performs tests that are not included under Subpart I, Proficiency Testing, the laboratory must have a system for verifying the accuracy and reliability of its test results at least twice a year."
Waived tests. JCAHO recognizes the waived tests as defined by CLIA'88, but requires more than just following manufacturer's directions. JCAHO's standard WT.1.3, "policies and procedures governing specific testing-related processes are current and readily available," includes equipment performance evaluation. In discussing the standard with Anne Belanger, director of JCAHO's Laboratory Accreditation Services, she indicated this means having some evidence (available for the inspectors) that all tests are meeting the needs of the testing site's clientele. The evidence includes: accuracy, precision, and reportable range (validity of low and high values) data. In addition, someone (i.e., the testing site's director) will need to verify that the reference range(s) taken from the manufacturer or elsewhere are appropriate.
For JCAHO, it is only necessary to evaluate the method, not all instruments, i.e., all glucose meters, employing the method. As part of JCAHO's total quality management philosophy, performance history, including QC data, can be used as the "evidence" of adequate method performance when a method used in one location is instituted in another.
Unmodified Moderate complexity tests. While JCAHO requires the same general QC requirements as CLIA, JCAHO also has specific method validation requirements (QC.1.2). JCAHO identifies the intent of QC.1.2 as follows:
"before a new test method is used to report patient results, the laboratory must verify that the method will produce accurate results on a consistent and reliable basis...the laboratory at least verifies accuracy, precision, and reportable range for a patient-testing procedure that is an approved, unmodified test of moderate complexity for which the manufacturer has established the performance specifications. The laboratory also ensures that the reference range applies to the specific patient population(s) tested."
"For previously established methods, QC data and test performance history are adequate to confirm validity."
High complexity tests. Testing sites that modify moderate complexity tests (this includes not following the manufacturer's directions) or use tests developed in-house or tests classified as high complexity under CLIA must follow all the method validation requirements identified in Section §493.1213 of the CLIA regulations for high complexity tests. These requirements include: accuracy; precision; analytical sensitivity and specificity, if applicable; the reportable range of patient test results, the reference ranges(s) (normal values); and any other applicable performance characteristic that may be appropriate for the test.
CAP-LAP's philosophy is that all clinical laboratory testing, including even CLIA waived tests, essentially need to meet the requirements defined for high complexity testing under Section §493.1213 of CLIA'88.
The 1998.1 CAP-LAP Checklist (Laboratory General -- Section 1) includes performance specification requirements -- accuracy (01:4202), precision (01:4202), reportable range (01:4204), and reference range (01:4208)-- for each test procedure. Sensitivity is discussed under reportable range and is identified as the "minimal limit of quantification." Specificity (01:4203) implies an evaluation of the method's ability to respond correctly to the concentration of analyte in the presence of interfering substances. CAP's Checklist now states: "that interfering substances may pose a significant problem to the clinical laboratory and healthcare providers...the laboratory must be aware of common interferences by performing studies or having available studies performed elsewhere (such as by the instrument-reagent manufacturer) that are consistent with NCCLS EP7-A."
You can select an approved ("deemed" status) organization whose requirements are at least equivalent to CLIA'88 regulations.
The Health Care Financing Administration (HCFA) will inspect any size laboratory, including physician office laboratories, for adherence to the CLIA requirements; the Commission of Office Laboratory Accreditation (COLA) inspects primarily physicians office laboratories for adherence to COLA standards which closely parallel the CLIA regulations; the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) inspects laboratories and testing sites as well as the entire hospital or healthcare organization; and the College of American Pathologists Laboratory Accreditation Program (CAP-LAP) mainly inspects large laboratories directed by pathologists. There are other government approved (deemed) organizations that have standards for laboratories to follow and some states, such as Washington, Oregon and New York, impose specific requirements.