Final5
CLIA Rule. Part IV: The New Method Validation Regulations
Final5
CLIA Rule. Part IV: The regulations disseminated on February 28, 1992 for laboratories to comply with the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) included specific quality control (QC) regulations for laboratories performing moderate and high complexity testing [1]. These regulations also contained specific method validation requirements for modified moderate and high complexity tests and tests developed in-house. Test sites performing unmodified, FDA approved, moderate complexity testing could accept the manufacturer's performance specifications and were not required to perform any method validation. On January 24, 2003, the Centers of Medicare and Medicaid Services (CMS) issued the final CLIA rules [2]. These rules bring ALL non-waived (moderate and high complexity) testing under uniform QC requirements, including validation of methods.
While CLIA sets the minimum testing requirements, testing sites can and do choose to meet the requirements of accreditation organizations with CLIA-deemed status such as the Joint Commission for Accreditation of Healthcare Organizations (JCAHO) [3], the College of American Pathologists (CAP) [4], and the Commission on Office Laboratory Accreditation (COLA)[5]. There are also state health laboratory organizations in Washington and New York, which are approved by the government (have exempt status) and impose specific requirements. And, it is always important to keep in mind those practices that would be expected as part of professional responsibility and good laboratory practice.
CLIA provides a broad framework of quality control based on a quality systems approach that follows the route of a specimen through the laboratory. Section §493.1200 of Subpart K, Quality Systems for Non-waived Testing, begins with the statement:
"Each laboratory…must establish and follow written policies and procedures that implement and monitor quality systems for all phases of the total testing process …"
Section §493.1250, Analytic Systems, states:"…The laboratory must monitor and evaluate the overall quality of the analytic systems and correct identified problems…
Subpart K under the analytic systems section defines a broad set of standards for quality systems, which include sections on each of the following:
- Procedure manual (§493.1251)
- Test systems, equipment, instruments, reagents, materials, and supplies (§493.1252)
- Establishment and verification of method performance specifications (§493.1253)
- Equipment maintenance and function checks (§493.1254)
- Calibration and calibration verification procedures (§493.1255)
- Control procedures (§493.1256)
- Specialty and subspecialty requirements (§493.1261-1278)
- Comparison of test results (§493.1281)
- Corrective actions (§493.1282)
- Test records (§493.1283)
- Analytic systems assessment (§493.1289)
The section on the "establishment and verification of method performance specifications" provides the recommendations for method validation studies.
The recommendations that must be followed depend on the "complexity" of the test. The categories are waived, moderate complexity, and high complexity, but for purposes of method validation, the CLIA Final Rule combines moderate and high complexity into the Non-Waived classification. Information about classification of specific tests is available from the CMS website at the address: http://www.cms.gov/clia/
Waived Tests. The minimum requirement for waived testing is to follow the manufacturer's directions. There are no recommendations for method validation.
Non-Waived Tests Approved by FDA. Most tests performed in laboratories today fall in these categories, thus most method validation studies should follow these recommendations. Sections §493.1253(b) identifies the requirements:
(b)(1) Verification of performance specifications. Each laboratory that
introduces an unmodified, FDA-cleared or approved test system must do the following before reporting patient test results:
(i) Demonstrate that it can obtain performance specifications comparable
to those established by the manufacturer for the following performance
characteristics:
(A) Accuracy.
(B) Precision.
(C) Reportable range of test results for the test system.
(ii) Verify that the manufacturer's reference intervals (normal values) are
appropriate for the laboratory's patient population.This would generally mean performing four experiments:
- A comparison of methods experiment to estimate inaccuracy or bias,
- A replication experiment to estimate imprecision,
- A linearity type experiment to estimate imprecision,
- A linearity type experiment to determine the reportable range, and
- Collecting reference values to verify the reference range [alternatively, the laboratory medical director can document that the manufacturer's ranges or textbook ranges are appropriate for the clientele being served].
Non-Waived Tests Modified or Developed In-House. Section §493.1253(b)(2) identifies the method validation requirements for these tests:
(b)(2) Each laboratory that modifies an FDA-cleared or approved test system, or introduces a test system not subject to FDA clearance or approval (including methods developed in-house and standardized methods such as text book procedures, Gram stain, or potassium hydroxide preparations), or uses a test system in which performance specifications are not provided by the manufacturer must, before reporting patient test results, establish for each test system the performance specifications for the following performance characteristics, as applicable:
(i) Accuracy.
(ii) Precision.
(iii) Analytical sensitivity.
(iv) Analytical specificity to include interfering substances.
(v) Reportable range of test results for the test system.
(vi) Reference intervals (normal values).
(vii) Any other performance characteristic required for test performance.(3) Based upon the performance specifications verified or established in accordance with paragraph (b)(1) or (b)(2) of this section, determine the test system's calibration and control procedures for patient testing as required under §493.1255 and §493.1256.
This would mean performing seven different experiments:
- comparison of methods experiment to estimate inaccuracy or bias
- replication experiment to estimate imprecision,
- detection limit experiment for estimating constant interferences,
- recovery experiments for estimating proportional interferences,
- linearity type experiment to determine the reportable range, and
- more extensive reference value study to estimate the reference range(s).
Documentation. Section §493.1253(c) requires that the laboratory must have documentation of the verification or establishment of all applicable test performance specifications.
Additional requirements for periodic validation of method performance. Ongoing (at least semi-annual) assessment of accuracy is required under Subpart K, section §493.1236 and is accomplished either through proficiency testing or by some other system for verifying the accuracy and reliability of its test results at least twice a year. The need for ongoing method comparison studies is given in §493.1281 Standard; Comparison of test results", which states:
"If a laboratory performs the same test using different methodologies or instruments, or performs the same test at multiple testing sites, the laboratory must have a system that twice a year evaluates and defines the relationship between test results using different methodologies, instruments, or testing sites.
Waived tests. JCAHO recognizes the waived tests as defined by CLIA '88, but requires more than just following manufacturer's directions. JCAHO's waived testing standard WT.1.3, "policies and procedures governing specific testing-related processes are current and readily available," includes equipment performance evaluation. For laboratories to meet this standard, they need some evidence (available for the surveyors) that all tests are meeting the needs of the testing site's clientele. The evidence includes: accuracy, precision, and reportable range (validity of low and high values) data. In addition, someone (i.e., the testing site's director) will need to verify that the reference range(s) taken from the manufacturer or elsewhere are appropriate.
For JCAHO, it is only necessary to evaluate the method, not all instruments, e.g., all glucose meters, once. As part of JCAHO's total quality management philosophy, performance history, including QC data, can be used as the "evidence" of adequate method performance when a method used in one location is instituted in another.
Non-Waived Tests Approved by FDA. While JCAHO requires the same general QC requirements as CLIA, JCAHO also has specific method validation requirements (QC.1.2). JCAHO identifies the intent of QC.1.2 as follows:
"before a new test method is used to report patient results, the laboratory must verify that the method will produce accurate results on a consistent and reliable basis…the laboratory at least verifies accuracy, precision, and reportable range for a patient-testing procedure that is an approved, unmodified test of moderate complexity for which the manufacturer has established the performance specifications. The laboratory also ensures that the reference range applies to the specific patient population(s) tested."
"For previously established methods, QC data and test performance history are adequate to confirm validity."
Non-Waived Tests Modified or Developed In-House. Testing sites that modify moderate complexity tests (this includes not following the manufacturer's directions) or use tests developed in-house or tests classified as high complexity under CLIA must follow all the method validation requirements identified in Section §493.1253(b)(2) of the CLIA. These requirements include: accuracy; precision; analytical sensitivity; analytical specificity to include interfering substances; reportable range of test results for the test system; reference intervals (normal values); and any other applicable performance characteristics required for test performance.
CAP's philosophy is that all clinical laboratory testing, including even CLIA waived tests, essentially need to meet the requirements defined under Section §493.1253(b)(2) of CLIA '88.
The 2002 CAP Checklist GEN (Laboratory General) includes performance specification requirements - accuracy and precision (GEN: 42020), reportable range (GEN: 40944), sensitivity (GEN: 42025), and reference range (GEN: 40946) - for each test procedure. Specificity (GEN: 42030) implies an evaluation of the method's ability to respond correctly to the concentration of analyte in the presence of interfering substances. CAP's Checklist now states: "The laboratory must verify or establish method performance specifications for analytic interferences. Interfering substances pose a significant problem to the clinical laboratory and healthcare providers who may be misled by laboratory results that do not represent the clinical conditions of patients. The laboratory must be aware of common interferences by performing studies or having available studies performed elsewhere (such as by the instrument-reagent manufacturer)."
You can select an approved ("deemed"status) organization whose requirements are at least equivalent to CLIA '88 regulations.
The Centers of Medicare and Medicaid Services will inspect any size laboratory, including physician office laboratories, for adherence to the CLIA requirements; the Commission of Office Laboratory Accreditation (COLA) inspects primarily physicians office laboratories for adherence to COLA standards which closely parallel the CLIA regulations; the Joint Commission of Accreditation of Healthcare Organizations (JCAHO) inspects laboratories and testing sites as well as the entire hospital or healthcare organization; and the College of American Pathologists Laboratory Accreditation Program (CAP-LAP) mainly inspects large laboratories directed by pathologists. There are other government approved (deemed) organizations that have standards for laboratories to follow and some states, such as Washington and New York, impose specific requirements.
- U.S. Department of Health and Human Services. Medicare, Medicaid and CLIA programs: Regulations implementing the Clinical Laboratory Improvement Amendments of 1988 (CLIA). Final rule. Fed Regist 1992; 57:7002-186.
- U.S. Department of health and Human Services. Medicare, Medicaid and CLIA programs: Laboratory requirements relating to quality systems and certain personnel qualifications. Final rule. Fed Regist 2003; 68:3640-714. (available at: http://www.phppo.cdc.gov/clia/pdf/CMS-2226-F.pdf)
- Accreditation Manual for Pathology and Clinical laboratory Services. Joint Commission on Accreditation of Healthcare Organizations (JCAHO). Oakbrook Terrace, IL, 2002.
- Laboratory Accreditation Program. College of American Pathologists (CAP). Northfield, IL, 2002.
- Accreditation Manual. Commission of Office Laboratory Accreditation (COLA). Colombia, MD, 2002.
