METHOD VALIDATION - REFERENCE INTERVAL TRANSFERENCE
Patricia L. Barry BS, MT(ASCP) and James O. Westgard, PhD
March 1999
METHOD VALIDATION - The reference interval is the last characteristic to be studied in the method validation process. It is generally studied last because the reference interval itself doesn't enter into the decision on method acceptability and the study isn't needed when method performance is unacceptable. If method performance is acceptable, then it is important to assess the reference interval(s) to support the interpretation of patient test results.
A reference interval is typically established by assaying specimens that are obtained from individuals that meet carefully defined criteria (reference sample group). Protocols such as those of the International Federation of Clinical Chemistry Expert Panel on Theory of Reference Values [1-6] and the National Committee for Clinical Laboratory Standards [7] delineate comprehensive, systematic processes that use carefully selected reference sample groups to establish reference intervals. These protocols typically need a minimum of 120 reference individuals for each group (or subgroup) that needs to be characterized.
For example, to establish a reference interval for hemoglobin - a test that is gender dependent - the laboratory would need to obtain hemoglobin results on 240 reference individuals (120 men and 120 women). These individuals are typically recruited from the general regional population (essentially the facilities' market-base) and then selected for inclusion in the study using carefully defined criteria. The selection is often accomplished by administering a health questionnaire. Sometimes a physical examination is also required as a way to determine acceptability for inclusion.
The establishment of reference intervals requires careful planning, control, and documentation of each aspect of the study. Thus, the resulting reference intervals are well-characterized in terms of the variation attributable to pre-analytical and analytical factors. These formal protocols are particularly helpful when a laboratory needs to establish its own reference interval for a particular test. This situation may occur if a laboratory has modified a previously FDA-approved method or developed an in-house test. Unfortunately, these protocols are resource-intensive and can be prohibitive for smaller facilities in light of current cost directives. Even large laboratories are finding it increasingly difficult to conduct these comprehensive studies cost-effectively. Therefore, laboratories are becoming more reliant on manufacturers to establish scientifically sound reference intervals that can be verified using simpler, less labor-intensive, and lower cost approaches.
In this lesson, the focus is on the "transference"
of reference intervals, which requires considerably less effort
and less data than necessary for the establishment of reference
intervals. The reference interval that is of most interest during
the method validation process is one that describes the test values
typically observed in a "healthy" population. This interval
has historically been referred to as the "normal range"
and is derived by assaying specimens from individuals who meet
criteria for "good" health (e.g., "have no known
health problems, are ambulatory, not on any regular medication
regimen, have a weight within the recommended norms, etc.").
The test results (reference values) from this sample group are
analyzed statistically to determine an interval of values that
includes a specified percentage of all the values (reference interval)
from the sample group. By tradition, this interval includes 95%
of the values (usually the central 95%). A pair of test values
(called the lower and upper reference limits) represents the boundaries
of the interval. Patient results falling outside the reference
limits are typically flagged in some way as "abnormal"
results.
The NCCLS Approved Guideline C28-A [7] describes different ways for a laboratory to validate the "transference" of established reference intervals to the individual laboratory:
1. Divine judgment. The acceptability of the transfer may
be subjectively assessed on the basis of consistency between the
"demographics and geographics" of the study population(s)
and the demographics of the laboratory's test population(s). The
laboratory simply reviews the information submitted and subjectively
verifies that the reference intervals are applicable to the adopting
laboratory's patient population and test methods. To do this,
all the information about the original study should be requested
and made available to the adopting laboratory. This includes the
demographics of the reference sample group, the selection process,
pre-analytical conditions of the study such as subject preparation
and specimen collection and handling techniques, the analytical
system used, and the statistical method used to establish the
intervals. Sometimes it is useful to request the original reference
values and to re-analyze them to verify the original statistical
analysis. Most cases for transference involve adoption of intervals
from another laboratory using the same analytical system or intervals
established by the method manufacturer.
2.
Verification with 20 samples. An experimental validation may
be performed by collecting and analyzing specimens from 20 individuals
who represent the reference sample population. If two or fewer
test results fall outside the claimed or reported reference limits,
the reference interval is considered verified, as illustrated
in the accompanying figure.
3. Estimation with 60 samples. An experimental validation may be performed by collecting and analyzed specimens from 60 individuals who represent the reference sample population. The actual reference interval is estimated and compared to the claimed or reported interval using a statistical formula comparing the means and standard deviations of the two populations.
4.
Calculation from comparative method. The NCCLS document also
recognizes - but doesn't endorse - another approach that would
adjust or correct the claimed or reported reference intervals
on the basis of the observed methodological bias and the mathematical
relationship demonstrated between the analytical methods being
used (as shown in the accompanying figure). The regression statistics
obtained from a comparison of methods study could be used to calculate
the reference limits (Xlower and Xupper)
to the new method (Ylower = a + bXlower,
Yupper = a + bXupper, where a is the y-intercept
and b is the slope of the regression line).
For tests where there are well-established reference intervals for the comparative method in your laboratory, transfer those intervals by calculation using the regression equation obtained from the comparison of methods experiment performed in your laboratory. Be sure the regression statistics are reliable by following the guidelines for performing the comparison of methods experiment and providing the proper statistical analysis of the data. Transference will require only a few additional calculations, making it quick and easy to determine the new limits. This approach should be suitable for common chemistry and hematology tests where one-to-one agreement can be expected.
For tests where there are systematic differences between the new and comparative methods, use the calculation approach, as above, to estimate the reference intervals and compare with the manufacturer's claims. Further verify the transferred limits by analysis of 20 specimens from healthy subjects. This approach would be appropriate with enzyme methods where proportional differences often exist between methods and immunoassays where systematic changes may occur between generations of measurement systems.
Use the 60 specimen approach to make estimates of reference interval when the reference interval information from the manufacturer is not adequate, when the new test method is based on a different measurement principle and different measurement specificity, or when the test is being applied to a different patient population than previously.
Use the divine judgment approach when there are no experimental
data to support transference of the reference intervals.
