DESIGNING
QUALITY CONTOL FOR NEONATAL SCREENING
ASSAYS1Suolinna E-M, 2Torresani T, 3Westgard JO
DESIGNING
QUALITY CONTOL FOR NEONATAL SCREENING
ASSAYSAs part of the follow-up to the previous QC application on Six Sigma and Neonatal Screening, we thought it would be valuable to include some of the supporting studies. The paper below was recently presented at the 4th Asia Pacific Regional Meeting of the International Society for Neonatal Screening, in Mandaluyong City, Philippines, Oct 17 -19, 2001 (the website is http://www.newbornasia.com/ ). A similar poster was also presented in the
Newborn Screening and Genetic Testing Symposium in Raleigh, North Carolina, May 6 - 9, 2001.
In neonatal screening a large population of apparently healthy infants is tested in order to find those that are at risk for a certain genetic disorder. The whole screening process requires careful quality assurance so that all infants with the disease are identified and the needed treatment can be started promptly. However, the information of quality requirements for neonatal assays is rather limited. In the beginning assays were semi-quantitative and most laboratories based their quality control on a long and good knowledge of the assays. With more precise and automated assays a quantitative approach to quality controls becomes essential. The analytical quality control in the laboratory can be managed by statistical QC procedures.
For the quality planning the following points need to be addressed:
- What is the required quality?
- How is the quality requirement translated into operating specifications for allowable imprecision and inaccuracy?
- What quality control is needed to detect medically important errors with minimum false rejections?
Required quality: The required quality can best be derived from the purpose of the screening test:
- To sort concentrations into one group that is below and one group that is above a certain cut-off level, and preferably by defining a gray zone between these.
Many neonatal screening laboratories provide guidelines that include a gray zone. Professional organizations also make recommendations that include a gray zone, e.g. the American Academy of Pediatrics guidelines for congenital hypothyroidism screening define a gray zone of 10 - 20 mU/L TSH in blood [1].
The gray zone or "decision interval" type of quality requirement can be used to derive specifications for imprecision, inaccuracy, and QC as described earlier [2]. The chart of operating specifications (OPSpecs chart) provides a practical tool for displaying the relationship between the quality required for the test, the inaccuracy and imprecision of the method, and different statistical control rules and numbers of control measurements [3,4].
The present work is based on the use of the EZ Rules software for constructing OPSpecs charts. The program automatically selects QC rules, based on either traditional statistical rules or average-of-normal results. The data are from the routine neonatal screening program in University Children's Hospital, Dept. of Endocrinology, Zürich using the PerkinElmer Wallac AutoDELFIA (Wallac Oy, Turku, Finland) neoTSH and neo17OHP kits.
The program uses an interview mode. After entering the test name the first question is about the type of QC rule to use. In the case of TSH the choice is usually for traditional statistical rules. For T4 or 17OHP the "average of normals" may be an alternative if large enough number of samples are analyzed.
The interview mode of the software then asks for the quality requirement - as mentioned above the use of the neonatal tests is best based on the clinical decision interval. For the neoTSH assay the Zürich screening laboratory uses 15 - 25 mU/L blood. Therefore the decision level is 15 mU/L TSH and the interval (10 mU/L) is 66.6% of the decision level.
Pre-analytical factors can also be taken into account, e.g. biologic variation, sampling variation and specimen bias. Biologic variation can account for a known group or within-subject variation. Data on TSH within-subject variation for neonates is not available, but for adults it is given as 19.7 %, and is used in this example. The other pre-analytical factors can be ignored, since the performance data have been obtained from filter paper samples and thus the sampling variation and specimen bias are already included in them.
The performance data are then entered. These can be either the data provided by the manufacturer in the kit insert or, as here, from data generated in the laboratory. In the Zurich screening laboratory the CV for the neoTSH assay at a level of 16 mU/L TSH is 7.3% (derived from 150 assays). For neo17OHP the CV is 10.6% at a level of 26 nmol/L blood. The instability of the assay is a measure of overall problems with the kit - if the kit performance is stable with infrequent problems, < 2% can be entered.
The program then asks for the number of controls to be used - the number of controls supplied with the neonatal kits is usually 2 (a low and high, normal and abnormal), but the laboratory can of course include in-house controls. After entering the number of controls, the program automatically generates a QC rule and the OPspecs chart.
The figure below shows the generated OPSpecs chart, with the allowable imprecision (as %CV) on the x-axis and allowable inaccuracy (as bias) as the y-axis. The "Operating Point" shows the position of the method in use. In the present example the chosen number of controls was 2, and the rule selected by EZ Rules program was 1:3s, which is shown as the bold line. However, the program generates a series of alternative QC procedures and gives the corresponding control rules in the table on the right in the same order as the graph (from down up).
Two possible control rules are:
- 2 controls and 1 3.0 S rule, i.e. a run is rejected if one of the two controls is outside ± 3.0 SD limits
- 4 controls and a 1 3.5 S rule, i.e. a run is rejected if one of the four controls is outside ± 3.5 SD limits
Both rules give a 90% error detection (Analytical Quality Assurance) and no false rejections, but many laboratories may prefer to use two controls, as it leaves more space for samples. N can be either the same control run N times or N separate controls. R is the number of runs that the control can be applied to, and would in the AutoDELFIA assay be one plate, as each plate should have its own controls.
With more precise and automated assays a quantitative approach to quality control becomes both possible and essential. Defining a grey zone is one way to approach the quality requirement - defining only one cut-off does not allow for any variation in the assay, or if defined conservatively it leads to many unnecessary repeats.
The example shows that with the performance of the AutoDELFIA neoTSH assay the needed quality control can be 2 controls and a 1 3.0 S rule. The two controls can be either the same control twice or two different controls. Runs should be rejected if one of the two controls is outside the 3.0 SD limits. An additional benefit of the program is the QC validation report, which can be commented and printed for documentation for e.g. accreditation purposes.
Many laboratories may lack a documented definition of the required quality and a 2 SD rule is often in use. A 2 SD rule leads to many unnecessary rejections as statistically 5 % of the runs will be outside even when everything is OK. This means too many false alarms, too many rejections of runs and too many repeats. The EZ Rules program provides a tool for a solid basis for selection of QC rules and security and a documented report.
The presented approach gives QC with scientific basis, and gives more confidence, less rejected runs and cost savings.
An additional new feature of EZ RulesTM is the ability to chart the Sigma value of a test. In this case, we can see the Sigma-metric of this method is 5.40, which is an excellent achievement. To learn more about Six Sigma and Sigma-metrics, click here.
- American Academy of Pediatrics. Pediat 1993;91:1203-1209.
- Westgard JO, Petersen PH, Wiebe DA. Laboratory process specifications for assuring quality in the U.S. National Cholesterol Education Program. Clin Chem 1991;37:656-661.
- Westgard JO. Charts of operational process specifications ("OPSpecs Charts") for assessing the precision, accuracy, and quality control needed to satisfy proficiency testing criteria. Clin Chem 1992;38:1226-1233.
- Westgard JO. Assuring analytical quality through process planning and quality control. Arch Pathol Lab Med 1992;116:765-769.
- Westgard JO, Stein B. Automated selection of statistical quality-control procedures to assure meeting clinical or analytical quality requirements. Clin Chem 1997;43:400-403.
- Westgard JO, Stein B, Westgard SA, Kennedy R. QC Validator 2.0: a computer program for automatic selection of statistical QC procedures in healthcare laboratories. Comput Method Program Biolmed 1997;53:175-186.
For more information on PerkinElmer Life Sciences products visit the website:http://www.perkinelmer.com
e-mail addresses authors:
else-maj.suolinna@perkinelmer.com
torret@kispi.unizh.ch
Why gray zone? Take the actual clinical cut-off value for congenital hypothyroidism as 15 mU/L TSH. A method with a 10% CV will give values of 12 - 18 mU/L (95% of results), even when everything is running well. Thus one would either have to reduce the cut-off, which would increase the false positives, or define a gray zone, for which e.g. repeating the sample in duplicate would confirm the result.
To download a Word document containing the full set of EZ RulesTM screenshots, click here.
