|Managing Quality in Networked Laboratories|
Designing QC for a single test can be a challenge. Imagine the task of selecting the appropriate QC for more than 70 tests on multiple instruments across multiple laboratories. In this essay, Nuthar Jassam and colleagues describe how this was actually done at a network of laboratories in the UK.
Managing Quality in Networked Laboratories:
Variation around the
cut off at
the cut off at
the cut off at
the cut off at
|ALP||300 iu/L||52 iu/L||258-342||262-338||258-342||253-347||8.0|
|LDH||430 iu/L||74 iu/L||355-505||355-505||349-511||339-521||6.0|
|K||4 mmol/L||0.35 mmol/L||3.6-4.4||3.6-4.4||3.6-4.4||3.5-4.5||3.4|
|AFP||5 iu/L||1.6 iu/L||4-6||4.0-6.0||4.0-6.0||4.0-6.0||3.6|
|Ca 125||10 ku/L||3.5ku/L||7-13||7.0-13||7.0-13||7.0-13||4.8|
|CEA||10 g/L||4 g/L||7-13||7.0-13||7.0-13||7.0-13||4.8|
|FSH||5 iu/L||1.3 iu/L||3.8-6.2||4.0-6.0||3.9-6.1||3.8-6.2||3.3|
|Total protein||70g/L||3.5 g/L||66-74||66-74||66 -74||65-75||2.4|
|TT3||3 nmol/L||0.75 nmol/L||2.1-3.9||2.5-3.5||2.4-3.6||2.4-3.6||1.7|
Analyte specific control rules were derived for 68/71 tests performed in the core biochemistry laboratories in the network. A total of 35/71 (50%) analytes were controlled by a single rule only. Thirty of these were for analytes with TE based on biological variation. The new rules were more relaxed than those originally used (12S) in the laboratories (32 analytes with 12.5S or 13S or 13.5S at N= 2 and 3 analytes with a 12.5S and N=4). These analytes were alkaline phosphatase (ALP), alanine transferase (ALT), amylase, gamma glutamyl transferase (GGT), creatine kinase (CK), C-reactive protein (CRP), cholesterol, HDL-cholesterol, creatinine, triglycerides, conjugated bilirbin, total bilirubin, lactate dehydrogenase (LDH), iron, lactate, uric acid, urea, Ca-19-9 and Ca-125, troponin (TnI), prostate specific antigen (PSA), salicylate and all tests for urine chemistry except urine albumin.
Eight out of 71 analytes had multi-rules 13S/22S/ R4S/ 41S at N=4, these were albumin, calcium, chloride, potassium, lithium, prolactin, thyroid anti-peroxidase (TPO) and thyroid stimulating hormone (TSH).
Maximum control rules 13S/22S/ R4S/ 41S / 8X at N=4 were required to control the performance of 11/71 analytes. These analytes were glucose, bicarbonate, magnesium, sodium, total protein, progesterone, testosterone, total triiodothyronine (TT3), total thyroxine (TT4), carbamazipne, and urine albumin.
14/71 analytes required different controls rules on different sites. These analytes were conjugated bilirubin, cholesterol, HDL-cholesterol, uric acid, α- feto protein (AFP), carcino-embryonic antigen (CEA), cortisol, luteinising hormone (LH), follicle stimulating hormone (FSH), human chorionic gonadotrophin (hCG), parathyroid hormone (PTH), paracetamol, theophylline, urine protein and urine uric acid. The requirement for different control rules was due to the difference in observed bias and imprecision across sites e.g. cholesterol; this analyte required 12.5S, N=2 on three sites and 13.5S, N=2 on one site. Serum uric acid required a single rule to control its performance on three sites and maximum rules 13S/22S/ R4S/ 41S /8X at N=4 on one site.
Three analytes had unacceptable performance and were impossible to control by any control rule; these were free thyroxine (fT4), digoxin and oestradiol.
Using the biological variation derived TE specifications, we found that 50% (23) analytes perform at a level of 5 sigma or higher, 20% (9) analytes performed at sigma varying between 4 and 5, 20% (9) analytes performed at 3-4 sigma and 10% (5 analytes) performed at sigma less than 3. Figure 2 gives the analytical performance of the biological variation based quality specification analytes in terms of sigma (σ).
Figure 2: The system capabilities given in sigma for the 46 analytes that have a quality specification based on biological variation.
The probability of false rejection was set to be maximally at 5% for the new rules selection. The 12s rule usually has a false rejection of 9%. Table 2 shows ped and pfr with the new rules and with the 12s rule for the general chemistry only.
Until recently we had used a simple Westgard rule system 12s (mean ± 2 SD) for controlling analysers on each site across our multi-site laboratory network. As described previously this was not satisfactory for the control of analysers across the network. Furthermore, this rule has Pfr of 9% at 2 control measurements . This high false rejection rate would render this rule a major waste of laboratory resources due to repeat analysis of controls, and patients’ samples resulting in an increase in the cost of the analytical process and a waste of time and effort. If we considered the number of analytes measured and the volume of testing performed on a daily basis in our network of laboratories, which exceeds 20,000 tests per day, one would assume this could be a significant waste of the laboratory budget.
This type of waste can be avoided by designing a quality control procedure that is based on the quality goal required clinically and the performance characteristics of each test/analyser. Therefore, the laboratory's efforts would be focused on the analytes that require the maximum control. The ideal IQC design should be derived for each individual test in a multi-test system, selecting where possible the combination of the highest Ped and the lowest Pfr . In this study, the quality control rules were selected at the Pfr has been set at a maximum of 5%. This has resulted in a cumulative reduction in the Pfr by more than 50% (Table 2).
|Analyte||New Rules||Old Rule: 12s|
|Systematic error||Random error||Systematic error||Random error|
Table 2: The probability of error of detection (PED) and the probability of false rejection (PFR) of the new quality control rules in comparison to the OLD QC RULES (12s rule).
The ALP and total protein examples are highlighted. ALP: the QC rules selected for this analyte will give the probability of systematic error detection of 89% and random error detection of 61%. The false rejection for these rules is zero. The current rule 12s has higher error detection but almost one run in ten has a false error. For total protein, both the new rules and the current rule have poor error detection. However, the false rejection in the current rule is over 40% ( 9/21). This means that over 40 times out of 100 runs the analyser would have been stopped for a false error flag.
It was also found that 19 analytes required multi-rules or maximum rules to control the analytical process required to satisfy the quality goal. This shows that the initial 12s rule alone has not been efficient enough to control the performance of these analytes, hence the variability in analytical performance between sites. Furthermore, 14 analytes required different quality control rules on different sites, or on different instruments within the same site. As the same analyte has the same TE across all sites, the difference in the quality control rules required stems from a difference in the performance indicators of these analytes, which as described previously can be ascribed to varying levels of hardware deterioration of the instruments on different sites. This can be due to different workloads or an inherent difference in the design of those instruments.
The new quality control rules rely on a scientific approach, taking into account the quality goal and the observed performance of an analytical method. For example, if the analytical process is more precise than medically required (e.g. iron, TE= optimal, σ= 7.5) the quality control process can be relaxed and would only need one rule such as 13s to achieve the required quality goal. Controlling these analytes with 12s or multi-rules would have resulted in a waste of resources. However, assays with unsatisfactory performance (low system capability or low sigma) may require the maximum control rules to achieve the desired goal. Our data showed that for 50% of total analytes measured in the core biochemistry a more relaxed rule than the initial rule in use could be used. This shows that this control rule design provides a cost effective model because the decrease in the cost of controlling the analytes with more relaxed control rules is transferred to those tests that require a higher degree of control.
World class quality is generally recognised as a 6 sigma process (3.4 defects per million on the sigma short-term scale). The 3 sigma performance (66,807 defect per million on the short-term scale) represents the minimum standard and defines the region of unacceptable performance . In the clinical laboratories improving processes to achieve a performance above 5 sigma (233 defects per million on the sigma short-term scale) is of little advantage and can be costly. Performance at 4 sigma (6,210 defect per million on the sigma short-term scale) is considered satisfactory for the clinical laboratory if the process is adequately controlled . Using the biological variation based TE resulted in achieving performance of 4-6 sigma for 70% of the analytes. However, it is important to recognise that the size of critical error (systematic/random) or sigma index depends on the allowable TE and the performance of the measurement system. A smaller total error or poor CV and bias would result in a small sigma or critical error. Biological variation based quality specifications represent a narrow form of quality specifications, which means that we have to detect a smaller but clinically relevant change in the analytical process. In this case, achieving a smaller sigma but a narrower quality requirement is better . Our data showed that 30% of the studied analytes had a performance consistent with a poor sigma (σ ≤ 4), however, most of these analytes had a CV within the biological variation for all the sites. For example, glucose had an average analytical CV ≤ the allowable CV (2.9% (desirable level)) on all sites. Given that the TEallowable is 6.9%, a CV of 2.9% would result in performance of 2.4σ, whereas improving the analytical performance to achieve a CV of 1.4% (optimal CV), which meet 5σ performance remains a goal for this analyte.
On the other hand, for some analytes that were allocated to the minimal level due to current unsatisfactory performance, achieving 5 sigma would be challenging without an improvement in the method technology and formulation.
In our study, we presented a central quality system that enables the identification of poor performance in the analytical process in a retrospective manner . The system was based on quality specifications derived from biological variation data and software that enabled the application of common specifications and across laboratory analytical performance comparison. In this paper, we have added the last dimension to this system. This was based on deriving analyte specific quality control rules that maximise the detection of analytical error during the analytical stage, and helps to maintain a stable analytical performance.
As far as we know this is the first attempt to test the practicality of quality specifications based on biological variation to control the analytical performance across a multi-site network. We have demonstrated that the biological variation model is a practical model with which to derive quality control specifications that are related to the clinical requirements of a test and we have designed quality control rules to maintain the stability of the analytical process
Biological variation based quality specification: Glucose
The formula for calculating the dispersion that encompasses 95% of values is:
CVTotal = ± 1.96 (CVa2+CVI2)1/2
SDTotal = CVTotal × test concentration / 100
95% dispersion interval = test concentration ± SDTotal
The ‘current' data field represents the worst imprecision over a 6 month period across all the analysers in the network.
The CVI values for glucose, 5.7% [12 ].
The effect of imprecision at the three performance levels around a glucose concentration of 7.0 mmol/L.
|Performance level||CV%||%CVTotal||95% dispersion interval
at glucose of 7.0 mmol/L
|Optimal||1.4||11.5||6.2 - 7.8>|
|Desirable||2.9||12.5||6.1 - 7.9|
|Minimal||4.3||14.0||6.0 - 8.0|
|Current||2.9||12.5||6.1 - 7.9|
Glucose is reported to one digit after the decimal point through the whole range. Our current performance lies within the desirable level. However, the optimal level gave the least dispersion interval. A cut off of around 7.0 mmol/L glucose is used diagnostically, hence we believe that the optimal performance level is the ideal target to achieve clinical decision making.
Total error estimation based on expert opinion: Prolactin
Biological variation data for prolactin (PRL) is not available. TE was estimated on the basis of the tolerance around a clinically relevant cut off value and current performance. The cut off for PRL was selected as the upper limit for monomeric PRL reference range (438 mu/L) .
Allowable CV= 10% (based on an average of 6 months performance).
Allowable bias: 10% (The 75 quartile of the method group in NEQAS scheme).
Allowable TE= Bias + 1.65CV
Allowable TE= 26.5%
Quality specifications based on pharmacokinetics theory: phenytoin
Some quantities that are measured in clinical biochemistry do not exhibit biological variability such as drugs. For these analytes, Fraser et al. derived a formula to calculate the desirable analytical goal for precision based on the fundamental pharmacokinetics theory . Phenytoin is an anti-convulsant drug with a half life in chronic administration of between 6 and 24 hours. .
|Half life||%CV analytical < 1/4 x [(2T/t - 1)/ ( 2T/t + 1) x 100|
In premature babies, phenytoin has a long half life due to the immaturity of the liver. The longest half life was used to derive a quality specification for this analyte, because it can deliver the most stringent analytical goal. Therefore the desirable CV is 8.3%.
Method bias= 10% (estimated from the External Quality Assurance Scheme).
TE allowable= B+1.65 CV
- Westgard JO. A method evaluation decision chart (MEDx chart) for judging method performance. Clin Lab Sci 1995; 8: 277-283.
- Westgard JO. Internal quality control: planning and implementation strategies. Ann Clin Biochem 2003; 40: 93-611.
- Westgard QC, Inc,. EZ Rules ® 3 automatic selection of statistical control rules for laboratory tests. Madison, WI: Westgard QC. Inc,. 2005.
- Housley D, Kearney E, English E, et al. Audit of internal quality control practice and processes in the south-east of England and suggested regional standards. Ann Clin Biochem 2008; 45: 135-139.
- Jassam N, Lindsay C, Harrison K et al. A system for managing analytical quality in networked laboratories: the implementation. [ in preparation]
- Fraser CG, Hyltoft Petersen P, Libeer JC, Ricos C. Proposal for setting generally applicable quality goals solely based on biology. Ann Clin Biochem 1997; 34: 8-12.
- Fraser CG. Desirable standards of performance for therapeutic drug monitoring. Clin Chem 1987; 33: 387-389.
- Westgard lesson. Six sigma quality management and Desirable laboratory precision. www.westgard.com/essay35.htm (accessed 2/12/2009).
- Westgard JO. Six sigma quality design and control: desirable precision and requisite QC for laboratory measurement process. Madison, WI: Westgard QC, inc., 2001.
- Westgard lesson. www.westgard.com/a-six-sigma-design-tool.htm (accessed 2/12/2009).
- Campbell BG. Evaluation of two types of medically significant error limits and quality control procedures on a multichannel analyser. Arch Pathol Lab Med 1989; 113:834-7.
- Ricós C, Alvarez V, Cava F, et al. Desirable quality specifications for total error, imprecision, and bias, derived from biological variation. www.westgard.com/biodatabase1.htm (accessed on 6/09/2009).
- Jassam NF, Paterson A, Lippiatt C, Barth J. Macroprolactin on the adiva centaur: experience with 409 patients over a three-year period. Ann Clin Biochem 2009; 46:501-504.
- Fraser CG. Desirable standards of performance for therapeutic drug monitoring. Clin Chem 1987; 33: 387-389.
- Mike Hallworth and Ian Watson. Ed. Therapeutic drug monitoring and laboratory medicine. ACB Venture Publications 2008.
|< Prev||Next >|