ISO

Contributing editor Dr. Pereira continues part 2 of a series on the ISO standards applicable to medical laboratories. The ISO 15189 is widely popular for laboratories, but many aspects are confusing, vague, and misunderstood. Dr. Pereira shows how to interpret the standard's requirements.

ISO series update

Part 2 - ISO 15189:2012 “Medical laboratories - Requirements for quality and competence”

Paulo Pereira, PhD
February 2017 Updated January 2020

Recap

ISO standards are intended to standardize practices globally. Unfortunately ISO implementation is frequently accompanied by misunderstandings. This series discusses the pros and cons, and some myths regarding the ISO standards’ implementation in medical laboratories. It will be divided into five parts:

• Part 1 - ISO 9001:2015 “Quality management systems - Requirements”
• Part 2 - ISO 15189: 2012 “Medical laboratories - Requirements for quality and competence”
• Part 3 - ISO 10012:2003 “Measurement management systems - Requirements for measurement processes and measuring equipment”
• Part 4 - ISO 19011 “Guidelines for auditing management systems”
• Part 5 - ISO 15190: 2003 “Medical laboratories - Requirements for safety”

Occasionally, the reader of an ISO standard may be challenged to identify what is mandatory and what is not required. ISO 9001 makes the reader’s life easier by noting in the introduction that “shall” specifies a requirement, “should” specifies a recommendation, “may” specifies a permission, and “can” specifies a possibility or a capability. Laboratories must to all “Shalls” but all the others are not mandatory.

Purpose

This text is probably the most challenging on the “ISO series update” since many reviews have been published related to ISO 15189 3rd edition [1]. Therefore, this essay is intended to discuss some specifications briefly and to debate what is happening with ISO 15189 implementation in the world. The standard document is focused on the medical laboratory, and its goals can be interpreted as the satisfaction of interested parties (4 of [1]). Stakeholders cannot be understood as customers only, such as patients, but inclusive of any internal or external involvement with the medical laboratory, including, but not only, professionals, suppliers and accreditation agencies. “The customers’ satisfaction” can be understood as the contribution of the reported results to an accurate clinical decision. For this purpose, this international standard is based not only on a management system but also on a set of medical laboratory technical specifications. Despite the lab sustainability should be controlled, it is not mandatory. Such as common in others ISO standards, “shall” stipulates a requirement, “should” specify a recommendation, “may” instructs permission, and “can” suggests a possibility or a capability. Only the “shall” signifies that a particular specification is compulsory.

The approach

ISO Technical Committee (TC) 212 has formed the ISO 15189 (Medical laboratory testing and in vitro diagnostic test systems) Working Group (WG) 1 (Quality and competence in the medical laboratory). The ISO 15189 debut edition [2] was published after a three-year hiatus from the final draft and four years after the publication of ISO/IEC 17025 first edition [3]. It quickly became a widely-accepted standard to be used for accreditation of medical laboratory competence. The 2nd edition [4] was published in 2007 to provide the same structure as ISO/IEC 17025, intended for testing and calibration in general laboratories. ISO 15189 can be viewed as the “ISO/IEC 17025” for medical laboratories. When a medical laboratory chooses an accreditation plan, it should select an accrediting body that operates according to appropriate international standards and which takes into account the particular requirements of this field. Before 2003 medical laboratories could be accredited according to the ISO/IEC 17025 approach, and they were able to change to ISO 15189 at their option. Depending on the local accrediting bodies it may be possible for a medical laboratory to choose between ISO/IEC 17025 and ISO 15189 or even to have both accreditations. For instance,  when a medical laboratory has ISO 15189-accredited tests and also has a calibration method intended to calibrate not only internal devices (which does not require an ISO/IEC 17025 accreditation) but also equipment for external customers.

The difference between ISO 9001 and ISO 15189 approach that is immediately recognized is the presence of medical-technical laboratory requirements in ISO 15189. The framework provides a quality management system close to the ISO 9001:2008 management requirements added by specifications for technical competence that are particular to medical laboratories.

The standard quality management model is based on the Deming TQM approach [5] [6] [7]. Figure 1 displays a quality cycle applicable to a medical laboratory under ISO 15189 accreditation. The leadership is critical to the success of all the cycle phases. Probably, on the next guideline revision, “customers” will be replaced by “interested parties,” including not only the clients but others that need to verify lab practices, such as the regulatory and accreditation agencies. This terminology is already revised in the current ISO 9001 edition (4.2 of [8]).

ISO 15189 technical requirements are applied for personnel, accommodation and environmental conditions, laboratory equipment, reagents, and consumables, pre-examination processes, examination processes, ensuring the quality of testing processes results, post-examination processes, reporting of results, the release of results, and laboratory information management. Table 1 summarizes these stipulations.

 Table 1: Summary of ISO 15189 specifications

Sub-chapters 5.3, 5.5, and 5.6 require specifications for which there is not a harmonization of practices - note that all the results are recorded, and its traceability is assumed:

a) 5.3.1.4 Equipment calibration and metrological traceability

The medical laboratory participates in programs to calibrate and verification of trueness, i.e., to determine and verify bias (systematic error analysis) defined as “the difference between the expectation of the test results and an accepted reference value” (2.18 of [9]). Measurement Precision (random error analysis) is also measured and verified. It is defined as “the dispersion of independent results of measurements obtained under specific conditions, is expressed such as standard deviation or coefficient of variation”(2.15 of [9]). Preferably, traceable metrological materials should be used. When these materials are not available, or their use is not significant to the estimate accuracy, alternative materials could be used. See for a more in-depth discussion see [10-12].

b) 5.5 Examination processes require:

• 5.5.1.1 Selection of examination procedure: New tests are selected on the basis of clinical purpose (intended use, fit for purpose). For instance, a screening test selection in a blood bank should assure that a method with high diagnostic sensitivity [13] is chosen to minimize the residual risk (2.29 of [14]) of post-transfusion infection. ISO 15189does not recommend any approach to select a new test. Usually, it is based on a literature review using validation cases of state-of-the-art methods.
• 5.5.1.2 Verification of examination procedure: All tests used without modification - “non-waived” tests - are verified using performance information data available from the manufacturer. The verification shall confirm with evidence that the laboratory performance claims have been met. The calculations are based on experimental data. The methodology is presented in guidelines CLSI EP15-A3 [15] for quantitative tests or CLSI EP12-A2 [16] for qualitative tests.
  
  
• 5.5.1.3 Validation of examination procedure:  Modified (“waved”) or “in-house” tests require a more complex validation process. Also, “standard methods used outside their intended scope” shall be validated. Again, these tests are validated according to the clinical test purpose/intended use/impact of the result on the clinical decision. Therefore, the specifications, such as the allowable total error, allowable diagnostic sensitivity or allowable diagnostic specificity are selected accordingly. ISO 15189 does not state any specific statistical approach for method validation, and it does not state any goals/targets/claims. As with verification of non-modified tests, experimental data is involved with validation of modified tests. The methodology is as complex as needed, and it is identical to what is required to the manufacturer in the validation phase. CLSI guidelines referred to in 5.5.1.2 can also be used, but further studies are needed. For example, the calculation of a clinical decision point based on representative samples of the population.
  
  
• 5.5.1.4 Measurement uncertainty of measured quantity values: This requirement is just for tests expressing quantitative results, as suggested by the subtitle. For qualitative results, measurement uncertainty cannot be computed. Even if the qualitative results are expressed on an ordinal scale according to cutoff, its determination is optional. ISO 15189 does not recommend a methodology to calculate measurement uncertainty. However, since ISO is a member of the working groups of the Guide to the uncertainty of measurement (GUM) [17] and the Vocabulary of international metrology (VIM) [9], it is presumed that an “Uncertainty approach” ([17]) model is mandatory. Empirical models should be used, preferably using data from the validation of the examination procedure phase. It is not recommended the use of external quality assessment (EQA)/proficiency of testing (PT) data as a primary source due to the heterogeneity of the results [18]. However, the EQA/PT is probably the most common model in medical laboratories with tests accreditated to ISO 15189. It does not comply with the standard since usually, this determination is not according to the results' intended use (clause 5.5.1.4 of [1]). ISO 15189 2nd edition required that measurement uncertainty is determined when its result was “relevant” and “possible” (5.6.2 of [4]) “Relevant” could be interpreted as when the outcome has a significant clinical value, and “possible” synonymous of the availability of a mathematical model. One such model available to labs is the total analytical error based on the “Error Approach” (also documented as Traditional Approach or True Value Approach). Measurement uncertainty is an additional model that is part of the evaluation of the quality of the reported result, which could be applied to any measurement. Target uncertainty must be defined, which can be very complex. See [19] for further details. Measurement uncertainty implementation has not been widely successful in medical laboratories, even after 23 years since GUM was published, such as demonstrated by the Westgard QC 2015 survey [20]. Note that the measurement uncertainty evaluation can be interpreted as redundancy of verification and validation, just differing due to the use of the “Uncertainty Approach.” For further details about models to determine measurement uncertainty in medical laboratories see [21].

c) 5.6 Ensuring quality of examination results

• Internal quality control: Design of internal quality control scheme is encouraged and implied, but no approach is recommended. Most of the methods used in labs are applications of the "Westgard Rules,” however just using a multirule QC approach does not automatically fulfill the principles [22], and around the world the practice of QC is not harmonized. An alternative to the Levey-Jennings charts could be used, such as the exponentially weighted moving average (EWMA) chart (9.2 of [23]). Nevertheless, the practical suggestion is to use statistical QC based on the Sigma-metrics [24] principally because it relates the determined total error with the allowable total error. The allowable total error is equivalent to the error that does not significantly contribute to incorrect clinical decisions. For a depth discussion on this issue, please see [25].
  
  
•  External quality assessment: Medical laboratories shall participate in programs for EQA/PT and shall provide evidence of corrective actions when results are out of control. For instance, when a result is out of acceptable group requirements. There are no specific recommended approaches. The use of the EQA/PT approach is sometimes misunderstood, because some calculations based on this data may be inconsistent. For example, EQA/PT could be an unreliable source for estimates of bias if the peer group’s results have a large discrepancy.

Figure 2 represents the steps from the test selection to the reported results. The accomplishment of the examination and post-examination phases are dependent on the pre-examination stage.

Frequently-Asked Questions about ISO 15189

Which books are suggested to support the ISO 15189 quality management system?

Principally two publications: David Burnett, Ph.D. “A Practical guide to ISO 15189 in laboratory medicine” (2013), and James Westgard, Ph.D. and Sten Westgard, M.Sc. “Basic quality management systems” (2014).

Which references can support ISO 15189 specifications on examination and post-examination activities?

a) Method selection

See Westgard QC lesson no. 20 Selecting a method to validate and Basic method validation 3^rd ed. (2008) book

b) Method verification and validation

Precision:

-Detection limit: See Westgard QC lesson no. 29 The detection limit experiment, Basic method validation 3^rd ed. (2008) book, and CLSI EP17

-Precision components: See Westgard QC lesson no. 22 The replication experiment, Basic method validation 3^rd ed. (2008) book, and CLSI EP15 (also EP5, EP9, and EP19)

-Bias: Proportional and constant bias: See Westgard QC lesson no. 23 The comparison of methods experiment, Basic method validation 3^rd ed. (2008) book, and CLSI EP15 (also EP9 and EP10)

- Bias: Drift and carryover: See Basic method validation 3^rd ed. (2008) book, and CLSI EP10

- Bias: Linearity: See Westgard QC lesson no. 26 The linearity or reportable range experiment, Basic method validation 3^rd ed. (2008) book, and CLSI EP6 and EP10

- Bias: Interferences: See Westgard QC lesson no. 27 Interference and recovery experiments, Basic method validation 3^rd ed. (2008) book, and CLSI EP7 and EP14

-Total error: See Westgard QC lesson no. 23 The comparison of methods experiment, Basic method validation 3^rd ed. (2008) book, and CLSI EP21

-Qualitative assays: See Westgard QC essay Basic validation of qualitative tests, Statistical methods in diagnostic medicine. 2nd ed. (2011) book, and CLSI EP12 and EP24

b) Measurement uncertainty

-Modular approach: See Westgard QC essay Time to engage in MU, GUM, EURACHEM QUAM books, and CLSI EP29

-Empirical approach: S: See Westgard QC essay The Hitch-hiker’s guide to MU in clinical laboratories, Uncertainty of Measurement in Medical Laboratories chapter, EURACHEM QUAM, NordTest TR 537, and EURACHEM Target Uncertainty books, and CLSI EP29

c) Internal quality control

See Westgard QC lesson no. 74 Best practices for “Westgard rules”, Six Sigma quality design and control 2^nd ed. (2006) book, and CLSI C24

d) External quality assessment/proficiency testing

See Westgard QC Quality Requirements, Six Sigma quality design and control 2^nd ed. (2006) book, and CLSI GP27

e) Reference intervals

See Westgard QC essay FAQ in reference intervals and biological variation, Statistical bases of reference values in laboratory medicine (1995) book, and CLSI C28

f) Risk management

See Westgard QC Risk management essays, Six Sigma risk analysis (2011) book, and CLSI EP18 and EP23.