Quality Requirements and Standards

As part of the Westgard QC global survey of analytical quality goals, we provided an opportunity for all respondents to make open comments. See the words beyond the survey answers.

Global Analytical Goal Survey Comments

Sten Westgard, MS
January 2015

While we collected more than 450 responses from 80 countries in our analytical goal survey, we didn't want to limit the responses only to the answers we offered for each question. With every question, we also included an "other" or "none of the above" option so that each respondent could customize their response, if they wanted. We also included a final open-response area at the end of the survey, so that any additional comments, statements, criticisms, etc. could be made.

Now, we're not going to reprint all of the comments here. Some of them were simply laudatory and not necessary. Others ("Nil" and "None") are not necessary to include for obvious reasons. So here we will print the relevant comments of the survey participants:

Start with the Final Word: the open-ended comments of survey takers (59 responses recorded)

"Review the analytical goals for sodium, it's too tight and almost impossible for some analytical systems."

"Total error is not homogeneous over the measuring range, therefore it is difficult im my opinion to define 1 total error goal which is representative for the whole measuring range. This is a difficulty of the Ricos database. In addition, certain quality goals are too strict for the current analytical quality of even very classic chemistries."

"Update CLIA guidelines would provide the greatest overall impact to improving laboratory testing and consistency. I also feel laboratories would benefit from receiving more education on employing evidence-based control ranges that lead to a high error detection rate and low false rejection rate rather than utilizing all-encompassing ranges - and reiterating that this practice is not isolated to hematology or chemistry but equally useful in all areas where patient testing is conducted."

"Time is there for implementation!"

"Quality / performance goals developed per test that vendors must meet. This would create a common standard to measure performance of a method and help in choosing the appropriate method for your client."

"I support the development of a model that mixes the technical capablity with the clinical intended use."

"Adopt analytical goals to improve test quality and at the same time more practical for laboratories."

"I focus on 6-sigma Westgard Rules. It is easy to use."

"Guidelines regarding the acceptable criteria for tests under validation must be standardized between different clinical laboratories. Currently there are no clear cut-off limits are available. Therefore the acceptable criteria are different between different laboratories for each test. If these criteria unified then the acceptable performance for each test can be easily decided i.e the same test can be accepted in one lab but not in other lab."

"More education about the use of analytical quality goals."

"I find it difficult to monitor total error periodically (e.g., monthly) due to limited software tools."

PT and EQA

"PT results often identify methods which lack harmonization but we don't have any incentives for manufacturers to resolve these issues."

"At present our main example of this subject is in the reports we receive as a part of the information given in the EQA report for a selected number of analytes."

Beyond Chemistry Assays

"Analytical quality goals are focused in chemical and hematological tests. I would like to have more support in Immunologic tests, where is so much difficult to apply quality control, in particularly for infectious disease."

"Understanding more about what IQC rules should be used for what particular immunoassay test."

"Quality recommendation for POC analysers."

"My job is, mostly, DNA diagnostics (qualitative and quantitative PCR). Therefore, we rely on manufacturer's controls and external control samples, if available for specific gene(s)."

"Some novel tests do not have quality goals (TEa). I hope that expert commitees will be established to update the CLIA quality goals to include those novel tests."

Developing Countries

"For resource poor countries, first there should be training and capacity buildings for analytical quality, quality management system exist in some private facilities and almost all public facilities are without quality management systems."

"Developing countries like mine, need more information and support."

"As we are developing country we quality of the lab is not good enough and we have lack of knowledge, experience. We are looking for more training opportunity."

"We from the developed countries really appreciate what Westgard Inc has been doing in terms of assuring excellent quality in lab results. It is just that we sometimes are not confident enough in justifying whether or not we are doing the right things right!"

"More simplified online workshop for developing country about the utilization of suitable quality goals."

"Focus in the developing countries with in use the methods but still not perform method verification."

Expert Groups

"Hold meetings to share experiences experts practices based on real statistical data from different countries, benefits, advantages and limitations in the application of the analytical objectives."

"Build reliable evidence base for state-of-the-art performance!"

"I can help in an International Group to develop and Formalize the use of biologic maximum allowable CV/Total Error as the goal for target maximum measurement uncertainty."

"I would like to find in the bibliography approved method to select analytical goals."

"Selection of the right goals is a very difficult task. Really need more expert groups to set the analytical goals to determine quality needed for clinical use."

"1-More practice trouble shooting communication between experts is highly recommended for labs with the same methods/machines
2-relating the analytical goal to the clinical decisions is the most important point that needs intensive investigations"

The Clinical Challenge

"Try and get the lab staff out of the lab and understand they are on a cog in the patient care wheel. An essential cog but if they do not understand the system much of what they do will be very under valued."

"Something needs to be addressed with respect to biological variation of specific analytes and interpretation of reference interval near the cut points to convince the clinicians that the reference interval is not written in sand stone."

Beyond the Analytical Phase

"Quality control over all steps in and out lab."

"More support needed on quality measures for reagent storage and pre-use procedures."

"Forget measurement of pre and post analytical processes, the variation is too wide."

Bias and Uncertainty

"For routine measurement procedures there are several causes that generate bias. When bias can't be corrected ( and such situations are quite frequently!) its value must be considered as a term for calculation of the measurement uncertainty."

"Harmonization and publication of Measurement uncertainty estimation protocols. the calculations also need to be simplified."

"This survey is excellent for analytical goals. I need target Measurement Uncertainity, how will be decide that Measurement Uncertainty of our lab is acceptable or not."

Beyond the Laboratory Walls

"Improving quality has a cost and who is going to invest in this?"

"Lab technicians and higher ranking people at hospital level and ministry of health do not understand the impact of not performing IQC as a routine. So ...?"

"Clinician understanding of quality required for intended use of a test is lacking. My impression of clinician understanding is that patients get an exact test result that would be the same if it was run 50 times and is commutable across all institutions. A clinician focused lab test 101 education is greatly needed. It is tough to begin the discussion of quality requirements and decision level without a basic understanding of laboratory clinical pathology testing."

"There has been very limited interest in this type of information from users of the clinical service. As well as improving laboratory performance we need to make sure that users , not just lab-staff, are aware of what analytical quality means."

Miscellaneous

"Dr Raymond Gambino developed a proprietary quality program called Smartech over a decade ago, it was (and is) ahead of it's time. It tracked patient moving averages and all QC results real time to develop tight requirements for instrument performance. Eloquent solution!"

"Inspectors of JCAHO and CAP are still stuck in the dark ages of 2SD limits. They have no clue as to how QC has evolved."

"Not enough information on how to best utilitze RiliBak and Sigma six to monitor hematology parameters."

Comments from Question: What Support would you want in the Utilization of Analytical Goals?

"allow understanding of assay performance to affect clinical decision making (eg monitoring of a patient with different methods when it is safe to do so, and not when method differences are too large)"

Comments from Question: What Changes would you like to see in Analytical Goals?

"Performance goals for 510k"

Comments from Question: What sources do you use to obtain your Quality Goals?

"From other ISO certified labs"

"UCAS"

"Westgard site"

"ISO 17025:2005; Nordtest and Eurochem"

"We work very closely with doctors on all issues and the clinical use and interpretation of data as well as chart review is a constant source for our goals . We also actively encourage our physicians to complain about all lab issues we result in a full investigation. This has led us to review quality goals on multiple occasions"

"Ricos Biodatabase"

"IFCC regulations and CAP PT specifications"

"Some labs still not perform IQC routinely"

"In-house Lab-Developed specifications from clinical guidelines OF Portuguese Heatlh Authorities usina Biological Variation Tools such homeoatatic point calculation, relevant pathophysiological Variation and maximum uncertainty measuramet alloable"

"UK Royal College of Pathologists MAPS"

"www.westgard.com"

"The Quality goal for a test is reviewed using all above to determin an appropriate and clinically relavent goal"

"Especificaciones estado de arte grupo español"

"Labquality ltd"

"Assay dependent"

"ASVCP guidelines"

"WLSH AAB"

"HPA Colindale"

Comments from Question: What Types of Quality Goals do you use?

"We use different specifications for different analytes"

 Comments from Question: How do you use Quality Goals in your laboratory?

"Our customers use quality control kits for lab tests internal quality control"

"Evaluating new reagent & calibrator lots"

"Training"

"Some of labs in Cambodia only"

"All areas addressed in the QC workshop"

"Validation of POCT"

"Deciding which methods most need improvement."

"Measurement uncertainty"

"Fulfill corporate quality goals"

"Only a few actually do at this stage, many are starting to show interest"

"Instrument comparison studies, changes in lot numbers"