 |
Participant Questions and AnswersThis section of the course is designed by YOU, the participants. You can help improve this course as well as your understanding of QC! If there's a question you want answered and can't find the answer in the course materials, please e-mail us the question. We post appropriate questions and answers as frequently as possible. As the list builds up, we'll try to incorporate them more directly in the course materials. |
View the Outline View the Syllabus View the Archives View the Reference List View the Glossary
Countries represented in the QC Planning Course:Argentina, Australia, Belgium, Brazil, Britain, Canada, Columbia, Denmark, Finland, Germany, Hong Kong, Ireland, Israel, Italy, Mexico, the Netherlands, Norway, Paraguay, Russia, Saudi Arabia, Singapore, Spain, Sweden, Switzerland, South Korea, Taiwan, Turkey, and the United States! |
Professionals interested in corresponding with fellow participants:Robert Broddle orrin@mcc-uky.campus.mci.net
Contact the webmaster if you want your name posted here. |
|
Want to ask a question right away?Go and log onto the AACC Online Forum for the course (you
will need your username and password): http://www.aacc.org/scripts/forums/enter.pl
April 30th, 1998:February 13th, 1998:Some participant questions about multirule QC have been posted on an updated version of the FAQ's about Multirule QC page. December 10, 1997:
November 28, 1997:April 30th, 1998:A group of 25 Norwegian laboratories recently acquired our QC Validator program. We've been expected some questions and here's a good one from Dr. Eikvar. Has anyone thought of using the program to work out limits of allowable bias, when given a TEa, a specified QC procedure, and a CV?This regards the situation of level shifts observed when introducing reagent lot shifts, calibrator lot shifts, or installation of new components on the instrument. In fact, this is an area of increasing concern to us, and we are wondering whether we have to demand that our supplier can comply with defined upper limits of bias for these matters. ANSWER: I think that OPSpecs charts provide a very useful tool to help manage the quality of one system over time (for changes in reagent lots, etc.) as well as managing two or more systems that perform the same tests. The approach is to manage the analytical process to assure errors won't exceed a defined total error requirement. Small changes in imprecision and bias within a system or between systems may be tolerable. The exact ranges can be found by preparing an OPSpecs chart for the defined TEa or defined DInt and for the QC rules and N of interest for the laboratory. As long as the operating point stays within the allowable imprecision and inaccuracy for the selected QC procedure, you are able to manage within the defined requirement. In the case of managing two or more systems that perform the same test, you would need to prepare an OPSpecs chart for each method or manually plot the second or third operating points on the OPSpecs chart. This might lead to different QC procedures for the different methods, but you would still be managing them to the same end quality. Small biases between these systems may not be important in terms of the end quality that you can guarantee. This is not meant as an endorsement of between-system biases. In principle, it is always best to make such biases as low as possible - ideally zero. However, experience generally shows that two models of the same analytical system will show some small differences and that these differences can be very difficult to eliminate. The OPSpecs chart should be helpful for defining what differences are allowable without compromising the end quality of the test results. December 10, 1997:How many levels of control materials are necessary?My understanding is that the practice promoted in the quality-planning models does not require multiple levels of control materials. However, in a recent website lesson on basic quality control - "QC - The Materials", it is recommended that two or three different concentrations are often needed and that these levels are chosen to correspond to medical decision concentrations or critical method performance limits. Which is correct? The numbers of control levels for tests are best determined by your judgments of what concentrations are critical for method interpretation and performance. For example, total cholesterol might be monitored with two levels (200 mg/dL or 5.2 mmol/L and 240 mg/dL or 7.1 mmol/L), whereas glucose might be monitored with three levels (low hypoglycemic range, upper limit of reference range, very high such as 2 to 3 times upper limit of reference range). The website provides a summary table of Dr. Bernard Statland's recommendations for medical or clinical decision levels [Statland BE. Clinical Decision Levels for Laboratory Tests, 2nd edition. Oradell NJ;Medical Economics Books, 1987]. In some countries, such as the USA, there are regulations that set the minimum number of control materials. For example, the CLIA regulations mandate at least two control levels. The quality-planning models, as implemented in versions 1.1
and 2.0 of the QC Validator program, consider only one medical
decision level at a time, however, you can set up multiple files
to consider multiple decision levels, then decide which should
be used for selecting a QC procedure. In the QC recommendations
that result from our QC planning process, the total number of
control measurements can be distributed on one or more control
materials, e.g., an N of 4 could be provided by 2 measurements
on each of 2 different control materials or 4 measurements on
1 control material. An N of 2 could be provided by 2 measurements
on 1 material or 1 measurement on each of 2 materials. In the
USA, it would be necessary to perform 1 measurement on each of
2 materials in order to satisfy the regulatory requirements.
When should proficiency testing and control samples be analyzed?OK, I've determined the rule or rules to monitor assay performance and am using two levels of control materials. The manufacturer recommends QC testing once a day. The instrument maintenance is always performed Monday, as are most assay calibrations. Proficiency testing (PT) samples are always analyzed in singlet on Mondays, within two hours of calibration and immediately following QC testing. Our proficiency testing performance is acceptable, our internal QC is acceptable, and we're meeting all the requirements for accreditation. But how subjective is our scheduling? When to analyze PT and QC samples are two quite different issues. Concerning the scheduling of PT samples, I've heard a variety of discussions about what's fair and legal. It sounds like this scheduling might be considered "special handling" with regards to the CLIA regulations in the USA. More important is the possibility that these PT results won't be representative of what happens for all patient samples. Some randomization of the time of analysis for PT samples would make sure that many of the other factors and variables that affect the patient test results also are included in the performance observed with the PT samples. Concerning the scheduling of control samples, that is also subjective, but there is some rationale and logic that can be applied.
Our QC planning process and quality-planning models do not provide specifications on when to analyze controls. They only help you figure out how many controls are needed and what control rules should be used to assess whether the testing process achieves a stated quality requirement. The timing for analyzing QC samples depends on the stability of the method, its susceptibility to problems, and the cost relative to the expected benefits. Here's where your experience in the laboratory is valuable and your understanding of factors that affect method performance is critical. In QC planning, there's nothing wrong with using good judgment,
particularly when there's nothing better available. Just be sure
that you don't substitute judgment for a quantitative answer
that's easily available, such as the control rules and Ns that
are required. Use your judgment to help you define the quality
requirement and the analytical run (or when to analyze controls).
Use the QC planning tools to select the rules and N. What should I do when the reagent lot to lot changes cause large changes in the means of the controls?I have observed changes of 5 to 15% in the means of my control materials when I changed reagent lots on certain immunoassay methods. My within lot CVs were about 6%, so the highest result of the last lot was actually lower than the lowest of the new lot. What should I do? The situation raised in this question is somewhat similar to the situation in an earlier question "Is QC planning really necessary? Again, to avoid the specifics of brand names, we've eliminated specific test and instrument names in this question and will try to deal with it in a general way. However, this question was definitely aimed at immunoassays, whereas the earlier question was interpreted as being concerned with routine chemistry tests. The answers, therefore, are somewhat different. The first concern is whether the method is really working okay. It would be good to call these changes to the attention of the manufacturer to see if there are some checks that could assure the method is performing properly. Manufacturers often have their own control materials and know what values are being observed on different reagent lots in different laboratories, therefore checking with the manufacturer is a good first step. If your are using control materials from other suppliers, you may also be able to access control data from peer laboratories who are using the same materials and your same method, allowing you to compare your mean values with those being obtained by others. In talking with Karen Mugan, the medical technologist who is our quality specialist in the immunoassay area, she indicated that these lot to lot changes would be of concern because they are greater than what we've observed when running this same method in our laboratory. It would be important to confirm that the method is properly calibrated by using a new, unopened set of calibrators of the same lot and also a new lot if available. This is one of the most likely causes of systematic shifts in the means of control materials. Immunoassays often show changes from one reagent lot to another, therefore, it would be best to establish control limits on the basis of cumulative means and standard deviations obtained from data collected over several reagent lots. For the particular method being questioned here, it happens that the cumulative CVs at the University of Wisconsin Clinical Laboratory are about 6% to 8%, whereas within lot CVs are from 4% to 6%. Therefore, calculating control limits from the cumulative statistics will give somewhat wider limits, but not enough to account for the size of the changes stated in the question. Again, it seems important to confirm that the method is working properly. In a few situations, Karen indicated it may be necessary to assign new mean values until sufficient data is available to make good estimates, but that is a last resort. By the way, for those interested in the selection of QC procedures
for immunoassays, Karen wrote a nice paper that illustrates application
of the QC planning process for seven different tests. [See Mugan
K, Carlson IH, Westgard JO. Planning QC Procedures for Immunoassays.
J Clin Immunoassay 1994;17:216-222]. November 28, 1997:Is QC planning really necessary?I use a BRAND A analyzer for clinical chemistry analysis and 2 control sera with multirule criteria to check and evaluate analytical quality. When I calibrate some analytes for a problem and QC is still "out", do I have to find new control limits for that test, i.e., run 20 new controls and calculate new control limits? If this is a problem that occurs when you calibrate a new lot of reagents, it is important first to make an assessment of what QC procedure is necessary and what control rules are appropriate to monitor quality. With high precision analyzers, there often are small shifts that occur with changes in reagents. That's basically what your control procedure is telling you. The control procedure is doing its job in telling you a change has occurred. You have to decide whether the change is important and needs to be eliminated, which probably means re-calibrating rather than re-evaluating control limits. It is possible that multirule procedures, particularly those using rules such as 41s and 10x, may detect small shifts that aren't medically important. That's why the first step in resolving this problem is to assess what QC is actually necessary - i.e., plan an appropriate QC procedure so that you know you are using control rules that will detect medically important errors. This course will show you how to make this assessment and give you access to tools that make it quick and easy to do. If with new calibrations or repeated calibrations you find
QC continues to be out, it may be that the means of the control
materials have shifted and are slightly different from before.
It would be best to find out why and correct any problem that
might have caused this. If the shifts are small and cannot be
corrected, or judged not to be medically significant, then it
may be appropriate to set new mean values and calculate new control
limits. Note that it is easier to estimate new mean values than
to estimate new SDs; generally 9-16 measurements should give
a good starting estimate, which can be updated as more measurements
are accumulated. If you adjust the mean values, it is desirable
to monitor long term stability and systematic error by comparing
results with another method or comparing proficiency testing
results versus the group. What is this thing called total error?Conceptually, total error is a parameter, never able to be fully measured. Surely then the measure of total error is an impossibility. Can't we only make an estimate of errors which will never be the absolute total error? This is a very difficult question. I was afraid that this might happen - the questions from the course participants will be better than the answers from the course instructor. Nevertheless, I will tell you how I think about this. The concept of total error is important for formulating an analytical quality requirement that encompasses both the imprecision and inaccuracy of the method and, therefore, relates to the actual quality of the test result that is delivered to the customer or consumer. A single test measurement may be in error due to imprecision, inaccuracy, or both. In providing laboratory service, we should be concerned that we produce test results whose total error doesn't invalidate or limit the medical use and interpretation of the test. Most analysts accept the concept of total error, even though there are difficulties in how to define the allowable total error and how to measure the actual total error. The practical problems with total error are similar to those with imprecision (since imprecision is a component of total error). What is the allowable random error? How do you estimate random error? Traditionally, we have handled this issue by using the standard deviation to characterize imprecision and then expressing the size of the error expected in a single test result by a statement such as "the 95% or 2 SD range is plus/minus 10 units." To describe or estimate total error, we need to make a similar assumption about what percent of the expected values are to be included, which leads to the use of expressions such as "bias + 2SD" for a 95% two side estimate (i.e., includes 95% by allowing 2.5% in each tail of the distribution) or "bias + 1.65SD" for a 95% one-sided estimate (i.e., includes 95% by allowing 5% in one-tail of the distribution). In our QC planning models, we use the one-sided expression, therefore we specify that a run is to be considered out-of-control if 5% of the values are expected to exceed the total error requirement. In using the QC planning models with the QC Validator program, you have the option of changing this expression by changing the "z-value" on the parameters screen. |
Questions, comments, suggestions, and
feedback are encouraged!Use feedback form on this website Or e-mail westgard@westgard.com.
