Tools, Technologies and Training for Healthcare Laboratories
  2. Home
  3. Essays
  4. "Housekeeping"
  5. Education and Training in Analytical Quality Management, Part Four: Internet Tools for QC Training

"Housekeeping"

Education and Training in Analytical Quality Management, Part Four: Internet Tools for QC Training

You've built a "lesson-base." You've created an online course. You've addressed the need to teach basic skills. Now what's missing? Of course: interactive tools for the Internet. Plotters, calculators and data simulation tools that allow the user to plug in numbers on the fly and see what happens. See the new tools and what you can do with them.

Please note that these are beta versions of these tools: we are making them available at this time for public comment - so let us know what you think of them.

Earlier essays in this series describe the development of a "lesson-base" for education and training in the quality management of laboratory testing processes. This lesson-base can be thought of as a textbook of materials, but with the added flexibility to quickly and easily extract or rearrange the "chapters" that are needed for a specific training course. The structure of an Internet training course was described in Part II, along with the step-by-step process for developing such a course. Example courses for Basic Method Validation and Basic QC Practices are now available for continuing education credit.

Basic QC Practices and Basic Method Validation deal with fundamentals. Part III in this series discussed the need for basic QC training and the availability of these new training materials on the Internet and in hard-copy format. These materials are now in use in laboratories and CLS training programs around the country. We are beginning to get feedback from users and will present some of these discussions in the near future.

In an effort to draw more users to the Internet, we are adding a new set of interactive QC training tools to demonstrate the advantage of the Internet materials. This update describes these new QC tools and their intended uses.

Suite of QC Training Tools

Five tools are provided. Each tool builds on the previous tool to provide complementary features and additional capabilities, as follows:

  • calculation of control data to provide monthly and cumulative means, SDs, CVs, and control limits;
  • preparation of a control chart and a plot of control data;
  • generation of control data to demonstrate the effects of method bias (systematic shifts of a control mean) and increases in
  • method imprecision (increases in the control standard deviation);
  • generation of control data to be interpreted by the user to judge control status, which is then verified by the program; and
  • assessment of control status for control data entered by the user and for control rules selected by the user.

QC Calculator

This tool can be used with the initial data obtained by analyzing a control material repeatedly over a period of time. The user enters the control values and the tool calculates the mean, SD, and CV. The user can also enter the multipler to be used in calculating control limits. Cumulative means, SDs, CVs, and control limits can be calculated when user enters summation terms from earlier calculations. Finally, a blank control chart can be prepared and printed.

Here are some typical exercises for this tool:

  • What are the mean, SD, and CV for a cholesterol control when the following values have been collected: 199, 195, 201, 205, 205, 207, 191, 199, 204, 196, 197, 193, 193, 196, 197, 192, 198, 198, 201, 199?
  • What are the control limits that would be setup to use a 13s control rule?
  • What are the cumulative mean, SD, and CV for this cholesterol control after the following values have been collected: 202, 200, 194, 204, 203, 195, 202, 202, 206, 198, 200, 189, 202, 205, 198, 194, 201, 205, 196, 200?
  • What are the cumulative control limits that would be used on a multirule Chart?
  • Prepare the control chart with cumulative control limits for use with a multirule QC procedure.

See the tool!

QC Plotter

The second tool assumes that the user already has information on the mean and SD of the control material, which are entered along with the desired control limit multiplier to set up the control chart. The user can then enter up to 60 control results that will be plotted on the control chart, which can then be printed.

Here are some typical exercises for this tool:

  • Setup a multirule chart for a cholesterol control material whose mean is 199.8 mg/dL and SD is 4.41 mg/dL.
  • Plot the following control data on the chart above: 199, 200, 197, 201, 198, 206, 199, 194, 192, 214, 200, 198, 207, 198, 197, 200, 191, 205, 197, 199.
  • How many control violations would be observed if using 2s control limits?
  • How many control violations would be observed if using 3s control limits?
  • How many control violations would be observed if using 13s/22s/R4s control rules?

See the tool!

QC Simulator

The third tool uses the same parameters to set up a control chart, but instead of having manual entry of control data, the tool generates sets of control data that demonstrate the distribution expected for error conditions selected by the user. For example, an accuracy problem can be simulated by changing the mean of the control data; a precision problem can be simulated by increases the standard deviation of the control data. Such trial sets of control data are quickly displayed on the control chart, along with the control limit lines specified by the user. This tool is especially useful for learning how different analytical errors are expected to appear on control charts.

Here are some typical exercises for this tool:

  • Given a cholesterol control whose stable mean is 200 mg/dL and stable standard deviation is 4.0 mg/dL, generate a set of 20 control measurements that show no analytical problems and observed how many exceed 2s and 3s control limits.
  • Describe the expected appearance of control data when there is an accuracy problem that shifts the mean to 204 mg/dL.
    Will you be able to detect an accuracy problem of this magnitude (shift equivalent to the size of the SD) whenever it occurs?
  • Describe the expected appearance of control data when there is a precision problem that increases the standard deviation to 8 mg/dL.
  • Will you be able to detect a precision problem of this magnitude (doubling of the SD) whenever it occurs?

See the tool!

QC Trainer

Like the tool above, but the QC Trainer displays only 2 to 4 control measurements at a time (per run) to be viewed and interpreted by the user. This tool is especially useful to practice interpreting control data with different control rules, such as 13s, 12s, and 13s/22s/R4s with N=2, 13s, 12s, and 13s/22s/R4s with N=3, and 13s, 12s, and 13s/22s/R4s/41s with N=4. The user selects the number of control measurements per run (N/run) and the control rules to be applied, displays control results for the next run, interprets that data to judge the control status, and has the program verify the correctness of that interpretation.

Here are some typical exercises for this tool:

  • For a cholesterol control material whose stable mean is 200 mg/dL and stable standard deviation is 4.0 mg/dL, use a 12s rule with N=2 to interpret 10 runs (20 data points) that represent stable performance. Do you observe any false rejections?
  • For the above cholesterol method, use a multirule QC procedure with N=2 to interpret 10 runs (20 data points) that represent stable performance. Do you observe any false rejections?
  • For the above cholesterol method, use a multirule QC procedure with N=2 to interpret 10 runs (20 data points) that represent an accuracy problem equivalent to a 1s shift. How often were you able to detect a 1s shift?
  • For the above cholesterol method, use a multirule QC procedure with N=2 to interpret 10 runs (20 data points) that represent an accuracy problem equivalent to a 2s shift. How often were you able to detect a 2s shift?

See the tool!

QC Checker

This tool demonstrates how a computer program can assist the user in the interpretation of QC data. The user selects a set of control rules and manually enters control values that can be displayed on a control chart and checked for violation of those control rules. The user enters the values for one run, checks control status, then enters the values for the next run, etc. This tool is very useful for checking specific sets of control data and for clarifying the interpretation of control results.

Here are some typical exercises for this tool:

  • Prepare a control chart for the cholesterol example method (mean of 200 mg/dL, standard deviation of 4 mg/dL) to demonstrate the violations of different control rules.
  • Prepare a set of control data for teaching the interpretation of multirule QC.
  • For real control data from a method in your laboratory, set up a control chart to analyze the data using various control rules. What numbers of rejections are observed when different single-rule and multi-rule QC procedures are applied?
  • For real control data from a method in your laboratory, compare the laboratory assessment of control status with that of the checker tool.

See the tool!

How sweet it is!

These training tools are similar to the research tools that have been used for many years to determine the rejection characteristics of statistical QC procedures. The benefits of simulation and modeling for analytical quality management were first demonstrated in clinical chemistry by deVerdier, Groth, and Aronsson at Uppsala University in Sweden in the early 1970s [1]. I was fortunate to have the opportunity to work with the Uppsala group in the mid 70s and to have the advantage of using these techniques to study statistical QC [2,3]. By simulating thousands of sets of trial QC data, we were able to understand the problem of false rejection and to develop multirule QC procedures to improve performance [4]. Since that time, simulation and modeling have been adopted by many investigators to optimize QC and improve analytical quality management [5].

It is a pleasure to be able to make these techniques available to you and others for QC training. With these QC training tools, you and your students can answer many "what if" questions by easily generating trial control data and preparing graphical displays. That's how I learned everything I know about QC. You now can learn in the same way. As Jackie Gleason said when concluding the Honeymooners TV show, "How sweet it is!" In this case, how sweet it is to have a new suite of interactive QC training tools that are available via the Internet!

References

  1. Aronsson T, de Verdier C-H, Groth T. Factors influencing the quality of anlaytical methods - a systems analysis, with use of computer simulation. Clin Chem 1974;20:738-748.
  2. Westgard JO, Groth T, Aronsson T, Falk H, de Verdier C-H. Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection. Clin Chem 1977;23:1857-1867.
  3. Westgard JO, Groth T. Power functions for statistical control rules. Clin Chem 1979;25:394-400.
  4. Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981;27:493-501.
  5. Westgard JO. Simulation and modeling for optimizing quality control and improving analytical quality management. Clin Chem 1992;38:175-178.

James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.