Education and Training in Analytical Quality Management, Part Four: Internet Tools for QC Training A Suite of QC Training Tools: QC Calculator QC Plotter QC Simulator QC Trainer QC Checker How sweet it is! References

A word from Dr. Westgard

Please note that these are beta versions of these tools: we are making them available at this time for public comment - so let us know what you think of them.

Earlier essays in this series describe the development of a "lesson-base" for education and training in the quality management of laboratory testing processes (Part I). This lesson-base can be thought of as a textbook of materials, but with the added flexibility to quickly and easily extract or rearrange the "chapters" that are needed for a specific training course. The structure of an Internet training course was described in Part II, along with the step-by-step process for developing such a course. Example courses for Basic Method Validation and Basic QC Practices are now available for continuing education credits through the ASCLS.

Basic QC Practices and Basic Method Validation deal with fundamentals. Part III in this series discussed the need for basic QC training and the availability of these new training materials on the Internet, in CD format, and in hard-copy format. These materials are now in use in laboratories and CLS training programs around the country. We are beginning to get feedback from users and will present some of these discussions in the near future.

In an effort to draw more users to the Internet, we are adding a new set of interactive QC training tools to demonstrate the advantage of the Internet materials. This update describes these new QC tools and their intended uses.

Suite of QC Training Tools

Five tools are provided. Each tool builds on the previous tool to provide complementary features and additional capabilities, as follows:

• calculation of control data to provide monthly and cumulative means, SDs, CVs, and control limits;
• preparation of a control chart and a plot of control data;
• generation of control data to demonstrate the effects of method bias (systematic shifts of a control mean) and increases in method imprecision (increases in the control standard deviation);
• generation of control data to be interpreted by the user to judge control status, which is then verified by the program; and
• assessment of control status for control data entered by the user and for control rules selected by the user.

QC Calculator

This tool can be used with the initial data obtained by analyzing a control material repeatedly over a period of time. The user enters the control values and the tool calculates the mean, SD, and CV. The user can also enter the multipler to be used in calculating control limits. Cumulative means, SDs, CVs, and control limits can be calculated when user enters summation terms from earlier calculations. Finally, a blank control chart can be prepared and printed.

Here are some typical exercises for this tool:

• What are the mean, SD, and CV for a cholesterol control when the following values have been collected: 199, 195, 201, 205, 205, 207, 191, 199, 204, 196, 197, 193, 193, 196, 197, 192, 198, 198, 201, 199?
• What are the control limits that would be setup to use a 1_3s control rule?
• What are the cumulative mean, SD, and CV for this cholesterol control after the following values have been collected: 202, 200, 194, 204, 203, 195, 202, 202, 206, 198, 200, 189, 202, 205, 198, 194, 201, 205, 196, 200?
• What are the cumulative control limits that would be used on a multirule Chart?
• Prepare the control chart with cumulative control limits for use with a multirule QC procedure.
  

See the tool!

QC Plotter

The second tool assumes that the user already has information on the mean and SD of the control material, which are entered along with the desired control limit multiplier to set up the control chart. The user can then enter up to 60 control results that will be plotted on the control chart, which can then be printed.

Here are some typical exercises for this tool:

• Setup a multirule chart for a cholesterol control material whose mean is 199.8 mg/dL and SD is 4.41 mg/dL.
• Plot the following control data on the chart above: 199, 200, 197, 201, 198, 206, 199, 194, 192, 214, 200, 198, 207, 198, 197, 200, 191, 205, 197, 199.
• How many control violations would be observed if using 2s control limits?
• How many control violations would be observed if using 3s control limits?
• How many control violations would be observed if using 1_3s/2_2s/R_4s control rules?

See the tool!

QC Simulator

The third tool uses the same parameters to set up a control chart, but instead of having manual entry of control data, the tool generates sets of control data that demonstrate the distribution expected for error conditions selected by the user. For example, an accuracy problem can be simulated by changing the mean of the control data; a precision problem can be simulated by increases the standard deviation of the control data. Such trial sets of control data are quickly displayed on the control chart, along with the control limit lines specified by the user. This tool is especially useful for learning how different analytical errors are expected to appear on control charts.

Here are some typical exercises for this tool:

• Given a cholesterol control whose stable mean is 200 mg/dL and stable standard deviation is 4.0 mg/dL, generate a set of 20 control measurements that show no analytical problems and observed how many exceed 2s and 3s control limits.
• Describe the expected appearance of control data when there is an accuracy problem that shifts the mean to 204 mg/dL.
• Will you be able to detect an accuracy problem of this magnitude (shift equivalent to the size of the SD) whenever it occurs?
• Describe the expected appearance of control data when there is a precision problem that increases the standard deviation to 8 mg/dL.
• Will you be able to detect a precision problem of this magnitude (doubling of the SD) whenever it occurs?

See the tool!

QC Trainer

Like the tool above, but the QC Trainer displays only 2 to 4 control measurements at a time (per run) to be viewed and interpreted by the user. This tool is especially useful to practice interpreting control data with different control rules, such as 1_3s, 1_2s, and 1_3s/2_2s/R_4s with N=2, 1_3s, 1_2s, and 1_3s/2_2s/R_4s with N=3, and 1_3s, 1_2s, and 1_3s/2_2s/R_4s/4_1s with N=4. The user selects the number of control measurements per run (N/run) and the control rules to be applied, displays control results for the next run, interprets that data to judge the control status, and has the program verify the correctness of that interpretation.

Here are some typical exercises for this tool:

• For a cholesterol control material whose stable mean is 200 mg/dL and stable standard deviation is 4.0 mg/dL, use a 1_2s rule with N=2 to interpret 10 runs (20 data points) that represent stable performance. Do you observe any false rejections?
• For the above cholesterol method, use a multirule QC procedure with N=2 to interpret 10 runs (20 data points) that represent stable performance. Do you observe any false rejections?
• For the above cholesterol method, use a multirule QC procedure with N=2 to interpret 10 runs (20 data points) that represent an accuracy problem equivalent to a 1s shift. How often were you able to detect a 1s shift?
• For the above cholesterol method, use a multirule QC procedure with N=2 to interpret 10 runs (20 data points) that represent an accuracy problem equivalent to a 2s shift. How often were you able to detect a 2s shift?

See the tool!

QC Checker

This tool demonstrates how a computer program can assist the user in the interpretation of QC data. The user selects a set of control rules and manually enters control values that can be displayed on a control chart and checked for violation of those control rules. The user enters the values for one run, checks control status, then enters the values for the next run, etc. This tool is very useful for checking specific sets of control data and for clarifying the interpretation of control results.

Here are some typical exercises for this tool:

• Prepare a control chart for the cholesterol example method (mean of 200 mg/dL, standard deviation of 4 mg/dL) to demonstrate the violations of different control rules.
• Prepare a set of control data for teaching the interpretation of multirule QC.
• For real control data from a method in your laboratory, set up a control chart to analyze the data using various control rules. What numbers of rejections are observed when different single-rule and multi-rule QC procedures are applied?
• For real control data from a method in your laboratory, compare the laboratory assessment of control status with that of the checker tool.

See the tool!

How sweet it is!

These training tools are similar to the research tools that have been used for many years to determine the rejection characteristics of statistical QC procedures. The benefits of simulation and modeling for analytical quality management were first demonstrated in clinical chemistry by deVerdier, Groth, and Aronsson at Uppsala University in Sweden in the early 1970s [1]. I was fortunate to have the opportunity to work with the Uppsala group in the mid 70s and to have the advantage of using these techniques to study statistical QC [2,3]. By simulating thousands of sets of trial QC data, we were able to understand the problem of false rejection and to develop multirule QC procedures to improve performance [4]. Since that time, simulation and modeling have been adopted by many investigators to optimize QC and improve analytical quality management [5].

It is a pleasure to be able to make these techniques available to you and others for QC training. With these QC training tools, you and your students can answer many "what if" questions by easily generating trial control data and preparing graphical displays. That's how I learned everything I know about QC. You now can learn in the same way. As Jackie Gleason said when concluding the Honeymooners TV show, "How sweet it is!" In this case, how sweet it is to have a new suite of interactive QC training tools that are available via the Internet!

References

1. Aronsson T, de Verdier C-H, Groth T. Factors influencing the quality of anlaytical methods - a systems analysis, with use of computer simulation. Clin Chem 1974;20:738-748.
2. Westgard JO, Groth T, Aronsson T, Falk H, de Verdier C-H. Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection. Clin Chem 1977;23:1857-1867.
3. Westgard JO, Groth T. Power functions for statistical control rules. Clin Chem 1979;25:394-400.
4. Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981;27:493-501.
5. Westgard JO. Simulation and modeling for optimizing quality control and improving analytical quality management. Clin Chem 1992;38:175-178.

James O. Westgard, PhD, is a professor of pathology and laboratory medicine at the University of Wisconsin Medical School, Madison. He also is president of Westgard QC, Inc., (Madison, Wis.) which provides tools, technology, and training for laboratory quality management.

Other Essays:

Myths of Quality Putting Quality into Quality Control Assuring Quality through Total Quality Management Trends in quality management: Utilization and Outcomes Quality Goals, Requirements, & Specifications Future Directions in Quality Control The Myth of Medical Decision Limits Quality by Design Tools and Technology for QC Training Education and Training for Analytical Quality Management, Part I Mapping the Road to Analytical Quality with OPSpecs Charts Quality and Commerce QC - Back to Basics Education and Training for Analytical Quality Management, Part II: Developing Web-courses Method Validation - The Inner, Hidden, Deeper, Secret Meaning Education and Training in Analytical Quality Management, Part III: Basic QC Training Electronic QC and the Total Testing Process From Rules and Tools to Technology and Training (Beijing) Quality Requirements: the debate heats up Z-Stats: A treat and a treatment The Need for a System of Quality Standards What's wrong with traditional QC? To be Uncertain or In Error? That is the Question QC 2000 Education and Training for Analytical Quality Management, Part IV: Interactive Training Tools Do's and Dont's of QC The Abbott Consent Decree: A Wake-Up Call WQC Y2K Sage Advice about new approaches to Quality Control EZ Rules for Assuring Quality Who will care to quality tomorrow? Quality is Job 1 when the rubber meets the road Errors in reasoning about Laboratory errors Six Sigma Quality Management & Lab Precision Six Sigma Quality Managment & Requisite Lab QC 2001: Year of the Odyssey essays CLIA Postponed again and again and again Repeated, Repeated, Got Lucky Six Sigma Staffing Strategies Technology for Implementing QC Right $aving the Cost$ of Poor Quality

Has Quality been "Enronned"? Why not Evidence-Based Method Specifications? Quality: "I think I got it!" Cooking the Books: Does it happen in the Lab? CLIA QC Clearance - A momentous happening Signs of Six Sigma Good Data Wanted, Bad Data Need Not Apply Final, final, final, final, final CLIA Rules The Truth Standard for Quality It's an Honor: Reflections on being a Teacher 2004 JCAHO Patient Safety Goals ISO Says So Medical Errors: Complexity and Its Solutions Giving Thanks for 2003: Observations on the state of Quality Autoverification: Taking QC to the next level - is that up or down? Think straight, Talk straight The Gospel According to ISO More on Eqc and "Quality-Less" Compliance Testing Equivalent Quality: A better way The Final Word on the Final Rule? Hear, Hear, Hearings on Untruth and Unquality, Part I Hearings on Untruth, Part II: Cracks Hearings on Untruth, Part III: Facts Hearings on Untruth, Part III: Broken Windows Connecting the Dots Hearings on Untruth, Part IV: Inadequate Inspections Hearings on Untruth, Part V: Bad Apples or Tip of the Iceberg? The Quality of Laboratory Testing, Part I No Laboratory Left Behind Vioxx and Values, Vaccines and Votes The Quality of Laboratory Testing: Methodology The Quality of Cholesterol Testing Bah, Humbug! How I learned to love EQC The Quality of Glucose Testing The Quality of Calcium Testing Blowing the Whistle on the Tip of the Iceberg The Quality of Glycohemoglobin The Quality of PSA Testing Solutions for the Taxing Problem of QC The quality of Coagulation Testing The variability of estimates from PT surveys Links to India, Part I Test Quality vs. Method Performance QC: Not just a technicality 2005 in Review: 100,000 miles to Quality Unannounced Inspections, Unknown Consequences Hopeful Healthcare in a Fearful Society Quality Indicators and Benchmarks Trouble with Tracking Tests