Module 6 QC Basic Rules and Charts Analysis

Module 6: QC Basic Rules and Charts Analysis, Charts and Interpretation of Rules Center for Global Health

Learning Objectives Upon completion of this module you will be able to: • Define: Control materials, Calibration materials, Levey-Jennings Chart, Westgard Single Rule and Multi-Rule Criteria, Systemic Error, & Random Error • Describe a quality control sample in terms of assayed versus unassayed and their uses

Learning Objectives Upon completion of this module you will be able to: • Discuss the following Westgard rules: – 22 s – 13 s – 6 x – 12 s – 7 t • Apply the Westgard rules listed above to decide whether an analytic run is accepted or rejected. • List which Westgard rules detect systematic versus random error.

Definitions • Control materials: Protein-based materials made or manufactured with a composition similar to patient samples that have or have not been analyzed for concentration so they have expected or no expected values.

Definitions • Certified reference material (CRM) – “A reference material that has one or more values certified by a technically valid procedure and is accompanied by, or is traceable to, a certificate or other document that is issued by a certifying body. ” [CLSI] NBS (National Bureau of Standards) • Calibrator: purified solution used to adjust electrical output of an instrument and should trace back to a CRM.

Definitions • Levey-Jennings Chart: A visual presentation of daily quality control values plotted on a chart using mean, standard deviation, and +/- 3 Std Dev range criteria • Westgard Single Rule and Multi-Rule Criteria: A set of rules used to evaluate quality control values as acceptable or not

Definitions • Systemic Errors: Errors in the test system usually caused by a malfunction that affects all tests • Random Errors: Errors that statistically occur unpredictably and do not indicate system malfunction but do indicate an individual test or component malfunction

Need for Quality Control Samples • Patient results are “unknowns”. Analysis of the patient sample produces a number. But…is the result valid? • Patient may have previous result. But… – Change from previous may be expected – No change from previous result may be expected

Need for Quality Control Samples Quality control samples are “knowns. ” They have been made or manufactured with a composition similar to patient samples and have been analyzed for concentration before they are put into use so they have expected results. Therefore, if Quality Control samples are analyzed and expected results are obtained, we can assume our system is operating correctly. or If Quality Control samples are analyzed and do not give expected results, what should be done?

Quality Control Samples Are • Preserved to maintain accurate and precise results • Assayed –provided with mean and s for analytes • Unassayed – you must determine mean and s

Interactive Opportunity Why is it advisable to run controls at different levels?

Quality Control Sample Use • Analyzed solely for quality control purposes • Not for calibration • Should be used for qualitative and quantitative testing as known results to check if method to measure patient results is reliable.

Interactive Opportunity Is it sufficient to use only one calibrator to set the output of an analyzer?

Qualitative Versus Quantitative Results • What are some examples of qualitative tests in the laboratory? • What quality control samples would you use to verify the test is working so that patient results may be reported?

QC Results and Expectations for Qualitative Tests • Quality Controls will be a known positive and known negative material • The results will be recorded in a log book with the date, techs initials, and either ‘positive’ or ‘negative’ written in by the tech • Reason for running controls with qualitative tests is to check that the reagent(s) are working as they are supposed to and give accurate results

QC Results and Expectations for Quantitative Tests Gaussian distribution

Interactive Opportunity What does the “Internal” Quality Control on a test cartridge indicate?

QC Sample Analysis as an Alarm System QC sample result should: • Indicate when the test system is not working properly • Not indicate a false alarm • It should be 100% reliable

Common QC Sample Rules Single Rule: • Accept the analytical run if both quality control results fall within mean +/- 2 standard deviations range. • Reject the analytical run and troubleshoot the problem if one or both quality control results are outside +/- 2 standard deviations from the mean.

QC Sample Result Outside of Expected Range QC value < +2 SD or > -2 SD from the mean – Expected most of the time – With 95% confidence. This means 95% of acceptable results are in this range. 5% could still be acceptable i. e. false alarm QC value > + 3 SD or < - 3 SD from the mean – Usually indicates a problem and should be followed up on

Historical Use of QC Sample Results • Shewart developed statistical tools to monitor quality • Levey and Jennings – quality standards for the laboratory • Westgard - established rules for monitoring quality control

Levey-Jennings Chart

Quality Control Result Limits • Mean is the central line. • One standard deviation above = + 1 s • One standard dev below = - 1 s • 2 standard dev above = + 2 s • 2 standard dev below = -2 s • 3 standard dev above = + 3 s • 3 standard dev below = -3 s

Calculating Confidence Limits on the Chart • For example, Control 1 has a mean of 200 and a standard deviation of 4 mg/d. L. • What is the 95% confidence limit or mean + 2 s limit? – The upper control limit, mean + 2 s would be: 200 + 2*4 = 208 mg/d. L – The mean – 2 s, lower control limit would be: 200 - 2*4 = 192 mg/d. L 95% confidence limit is 192 -208 mg/d. L

Quality Control Sample Rules • Decision Criteria • In control versus out of control • Control limits – Mean + 2 s – 95% limits – 5% false rejection – If 1 result exceeds 2 s but not 3 s and no other shifts or trends, it is usually a warning but not a reason to reject the results.

QC Sample Measured with each Analytical Run How long is a run? • Manual • Automated method

QC Sample Analysis Procedure Laboratory Quality Control Policy (Quality Manual) stating your laboratory will not report results without appropriate QC. Quality control Analysis SOP • • • How many control samples How often run How documented How problems are solved What Westgard rules are used.

Lab Results Need to Be. . . • Accurate • Precise • Quality Control Rules should help to assess accuracy and precision. • After verification with QC Rules, lab results also need to be reported timely.

Accuracy • Closeness of the measured result to the true value • Your control values may show precision but be inaccurate • Instrument can repeatedly get the same value, but it’s not the right value 30

Precision is: The reproducibility or closeness of results to each other 31

Interactive Opportunity How can we check for accuracy and precision? Can you have precision without accuracy? Can you have accuracy without precision?

Possible Causes for Random Errors • Can be due to variations in line voltage, pipetting, dispensers, contamination, volume dispensed, bubbles in lines of reagents, etc*… • A random error can be either positive or negative • The direction and exact magnitude of the random error cannot be exactly predicted • Random Errors affect reproducibility or precision of a test system

Random Errors and Possible Causes 1 3 s Rule Violation: • Usually indicates a Random Error • When reviewing statistical data remember when using +/- 3 SD, there is a 0. 3% chance of getting a control value outside the +/- 3 SD range • Before running patient specimens you must troubleshoot AND then rerun the control and the values must be within +/- 2 SD • Check that controls are in date and well mixed*

Westgard Rules • Random error detected by 1 3 s • This means that 1 QC value exceeds -3 s • Could be below -3 s or above + 3 s

Possible Causes for Random Errors 2 2 s Rule Violation: • Usually indicates a Systematic Error • Remember, the first time single control value is > or < 2 SD it serves as a warning and you can run and report out your patient values • BUT, the second sequential time the control value is > or < 2 SD, you must trouble shoot and find the reason for the error BEFORE you can run patient specimens and report out patient values • Check that controls are in date and well mixed

Westgard Rules • Systematic error detected by 2 2 s • Could be below -2 s or above + 2 s

Systematic Errors • Systematic Errors include bias/shifts, & trends • These errors affect accuracy of the test system. • Usually 6 x and 7 t Rule Violations • Can be due to calibration lot changes, temperature changes in incubator unit, light source deterioration, electronics, reagent lot changes, etc. • Systematic errors are always in one direction on a L J chart.

Bias/Shift: 6 x Rule Violation • A systematic difference or systematic error between an observed value and some measure of the truth. • 6 x Rule violations: – Usually indicate an error with your test system – On day 7, after 6 days of one or more control values above or below the mean*, you must trouble shoot and find/correct error before running patient specimens *Note the values of the 6 days are within +/- 2 sd

Westgard Rules • Systematic Error detected with 6 x: shift • 6 results are on the same side of the mean

Trend: 7 t Rule Violation • Systematic error when a series of control values are treated as a time series • 7 t Rule violations: – Usually indicate an error with your system – On day 8, after 7 days of one or more control values moving either upward or downward, you must trouble shoot and find/correct error before running patient specimens

Westgard Rules • Systematic error detected with 7 t: trend • 7 results falling in a pattern, up or down

Levey Jennings Charts: Evaluation of Results

Westgard Rules: Single or Multi Rule QC • Single rule QC – Procedure uses a single criterion or single set of control limits, such as a Levey-Jennings chart with control limits set as either the mean plus or minus 2 standard deviations (2 s) or the mean plus or minus 3 s. There is more than 1 single rule but each rule is assessed individually. • Multi-rule QC – Uses a combination of decision criteria, or control rules, to decide if an analytical run is in-control or out-of-control – Uses 5 different control rules to judge acceptability of an analytical run

Westgard Rules • Many laboratories follow the Westgard Single-Rule policy. • The rules are assessed individually to check for random and systematic errors.

Westgard Multirule System • These rules work best together in a multi-rule system with automated analyzers • Rules evaluated in a certain order • Starts with the warning rule (12 s ) that triggers the need to look for other violations:

Westgard Multirules • This decision tree helps to decide whether to reject or accept a run. If you answer yes, it means that rule has been violated.

Key Points • Describe a control in terms of – Assayed (mean and s provided) – Unassayed (mean and s not provided) – Use (validation of run not calibration) • Calculate 95% limits for quality control samples • Detect errors in quality control using 95% and Westgard rules

Key Points • Levey-Jennings chart include values and markings on the x (days or time) and y axis (concentration mean and s)

Key Points • Westgard rules increase ability to detect systemic errors.

Key Points • Westgard Rules increase ability to detect random errors.

Key Points • Westgard Rule, 1 2 s is provided as a Warning

References • Shewhart WA. Economic Control of Quality of Manufactured Product. New York; D. Van Hostrand Company, Inc. , 1931. • Levey S, Jennings ER. The use of control charts in the clinical laboratory. Am J Clin Pathol 1950; 20: 1059 -66. • Henry RJ, Segalove M. The running of standards in clinical chemistry and the use of the control chart. J Clin Pathol 1952; 27: 493 -501. • Westgard JO, Groth T, Aronsson T, Falk H, de. Verdier CH. Performance characteristics of rules for internal quality control: probabilities for false rejection and error detection. Clin Chem 1977; 23: 1857 -67.

References • Westgard JO, Barry PL, Hunt MR, Groth T. A multi-rule Shewhart chart for quality control in clinical chemistry. Clin Chem 1981; 27: 493 -501. • Westgard JO, Barry PL. Cost-Effective Quality Control: Managing the Quality and Productivity of Analytical Processes. Washington, DC: AACC Press • www. westgard. com