Diagnostic Accreditation Program A Systematic Approach to Quality

Diagnostic Accreditation Program A Systematic Approach to Quality and Safety in Diagnostics with emphasis on Medical Peer Review Dr. Carlow, MD CCFP

Objectives § To describe why this is important § To identify what is being done throughout healthcare § To define key principles and practices of a systematic approach § To define issues and solutions for two diagnostic modalities including examples from the field

Why is this important? • An elderly man underwent chemotherapy for GI cancer at BCCA in the early 1990’s. In error, he received 10 times the normal dose of 5 FU and died as a consequence. • A Boston health reporter in her late 30’s received a large overdose of chemotherapy for breast cancer at the Dana Farber Cancer Institute and died. • Both of these tragic events led to major systematic changes.

Why is this important? • A 34 year old woman diagnosed with neuroendocrine cancer had five surgeries to exise a cyst, remove lower jaw and teeth, and undergo facial reconstruction. Her slides were contaminated by cells from another patient. She did not have cancer. • A patient with a positive pregnancy test underwent pelvic ultrasound. The uterus was described as empty. The patient received methotrexate for the treatment of ectopic pregnancy. On review, another radiologist subsequently diagnosed normal intrauterine pregnancy.

Why is this important? Diagnostic Errors in the daily News • Pathology errors force thousands to be retested in New Brunswick, G & M Feb 08 • Disgraced Ontario pathologist says errors not all his fault G & M Mar 08 • Serious quality-control problems plague hospital labs in Canada G & M Mar 08 • 108 women died after botched cancer tests Newfoundland says G & M Mar 08 • Errors found in work of another pathologist (6% error rate) G & M May 08

Why is this important? More than Anecdotes • Harvard medical practices study (Leape NEJM 91) 3. 7% with disabling injuries caused by medical treatment. • Institute of Medicine (U. S. ) report released in 1999 – To Err is Human: Building a Safer Health System • • 44, 000 to 98, 000 preventable deaths Canadian adverse events study: • Adverse event rate of 7. 5 per 100 hospital admissions Baker et al CMAJ 2004

Why is this important? More than Anecdotes • American physicians got it right 55% of the time Mc. Glynn et al NEJM 2003 • Many studies reveal significant variations in practice/low rates of standardization • Swiss Cheese

Why is this important? What about diagnostic errors? • • • What types of medical errors occur more frequently – medication or diagnostic errors? In two recent studies of malpractice claims – diagnostic errors far outnumbered medication errors as a cause for claims Diagnostic errors are underemphasized and understudied

Why is this important? What about autopsy discovered errors? • Multi decade study Shojania et al JAMA, 2003 • Median error rate 23. 5% for major errors • Although error rates have declined over the decades, rates are sufficiently high enough that ongoing use is warranted U. S. national average autopsy rate is 5% •

What is being done? • • Agency for Health Care Research & Quality (AHRQ) Canadian Patient Safety Institute (CPSI) Institute for Health Care Improvement (IHI) Greater expectations from standard setting bodies: • • CCHSA JCAHO CAP ACR

What is being done? • • • Safer health care now Collaboratives 5 million lives campaign Hospital standardized mortality ratio (HSMR) Global Trigger Tool

What is being done? • • • Quality networks Provincial councils on Quality and Safety Governing Board’s focusing more on quality and safety Standards of professional bodies Recertification

What is being done? Organizational Initiatives • • Veterans Administration/Kaiser Permanente BC Cancer Agency These are two examples of the systematic application of evidence and the integration of quality and safety.

What is being done? • Chronic Disease Management Initiatives • • Hypertension CHF Diabetes Evidence based stroke program • Campbell River hospital

What is being done? What has been learned about the major attributes of a systematic approach? • • • Fragmented and isolated initiatives are quite pervasive and ineffective Importance of research driven evidence based care The important role of clinical decision support systems and tools

What is being done? • • The integration of quality and safety The importance of overall system design and clinical governance: • • • e. g. Trauma system e. g. micro systems Thorough knowledge of improvement methods and tools including: • • Knowledge of processes Quality improvement cycles Root cause analysis Rapid cycle improvement

What is being done? • Quality planning and priorities • • A clearly set out agenda for quality and safety An enabling culture • • • Leadership commitment Professional responsibility Inter-professional collaboration Non-punitive reporting Disclosure Improvement mindset Thinking and acting as a system Accountability Breakthrough thinking/aggressive targets

What is being done? • • • Surveillance/Monitoring/Measurement of processes, outcomes and benchmarking Quality and Safety infrastructure support Technological support Standards of professional bodies are now reflecting these attributes

What is being done? Causes of Error • • Variation in practice with variable inputs Complexity – too many steps Inconsistent knowledge, training and language (terminology) Human factors in routine repetitive tasks Deadlines/stress/excessive workload Handoffs – transfer of information Cultural issues – lack of openness and freedom of expression Unsystematic/adhoc approaches • Swiss cheese effect

How should we proceed in Diagnostic Services? • • Do we know enough about the various attributes of diagnostic errors? Are not traditional methods of medical peer review adequate and working well?

How should we proceed in Diagnostic Services? Areas that need attention • • • Better definition of what constitutes an error Greater consistency in definitions, terminology and standardization of reporting Better tools to assess significance of errors

How should we proceed in Diagnostic Services? Areas that need attention • • • More research on the extent of errors and their causes More research on the relationship between errors and adverse affects Being clearer about acceptable rates of errors

How should we proceed in Diagnostic Services? Traditional methods of peer review • • Morbidity and mortality conferences Autopsy Malpractice claims analysis Error reporting systems

How should we proceed in Diagnostic Services? Traditional methods of peer review • • • Chart review Observation of patient care Clinical surveillance Administrative data analysis Electronic medical record review

How should we proceed in Diagnostic Services? Many of these have positive attributes, however: • • • Low case numbers Hind sight bias Under reporting Absence of standardization Some have a linkage to total organizational effort Some not specific enough for program or department

Anatomic Pathology Errors • In general anatomic diagnoses are highly accurate? • • In the opinion of several, errors are not rampant Diagnostic variation is not uncommon, but not all harmful Depends on what are acceptable results Medical quality affected by all phases of the system

Anatomic Pathology Errors Life Cycle Data • Data indicates the importance of gathering information over the whole testing cycle • Carroro et al in Clinical Chemistry 2007 report • 61. 9% pre analytic errors • 15% analytic • 23% post analytic

Anatomic Pathology Errors Pre Analytic Phase • In this phase of the test cycle the problems more frequently relate to: • • • Specimen I. D. Sample quality Availability of clinical information

Anatomic Pathology Errors Pre Analytic Phase • • In one large study 6% of cases were defective at accessioning with defective I. D. as the 2 nd largest category Nakhleh et al CAP Q probes APLM 1996 A survey of 341 labs revealed no clinical history in 2. 4% of cases. When corrected – change in diagnoses in 6. 1% of cases Nakhleh et al CAP Q-probes APLM 1998

Anatomic Pathology Errors Pre Analytic Phase • Patient I. D. errors in SP are the most rapidly growing category of malpractice claims in the U. S. Most involve switch of specimens and most involve needle biopsy of prostate and breast

Anatomic Pathology Errors Pre Analytic Phase • • Errors in thyroid gland FNA with relatively high false positive and false negative rates – quality of tissue sampling by non-pathologists FNAhistologic correlation reveals ¼ of thyroid cancer patients are misdiagnosed as not having cancer due to: • • Errors in specimen quality Misinterpretation Raab et al ASLP 2006

Anatomic Pathology Errors Analytic Phase • In a 4 hospital review up to 12% of tissues examined by pathologists resulted in errors, more than 1/3 were associated with harm (AHRQ funded) Raab, Cancer 2005 • Up to 15% of patients with lung mass misdiagnosed due to pathology errors, different rates among hospitals due to “Big Dog” effect and using different methods

Anatomic Pathology Errors Analytical Phase • Average discrepancy frequency in pathology reports from 74 labs on secondary review is 6. 7% with 5% of these having an affect on patient care (1% of all cases) • Canadian Pathology Error Rates: • Retro • Prospect 14. 1% overall rate 1. 2% major 13% overall 1. 7% major Lind 1995 AJSP

Anatomic Pathology Errors Analytic Phase • • • Most studies are single institution hence variation However multi institutional studies reveal a discrepancy rate of 6. 7% with between 1 and 1. 7% causing harm What is an acceptable level of performance?

Anatomic Pathology Errors Analytic Phase • • Consider that a 1% error rate equates to 10, 000 errors per million Industrial six sigma standard is 3. 4 defects per million Industry average – four sigma = 6210 defects per million Should a six sigma standard apply to pathologists?

Anatomic Pathology Errors Post Analytic Phase • Two aspects of the post analytic phase that are the most important: § § completeness of reporting; 28. 4% increase in complete reporting using computer based synoptic reports Communication of critical results and customization of critical values for each institution

Anatomic Pathology Errors § Medical Quality Improvement is most effective if collection, processing, interpretation and connection to care providers are considered as an integrated system

Anatomic Pathology Errors Solutions/Tools § Have a plan and priorities for quality improvement and safety, consider: § § § Health Authority priorities Standards of professional bodies Guidance in literature Performance data Internal assessment – Process map Identify priority projects § § § Mission Culturally aligned teams Improvement methods (PDCA) and root cause analysis, lean design 6 sigma leap frog

Anatomic Pathology Errors Solutions/Tools Generic Laboratory Test Cycle Phases Test Request Report Interpretation Procedural Patient and specimen preparation, identification, transportation, handling, accession Technical & Diagnostic Test method, lab protocols, criteria, terminology, accuracy, report content, analytic timelines Communication Report delivery, format, clarity, overall timeliness, integration of information, satisfaction Preanalytic Analytic Postanalytic

Anatomic Pathology Errors Solutions/Tools Error types and test-cycle phases.

Anatomic Pathology Errors Solutions/Tools • Standardization • • • Terms, language, processes, tasks – work is to be done in a certain way Adopting standardized, structured, synoptic reporting formats province wide Consider computerized capture of structured data/synoptic reports linked to databases allowing best practice comparisons, information distribution, trend analysis and discrepancy identification • e. g. m. Tuitive

Anatomic Pathology Errors Solutions/Tools • Peer review • • • Blinded unbiased double slide review, selecting areas of high risk for error Amended reports are decreased with 2 nd pathologist review Nakhleh et al APLM 1998 Prostate cancer – impact of 2 nd pathologist on Gleason score: • 25. 2% change • 14. 8% change in management Thomas et al Brachytherapy 2007

Anatomic Pathology Errors Solutions/Tools • • Double viewing dilemma: Is error reduction frequency sufficiently high to warrant the effort? Consider digital pathology system (e. g. Scan. Scope) • • • Digitize slides Desktop computer viewing Multiple viewer conferencing Can Link through telepathology to remote locations/single pathologists Improved turnaround and better use of path times Can correlate slides with CT and MRI scans

Anatomic Pathology Errors Solutions/Tools • Frozen/permanent section, discordant, monitoring – sustained improvement in performance Raab et al ADLM 2006 • § FNA / Histologic correlation • Toyota production system redesign – standard terminology and immediate interpretation • Fewer diagnostic errors Raab et al ASCP 2006 • Improving skill / concentrating expertise in FNA Cytology/Histologic correlation Q tracks program showed improvement in pap. smear performance in preanalytic sampling Raab et al APLM Jan 08

Anatomic Pathology Errors Solutions/Tools • Adopt a system for measuring performance of key processes • • Participation in cooperative programs access multiple institutions/databases: • • e. g. IQLM (U. S. ) – 12 core indicators to evaluate lab quality Q-Tracks Q-Probes System wide approach to reporting critical values Conference, random, focused, amended report, tumour board reviews

Anatomic Pathology Errors Solutions/Tools • Improve access to clinical information • • • A culture that supports change • • • Electronic Medical Record Better defined linkages to a large variety of clinical microsystems (users) Teamwork Willingness to challenge each other Acknowledging error in a non-punitive way Sharing performance information Knowledgeable well trained staff • • Departmental CME Education in QI and safety methods

Diagnostic Imaging Errors • • Technological and manpower factors influence quality in diagnostic imaging Radiology’s Achilles heel: • Error and variation in the interpretation of the Roentgen Image, now the weakest aspect of clinical imaging Robinson, St. James UH Leads, UK 1997

Diagnostic Imaging Errors • Observations of Henry Garland in 1959: • • • 30% of chest radiographs that are positive for disease will be missed Awakened the profession to the extent of errors But have things changed? • • Goddard et al BJR 2001 – little change in past 50 years Internal error rate by same radiologist can be as high as 25% - 30%

Diagnostic Imaging Errors • Shively – Imaging economics 2003: • • • Many could be avoided if a simple protocol followed Errors in stroke CT fell from 15% to 1% Shriger, JAMA 1998 • 49% of radiologists reading CT Scans as part of a large study missed at least 1 stroke

Diagnostic Imaging Errors • Turkington et al PMJ • • • 14 out of 57 cases of confirmed lung cancer missed Delays in diagnosis and treatment Non-radiologists in emergency departments – rate of misinterpreted radiographs is high (many studies) 20 -25% for CT scans

Diagnostic Imaging Errors • Kruskal, Radiology 2006 • • On-line quality assurance reporting system – Beth Israel (Harvard) – 329 cases in 9 months • Communication errors 18% • Interpretation errors 20% • Missed diagnoses 30% • Procedural complications 16% Renfrew – Radiology 1992 • • • 182 reported errors 126 perceptual 56 mishaps

Diagnostic Imaging Errors • Washington Post 2006 based a study by U. S. pharmacopeia • Medication errors that cause harm are 7 times more frequent in radiology departments than in other hospital settings

Diagnostic Imaging Errors Reasons for Error • • Failure to consult old reports Incomplete clinical history Failure to suggest next appropriate procedure Technique limitations

Diagnostic Imaging Errors Reasons for Error • Knowledge problems • • Errors in interpretation Errors in perception Failure to communicate in a timely or clinically appropriate manner Interpretation by non-radiologists

Diagnostic Imaging Errors Reasons for Error • Quality performance bar, Lau BIIJ 2007 Access Workload Access Workload Accuracy TAT • Interlinked • Output pie is only so big • Increased expectations compromise accuracy

Diagnostic Imaging Errors Solutions/Tools § Have a plan and priorities for quality improvement and safety, consider: § § § Health Authority priorities Standards of professional bodies Guidance in literature Performance data Internal assessment – Process map Identify priority projects § § § Mission Culturally aligned teams Improvement methods (PDCA) and root cause analysis, lean design 6 sigma leap frog

Diagnostic Imaging Errors Solutions/Tools QUALITY MAP Patient Physician Patient Exam Appropriateness Orders Test Access Radiology Department Schedules Waiting Times Standard Protocol Global Outcome Reviews finding/ treats patient Finalization Times Exam Performed Performance Outcomes Patient Satisfaction Interpretation Structured Report Radiologist Protocol Selection Finalization

Diagnostic Imaging Errors Solutions/Tools Radiology scorecard. Each quality metric from the quality map (Figure 1) and key safety metrics are listed in the left-hand column. Departmental divisions and operational groups are listed in the top row. Metrics are provided for each box in the scorecard, and the box is color coded (green, yellow, and red) depending upon operational performance. Practice problems can be quickly identified using this tool.

Diagnostic Imaging Errors Solutions/Tools • Examples: • • Knowledge of history and clinical findings – EMR Careful selection of radiological investigation and linkage to clinical protocols A process to ensure comparisons with previous studies Improvement in working conditions and available time

Diagnostic Imaging Errors Solutions/Tools • Examples: • • A process for review and timely follow-up on all discrepancies on images ordered by and interpreted by non-radiologists Development of a quality and safety performance reporting/monitoring system Develop targeted areas for prospective clinical surveillance to identify areas needing improvement Address potential for medication incidents

Diagnostic Imaging Errors Solutions/Tools • Structured anatomic/region specific reports – technology assisted • Voice automated • Structured report templates • Sensitive to clinical requirements • Standardization/consistency • Reduces transcription errors • Faster TAT • Improves report clarity • Linkage to database • Facilitates peer review

Diagnostic Imaging Errors Solutions/Tools § Peer review, consider: § § 5% review mandated by ACR. Will small sample size enable valid individual or departmental reviews? SMPBC • • False negatives identified through linkage to cancer registry Feedback to program leaders and individual radiologists

Diagnostic Imaging Errors Solutions/Tools • Technology enabled peer review built into work routine (e. g. RADPEER) • • • Software enable second review of past reports/films to be submitted on electronic format Can evaluate past reports – scoring system Can be done rapidly Central data bank for peer comparisons, departmental reviews and individual reviews Meaningful data International Radiology Quality Network

Pathology/Diagnostic Imaging Clinical Integration • • • Closer interaction between clinicians and those in diagnostic services is associated with better outcomes The development of clinical protocols and structured diagnostic reports can enable requirements of each to be addressed Consider the clinical microsystem to strengthen engagement with clinical care teams

Health System as an inverted Pyramid Clinical Microsystem Patients & Family Needs CCU Stroke ICU Renal ED Mesosystem • departments • Programs • Clinical evidence base • System support • Clinical quality measures Macrosystem Senior Leaders Board Blunt end Quality by Design Batalden Sharp end

Diagnostic Services and Clinical Microsystems Diagnostic Services • Integration • Information transfer • Coordination • Participation in clinical requirements

Summary/Conclusions • Diagnostic Services • • Consider QI and safety as part of a system Have a plan with priorities Address cultural barriers Develop knowledge in QI methods/tools/root cause analysis Develop performance measurement Enhance clinical integration Push for technology

Summary/Conclusions § Provincial Policy and Health Authorities § § § • Enabling technology Redesign/reengineer the system Capital equipment planning DAP • • • Peer review standards Surveyor preparation/survey tools Facilitate sharing/best practice dissemination