Using routine data to measure and promote safety

Using routine data to measure and promote safety and quality in hospitals Stephen Duckett Adjunct Professor, ACERH/UQ (Joint work with Dr Terri Jackson Associate Professor, ACERH/UQ) THE UNIVERSITY OF WESTERN AUSTRALIA

Overview § What do we know about patient safety and patient harms in Australia? § What do we count as patient harm? § How can we measure and monitor it? § How much does it cost? § How can use of routine data help us understand the economics of improving patient safety? § Payment policy and patient safety

What do we know about Australia? § Landmark 1995 QAHC study: • • Careful (expensive) methods Incidence: 16. 6 % of multi-day inpatient stays Revised down to 10% (comparative US study) Costs to system: $900 mil pa § The newspapers: • Bundaberg • The NSW ‘Cam affair’ (Cambelltown & Camden) • WA’s Royal Commission § Australian Commission on Safety & Quality in Health Care (Building on Aust Council for S&QHC (1999 -2005)

What and how you ‘count’ depends on why you want to count it WHY count?

We need to monitor rates routinely: using hospital discharge data § ICD-10 -AM has specific codes: • T 80. 0 -88. 9 ‘Complications of surgical & medical care’ • ‘End of chapter’ codes • Y 40 -84. 9 ‘External cause--complication of surgical or medical care’ § Australia has world-class quality hospital data § Victorian (and now Oz) ‘condition-onset flag’ (C -prefix) denoting: • Condition required treatment or extended LOS • Condition was not present on admission

Improving patient safety requires hospital-acquired incident cases § Problem of distinguishing ‘comorbidities’ from ‘adverse events’ § Incident vs prevalent cases • Not GP, nursing home, admission from another hospital • Overall rate in Victoria (2000/01): 8. 25% (15. 9% for multi-day stays) • Only two-thirds (5. 61%) were ‘Incident’ cases § The ‘C-prefix’ adds valuable information • 41% of all hospital-acquired diagnoses were missed by ICD alone, eg, UTI, atrial fibrillation, pneumonia • Result: new national ‘Condition Onset’ flag on all Dx Jackson TJ, Duckett SJ, Shepheard J, & Baxter KG. ‘Measurement of adverse events using ‘incidence flagged’ diagnosis codes’ Journal of Health Services Research and Policy, 11 (1): 21 -25; 2006.

Strengths of these data § Strengths • • • Timely & cheap data collection Standardised definitions and coding rules Includes both dramatic and mundane Independent reporting (not bedside) Current payment incentives for thorough coding

Weaknesses of these data • May miss same-day patient harms § Less coding investment § Symptoms appearing post-discharge • Prefix not currently audited • Coders may miss clues clinicians could spot • No judgement about ‘preventability’ § Notorious inter-rater reliability problems in this judgement § May be a ‘strength’ as today’s routine complication becomes tomorrow’s ‘preventable’ adverse event…

Risk factors and outcomes for hospital-acquired diagnoses (HA Dx) § Age is an important predictor of HA Dx • 29% of patients over 85 vs 9% in the 5 -9 age group § Risk of HA Dx varies considerably by medical specialty • Gastroenterology 86. 0% vs ENT surgery 10. 5% § ALOS strongly associated with HA Dx • 11. 2 days with HA Dx vs 6. 9 days without § ALOS both a risk factor and an outcome § In-hospital mortality associated (not caused) • 4. 09% with HA Dx vs 1. 75% without Duckett SJ, Jackson TJ, Hong Son Nghiem (2008). Risk factors and outcomes for incident complications in multi-day admissions to Victorian hospitals (manuscript).

Voluntary Sentinel Event reporting compared with routinely-coded hospitalacquired diagnoses* § Study Objective: To compare two sources of data on hospital sentinel events to evaluate strengths and weaknesses of both § Problems with voluntary reporting: • • Stigma and blame attached to involvement in adverse outcomes Resulting reluctance to report Safety-aware hospitals appear to have ‘worse’ outcomes Focus on single events rather than rates reinforces individual rather than system causation Jackson T, Moje C, Shepheard J, and Mc. Millan A. Poster presented to the 2007 Australian Conference on Safety and Quality in Healthcare, August, 2007.

Using routine data to identify sentinel events This SE Reporting Results Study Victoria National 05/06 04/05 SE 1 Wrong patient or body part 13 25 53 SE 2 Suicide in an inpatient unit 3 7 25 53 6 27 SE 4 Intravascular gas embolism 6 0 1 SE 5 Transfusion reaction 0 0 1 20 2 7 SE 3 Retained instruments /material SE 6 Medication error SE 7(1) Maternal death (O 95, O 96, O 97) 0 SE 7(2) Any maternal death 0 2 16

Public Reporting § Risk adjustment important for public comparisons • Reputation risk • Risk of patients being denied treatment • Fairness to providers § Public officials expected to intervene when quality declines § What do you report? • What providers have ‘done’ or • What they are now doing to improve?

Any measurement approach must balance the risks ‘GOLD STANDARD’ True Positive True Negative Identified Positive Identified Negative False Negative: harm to future patients False Positive: harm to reputations

Problems with public reporting using routine clinical data § Timeliness – data provided on an annual basis, about 15 month delay in publication § Mistrust of ‘administrative’ data • • Not ‘risk adjusted’ Insufficient clinical detail § Aggregated Data – unable to detect runs § Nothing about cause of differences in rates

Queensland Health’s Variable Life Adjusted Display (VLAD) Plot of the cumulative difference between expected and actual outcomes over a period of time Coory M, Duckett SJ, Sketcher-Baker K. Using control charts to monitor quality of hospital care with administrative data. International Journal for Quality in Health Care 20(1): 31 -39.

Simplified Variable Life Adjusted Display (VLAD) Patient survives - VLAD increases by the probability of the patient dying

Simplified Variable Life Adjusted Display (VLAD) / 2 Patient dies: VLAD decreases by the probability of the patient surviving

Simplified Variable Life Adjusted Display (VLAD) / 3 2 nd patient survives: VLAD increases by the probability of the 2 nd patient dying

Simplified Variable Life Adjusted Display (VLAD) / 4 2 nd patient dies: VLAD decreases by the probability of the 2 nd patient surviving

Variable Life Adjusted Display (VLAD) Plot of the cumulative difference between expected and actual outcomes over a period of time

Lessons: § Sometimes statisticians’ methods are somewhat dubious § Be wary of replicating them, even if you think you understand them

Specific Clinical Indicators § Developed in consultation with clinical expert groups § Indicators reviewed annually and refined on basis of feedback § Data from the Queensland Hospitals Admitted Patients Data Collection and the Perinatal Data Collection § Sex, age and comorbidities used to risk-adjust for illness severity and co-existing conditions

Clinical Indicators being developed Mortality Ø Ø Ø Readmission Acute Myocardial Infarction Heart Failure Stroke Pneumonia Fractured Neck of Femur Ø Ø Ø Ø Complications of Surgery Ø Ø Ø Ø Laparoscopic Cholecystectomy Vaginal Hysterectomy Abdominal Hysterectomy Fractured Neck of Femur Colorectal Carcinoma Knee Replacement Hip Replacement Prostatectomy Acute Myocardial Infarction Heart Failure Knee Replacement Hip Replacement Depression Schizophrenia Paediatric Tonsillectomy and Adenoidectomy O&G Ø Ø Ø Future Ø Ø Ø Caesarean Section Rate Std Primiparae Induction of Labour Rate Std Primiparae Perineal Tears (3 rd or 4 th Degree) Hysterectomy – on women < 35 yo Falls Pressure ulcers Overall Surgical Complications AHRQ Patient Safety Indicators

Web-based documentation of VLADs

How bad is a ‘bad run’?

Variable Life Adjusted Display with control limits

Flagging criteria • 30% higher than expected mortality– automated message emailed to the district manager and clinical lead, encouraging internal investigation and report to Area Clinical Governance Unit (50% for non-mortality flags) • 50% higher than expected – flagged to Area Clinical Governance Units to ensure they are involved in further investigation (75% non-mortality) • 75% higher than expected - identified to State Patient Safety and Quality board and in public reporting as being statistically significantly different from the average (100% non-mortality) Duckett SJ, Coory M, Sketcher-Baker K. Identifying variations in quality of care in Queensland hospitals. Medical Journal of Australia 187 (10): 571 -575.

VLAD charts allow tracking to specific case files

Detailed data underpinning the VLAD system

QH Pyramid Model of Investigation

Key strength of routine data: links to patient costs § Victorian and Queensland CWS data • n≈ 1. 6 mil records pa • n≈ 100 large public hospitals • Validated for use in hospital funding § What can cost data tell us? • Economic burden of adverse events • Economic priorities for prevention efforts • Business case for: § prevention efforts § medical research • Cost-effectiveness analyses for: § prevention programs § new patient safety devices and procedures

Incident cases represent a large economic burden to the health care system § Patients with at least one C-prefixed adverse event: • Stay nearly 10 days longer than other patients • Cost $ 6826 more per episode (controlling for DRG, age and comorbidity) § Extrapolated to entire hospital system: • • At least $511. 5 mil additional cost in Victoria (2003/04) Adds 18. 6% to hospital expenditures Around $2 bil pa nationally Even if only 40% preventable: $200 mil pa could be saved in Vic; $800 mil nationally Ehsani JE, Jackson TJ and Duckett SJ. ‘The incidence and cost of adverse events in Victorian hospitals, 2003 -04’ Medical Journal of Australia, 184; 11; 5 June 2006

Re-admissions* add to this cost burden § Current work on Victorian separations with a PDx in the T 80 -88. 9 range of ICD-10 -AM: • 16, 734 admissions with a PDx of a ‘complication of surgical or medical care’ • $70. 6 mil pa additional public expenditure on these cases in Victoria** *Includes admissions for HA Dx from primary care and nursing homes **Mc. Nair P, Jackson TJ, Borovnicar D. ‘Costs of Victorian admissions for treatment of adverse-event principal diagnoses, 2005/06’ submitted for publication, August 2008.

Queensland Health P 4 P approach § Incentives at margin ($8 M out of $7 B budget, 0. 1% of total, larger % of specialty) § Range of specialties, data collection approaches, foci § Specialties: • • • Mental health Stroke Emergency department care COPD Medication Duckett, S. J et al (2008) ‘P 4 P in Australia: Queensland’s New Clinical Practice Improvement Payment’ Journal of Health Services Research and Policy 13: 174 -177

Mental health indicators Indicator Payment amount etc Patients with the DRG Schizophrenia seen $100 by a community mental health professional Volume: 4518 within 7 days following discharge from the same district mental health service provider. Electronic data collection in place Recording of antipsychotic injection (depot) medication on i. Pharmacy for DRG Schizophrenia $100 Volume: 4518 Electronic data collection in place

Acute stroke indicators Indicator Payment amount etc Acute Stroke patients with acute ischaemic stroke receiving antiplatelet therapy within $50 48 hours if clinically appropriate Acute Stroke patients receiving dysphagia screen (minimum requirement) within 24 hours Volume: 6777 Manual data collection system in place $125 Volume: 8699 Manual data collection system in place

Emergency indicator Indicator Payment amount etc All patients aged 65 years (or 50 years if ATSI) and over who are admitted to and discharged from an emergency department to home/nursing home have $100 evidence of communication back to the GP/LMO Volume: 28962 Electronic data collection in place

Medication indicator Indicator Payment amount etc Electronic Discharge Medication Record completed where patient is over 65 years and has a complex medication regimes $100 Volume: 70000 Electronic data collection in place

COPD indicator Indicator Payment amount etc Pulmonary rehabilitation program meets an acceptable standard as defined by the Statewide Chronic Obstructive Pulmonary Disease (COPD) Clinical Network. $10 000 Volume: 20 (review of unit) 12 month audit of all sites (one off data collection)

Queensland Health approach § § § Moral suasion will be used to encourage most of (80%) CPIP payment to flow to clinical unit where indicator activity is being undertaken. CPIP only to be used for non recurrent expenditure items CPIP to be used to enhance not expand service (eg increase number of operations performed)

CPIP - summary § Started in January 2008 § Experiments with very different approaches initially § Mostly Pay for process adherence/reporting rather than outcomes § Can be rolled out to other indicators, add/drop indicators § Stephen_duckett@health. qld. gov. au