Slide 1 © 2003 By Default! Physician Prognostic Accuracy for In-Hospital Mortality in Percutaneous Coronary Intervention Michael E. Matheny, MD Medical Informatics Fellow Decision Systems Group Brigham & Women’s Hospital Boston, MA A Free sample background from www. powerpointbackgrounds. com
Slide 2 © 2003 By Default! Specific Aims n Primary Hypothesis – Accuracy of subjective physician estimations of in -hospital mortality will be similar or improved when compared with accepted objective risk assessment methods for percutaneous coronary intervention (PCI) A Free sample background from www. powerpointbackgrounds. com
Slide 3 © 2003 By Default! Specific Aims n Secondary Hypotheses – Accuracy of subjective physician estimations of inhospital major adverse cardiac events (MACE) will be similar or improved when compared with accepted objective risk assessment methods for PCI – Qualitative collection of risk factors could identify additional important risk factors currently not included in the objective risk models – Incorporating subjective physician estimates into an objective risk model will outperform either separately A Free sample background from www. powerpointbackgrounds. com
Slide 4 © 2003 By Default! Background n Discrimination – Ability to predict an outcome on a population level – Area under the Receiver Operating Characteristic Curve (AUC) A Free sample background from www. powerpointbackgrounds. com
Slide 5 © 2003 By Default! Background n Calibration – Ability to predict an outcome on a case/small group level – Hosmer-Lemeshow Goodness-of-Fit Test (HL-GF) – Brier Score A Free sample background from www. powerpointbackgrounds. com
Slide 6 © 2003 By Default! Background n Subjective vs APACHE II Medical ICU Mortality 1 – Discrimination: Objective Better – Calibration: Subjective Better – Forecasting Improves with Training n Subjective vs APACHE II Medical ICU Mortality 2 – Discrimination: Subjective Better – Calibration: No Difference A Free sample background from www. powerpointbackgrounds. com
Slide 7 © 2003 By Default! Background n Subjective vs LR Acute CHF 90 day and 1 year Mortality 3 4 – Discrimination: – Calibration: – All estimations poor No Difference n Subjective vs SNAP Neonatal ICU Mortality 5 – Discrimination: – Calibration: A Free sample background from www. powerpointbackgrounds. com No Difference
Slide 8 © 2003 By Default! Background n Subjective + PRISM III Pediatric ICU Mortality 6 – Discrimination: – Calibration: – Combined model • Discrimination: • Calibration: A Free sample background from www. powerpointbackgrounds. com No Difference Improved from either
Slide 9 © 2003 By Default! Background n Subjective vs LR Model of Post-Op mortality for Open Heart Surgeries 7 – Discrimination: – Calibration: – Combined model • Discrimination: • Calibration: No Difference – Subjective assessments were more calibrated at the extremes of probability A Free sample background from www. powerpointbackgrounds. com
Slide 10 © 2003 By Default! Background n Subjective Physician Assessments – Multiple Forms of Bias 8 • • Ego Bias Regret Ignoring Negative Evidence Framing A Free sample background from www. powerpointbackgrounds. com
Slide 11 © 2003 By Default! Background n No work has been done evaluating subjective physician estimates for in-hospital mortality in percutaneous coronary interventions. A Free sample background from www. powerpointbackgrounds. com
Slide 12 © 2003 By Default! Background PCI Objective Risk Model Gold Standards n Logistic Regression Models • National – American College of Cardiology 9 – 50123 pts 1998 - 2000 • Regional – Northern New England 10 – 15331 pts 1994 - 1996 • Local – Brigham & Women’s Hospital 11 – 2804 pts 1997 - 1999 A Free sample background from www. powerpointbackgrounds. com
Slide 13 © 2003 By Default! Background Pilot Data n Recent Evaluation of Models on Local Institution Data 12 – Discrimination (AUC) • ACC 0. 90 • NNE 0. 89 • BWH 0. 89 – Calibration (HL-GF) • ACC <0. 001 • NNE <0. 001 • BWH <0. 001 A Free sample background from www. powerpointbackgrounds. com
Slide 14 © 2003 By Default! Background n Objective Assessment Models – Multiple Forms of Bias • Population/Demographic Bias – Regional Variances • Selection Bias – Population referral bias • Temporal Bias – Medical Care Standards – Data Documentation • Data Noise – Heterogeneous Data Standards – Variation in Data Element Collection – Data Entry Quality Variations A Free sample background from www. powerpointbackgrounds. com
Slide 15 © 2003 By Default! Background n Incomplete model information? n Best Described Risk Factors Age Sex Ejection Fraction Recent MI Hemodynamic Stability Intra-Aortic Balloon Pump A Free sample background from www. powerpointbackgrounds. com CHF Unstable Angina Renal Failure Hx (CAD, DM, COPD, HTN) Prior CABG or PCI
Slide 16 © 2003 By Default! Study Design n Prospective Cohort Study A Free sample background from www. powerpointbackgrounds. com
Slide 17 © 2003 By Default! Study Population n Location – Brigham & Women’s Interventional Cardiology Suites n Inclusion Criteria – All Patients presenting for pre-operative evaluation for PCI n Exclusion Criteria – Procedural Team declines to participate in survey A Free sample background from www. powerpointbackgrounds. com
Slide 18 © 2003 By Default! Study Data Collection n Paper Survey – Administration refused to allow survey to be part of medical record n Subjective mortality assessment (0 -100%) before and after procedure – Attendings – Fellows – Scrub Nurse n Qualitative additional risk factors from Attendings A Free sample background from www. powerpointbackgrounds. com
Slide 19 © 2003 By Default! Exposures n Percutaneous Coronary Transluminal Angiography with or without Coronary Stenting A Free sample background from www. powerpointbackgrounds. com
Slide 20 © 2003 By Default! Covariates/Confounders Age Sex Ejection Fraction Recent MI Hemodynamic Stability Intra-Aortic Balloon Pump A Free sample background from www. powerpointbackgrounds. com CHF Unstable Angina Renal Failure Hx (CAD, DM, COPD, HTN) Prior CABG or PCI
Slide 21 © 2003 By Default! Outcomes n In-Hospital Death n In-Hospital MACE A Free sample background from www. powerpointbackgrounds. com
Slide 22 © 2003 By Default! Analysis Plan n Measure Discrimination & Calibration on local data for: – Objective MACE & Mortality Models • • • National Regional Local – Subjective MACE & Mortality “Models” n Pair-wise Comparison of Objective and Subjective models for statistical differences n Develop LR model with subjective data as a covariate, and perform pair-wise comparisons with objective and subjective models to determine if new model shows improvement A Free sample background from www. powerpointbackgrounds. com
Slide 23 © 2003 By Default! Analysis Plan n Local Institution Data – ~1% Death Rate – ~5% MACE Rate – ~200 Cases / month A Free sample background from www. powerpointbackgrounds. com
Slide 24 © 2003 By Default! Analysis Plan Rough Guess n Sample Size Calc – Binomial Fisher’s Exact – Α = 0. 05 – Power = 0. 80 – Effect Size & Estimated Sample • Mortality – 1% to 1. 5% = 8150 – 1% to 2% = 2514 • MACE – 5% to 7. 5% = 1550 – 5% to 10% = 473 A Free sample background from www. powerpointbackgrounds. com
Slide 25 © 2003 By Default! Analysis Plan Recruitment n Multi-Center – Exploring recruitment possibilities from Beth. Israel and Massachusetts General Cath Labs n No Termination Date – Implemented as Quality Control method in BWH Cath Lab A Free sample background from www. powerpointbackgrounds. com
Slide 26 © 2003 By Default! Limitations n Sample Size n Paper Survey n Study Population Compliance n Selection Bias A Free sample background from www. powerpointbackgrounds. com
Slide 27 © 2003 By Default! Time Table n IRB Approval – Completed n Physician Survey Template – August 2005 n IC Lab Tech Data Collection Training – September 2005 n Data Collection – September 2005 - Open A Free sample background from www. powerpointbackgrounds. com
Slide 28 © 2003 By Default! Acknowledgements n Co-Authors – Nipun Arora, MD – Lucila Ohno-Machado, MD, Ph. D – Frederic S. Resnic, MD, MS n Funding – NLM 1 -T 15 -LM-07092 A Free sample background from www. powerpointbackgrounds. com
Slide 29 © 2003 By Default! The End mmatheny@dsg. harvard. edu Michael Matheny, MD Brigham & Women’s Hospital Thorn 309 75 Francis Street Boston, MA 02115 A Free sample background from www. powerpointbackgrounds. com
Скачать презентацию Slide 1 2003 By Default Physician Prognostic
4e30991c6784e0a57df738e0ece97750.ppt