Department of Biomedical Informatics University of Utah Practice-Based Evidence: A New Paradigm for Comparative Effectiveness Research October 23, 2014 by Susan D. Horn, Ph. D Institute for Clinical Outcomes Research 699 East South Temple, Suite 215 Salt Lake City, Utah 84102 801 -466 -5595 (V) 801 -718 -9149 (C) shorn@isisicor. com www. isisicor. com 1
History of the Problem • How best to treat the patient in your office right now? • “Scientific studies” (i. e. , RCTs) provide imperfect guidance • Clinical medicine is untidy; innumerable variables describe patients, providers, practices, and outcomes • Must consider clinical variability of patient populations, intervention combinations, and outcomes 2
Objectives of Presentation 1. Present examples of findings from PBE studies in stroke, traumatic brain injury, and RSV infections in children 2. Describe PBE study designs used in these examples and compare PBE to other designs used in CER 3
Post-Stroke Rehabilitation Study 2001 – 2003; 1, 161 patients Study Objectives PBE study designed to discover what combinations of medical devices, therapies, medications, feeding approaches, and their interactions worked best for specific types of stroke patients treated in realworld practices. 4
Post-Stroke Rehabilitation Study Trans-Disciplinary Project Clinical Team • • • Physicians Nurses Social Workers Psychologists Physical Therapists • • • Occupational Therapists Recreation Therapists Speech/Language Pathologists 5
Post-Stroke Rehabilitation Study Examples of OUTCOME VARIABLES • Change in FIM score • Deep vein thrombosis • Length of rehab stay • Major bleeding • Discharge disposition • Pulmonary embolism • Contracture • Pressure ulcer • Death • Pneumonia 6
Post-Stroke Rehabilitation Study Examples of PROCESS VARIABLES • • • Medications • Intensity, frequency, and duration of OT interventions • Intensity, frequency, and duration of SLP interventions • Otherapy interventions and dosage Nutritional process Pain management Time to first rehab Intensity, frequency, and duration of PT interventions 7
Post-Stroke Physical Therapy Form 8
Traumatic Brain Injury What treatments are associated with better outcomes at rehabilitation discharge for patients with traumatic brain injury? 9
Children Hospitalized with RSV What treatments are associated with better outcomes for children hospitalized with RSV infections, controlling for child differences? 10
What We Have and What We Need We have Efficacy trials that determine whether an intervention produces a specified result(s) under well controlled conditions in a selected population – includes randomized controlled trials (RCTs). We need Effectiveness trials that measure outcomes of an intervention under “real world” conditions in an unselected clinical population. Hypotheses and study designs for an effectiveness trial are formulated based on conditions of routine clinical practice and on outcomes essential for clinical decisions. 11
Databases for Effectiveness Trials • RCT databases • Large claims databases, e. g. , Medicare, Medicaid, CDC • HMO or VA databases from claims and electronic medical records • Specific condition registries, such as arthritis registry • Practice-based evidence study registries PBE studies overcome limitations of RCTs (that limit patient types and treatments) More detailed patient, process, and outcome evaluation than is possible with 12 traditional registries or large claims datasets
RCT Databases for Comparative Effectiveness Research • RCT databases – gold standard for efficacy and value of interventions • Advantages include: Ø High internal validity Ø Causal inferences can be made since patients with known confounders are excluded and randomization eliminates unknown confounders 13
RCT Databases for Comparative Effectiveness Research (cont) • Limitations include: Ø Small sample sizes – too small to detect uncommon risks Ø Follow-up periods too short to assess long-term benefits/risks Ø Higher-risk patients are excluded typically; limited external validity Ø Level of monitoring is more rigorous than done in routine practice Ø High rates of treatment discontinuation 14
Non-RCT Databases for Comparative Effectiveness Research • Advantages include: Ø Cover thousands to millions of people including minority and elderly Ø Ability to provide treatment exposures and adverse events, including hospitalizations and mortality, over extended periods of time Ø Provide population and subpopulation-based estimates for various outcomes Ø Better suited to evaluate safety as opposed to effectiveness Ø Many exist in electronic form so some data elements may be exported for use in comparative effectiveness research analyses 15
Non-RCT Databases for Comparative Effectiveness Research • Limitations include: Ø Restricted ability to capture patients’ severity of illness, functional and cognitive status, health behaviors (other than smoking), pain, etc. These can be important unmeasured confounders Ø Restricted access for CER unless researcher is part of organization that owns the data Ø Often required variables are in text format so are not exportable 16
Issues to Address in CER Studies • Minimizing Bias in drawing conclusions from existing databases • Capturing variation in patients • Capturing variation in interventions/ intervention combinations • Capturing variation in outcomes 17
Basic Problem in Non-Randomized Studies • Confounding by Indication « Therapies are administered in non-random fashion « Prognostic characteristics influence therapy used « Recipients of therapy are at high risk for outcomes « Users differ from non-users in key respects 18
Propensity Score Theory to Address Confounding by Indication • PS is a multivariable scoring method that collapses multiple observed predictors of treatment into a single value (a probability score) » PS represents the probability that a subject with given characteristics receives specified treatment » Used to: match, stratify, or model » Assumption is by matching on propensity score, it removes confounding by components included in the score » Sensitive to unobserved predictor variables such as missing severity of illness measures 19
Overcoming Selection Bias/ Confounding by Indication • Statistical adjustments: – Matching – Propensity score or instrumental variables – Covariate adjustments (Severity of Illness) • Ongoing debate about the adequacy of adjustments 20
PBE Methodology What makes this approach different? Why do a PBE study? 21
History of PBE • Started with development of Comprehensive Severity Index (CSI) to measure risk-adjusted outcomes at The Johns Hopkins Hospital • CSI found to explain up to 50% of the variation in outcomes of cost, LOS, mortality • Found patients with the similar CSI scores for a condition could have very different treatments • This stimulated development of PBE study design to account for Ø patient heterogeneity Ø treatment heterogeneity Ø outcome heterogeneity 22
CER Issues Addressed Using PBE • Both patients and providers report data • Data come from existing EMR with standardized structured data elements about patient characteristics, treatments, processes, patient-reported data, and multiple outcomes • Data are part of routine documentation, so not an ‘add-on’ • Rapid patient accrual since documentation is standard of care • Longitudinal and ongoing 23
CER Issues Addressed Using PBE (cont) • Patient comparability is addressed with the Comprehensive Severity Index (CSI): disease-specific, physiologic-based, >2, 200 criteria, >5, 500 disease-specific criteria sets • CSI addresses confounding by indication and selection bias • Database includes all treatments with date/dose/intensity/route. Many details collected in point-of care (POC) documents. • Can assess drug and non-drug combination therapies • Findings of PBE-CER are more readily translated into practice 24
Components of Practice-Based Evidence Designs Standardize documentation for : Process Factors • Patient Education and Management Strategies • Interventions and surgeries • Medications Control for: Patient Factors • Psychosocial/demographic Factors • Co-occurring Conditions • Severity of Illness and Injury • Genetic information • Measured at Multiple Points in Time Measure: Primary Outcomes • Clinical • Health Status • Functional • Cost/LOS/Encounters • Discharge Disposition • Post-discharge Outcomes 25
7 Signature Features of PBE Studies 1. Hypotheses can be focused or broad 2. All interventions are considered to determine the relative contribution of each 3. Broad patient selection criteria maximize generalizability and external validity 4. Detailed characterization of the patient by robust measures of patient severity, genetic information, and functional status 26
7 Signature Features of PBE Studies 5. Patient differences controlled statistically rather than through randomization 6. Facility and clinical/patient buy-in through use of trans-disciplinary Clinical Practice Team 7. Strength of evidence built through the research process PBE findings are more generalizable and transportable 27
PBE Study Hallmarks • Decisions are made by front-line clinicians vs. researchers • “Bottom-up” vs. “Top-down” approach • Guidance from researchers (scientific advisory board) and patient experience 28
PBE Study Hallmarks – Non-experimental: Follows outcomes of treatments actually prescribed – Inclusive: Uses patient populations undergoing routine clinical care – Pragmatic: Uses actual clinical outcomes – Lower Cost than RCTs – Faster than RCTs 29
Practice-Based Evidence Study Design • High external validity - Includes essentially all patients with specific condition or in specific setting(s) - Captures confounders that could affect relevant treatment responses - Reduces accidental associations between treatments and outcomes 30
Components of Practice-Based Evidence Designs Standardize documentation for : Process Factors • Patient Education and Management Strategies • Interventions and surgeries • Medications Control for: Patient Factors • Psychosocial/demographic Factors • Co-occurring Conditions • Severity of Illness and Injury • Measured at Multiple Points in Time Measure: Primary Outcomes • Change in CSI/discharge CSI • Discharge Disposition • Length of Stay • Post-discharge Outcomes 31
Examples of Severity Systems to address Selection Bias/Confounding by Indication Diagnostic/Procedure Based Systems • Clinical definition of severity • Body Systems Count • Charlson Comorbidity Index (1 -yr death) Physiologic/Clinically Based Systems • 2 Apache II & III (ICU death) • 2 Medisgroups (Atlas) (hosp death) • 3 Disease Staging (hosp death) • Patient Management Categories (hosp death) • Resource definition of severity • Case Mix Groups[CMGs] (rehab LOS, $) • Acuity Index Method (LOS) • APR DRGs (hosp $) • Patient Management Categories (hosp $) • Refined DRGs (hosp LOS, $) CSI® 32
Comprehensive Severity Index (CSI®) used to account for selection bias or confounding by indication • Severity defined as “physiologic complexity presented to medical personnel due to the extent and interactions of a patient’s diseases” • Disease-specific: 5, 500 disease-specific groups; over 2, 200 distinct criteria. • • • No treatments used as criteria • Can measure severity at multiple time points • Allows statistical comparison of interventions without confounding by severity of ICD-9 codes trigger disease-specific patient signs, symptoms, and physical findings used to score disease-specific and overall severity levels Comprehensive (all diseases) Clinically credible: computes disease-specific and overall severity levels illness 33
CSI Severity Indicators · Physiological signs and symptoms of a disease - Vital signs - Laboratory values - Radiology findings - Other physical findings · Severity indicators are specific to each disease based on ICD-9 -CM coding 34
Pneumonia Criteria Set 480. 0 -486; 506. 3; 507. 0 -507. 1; 516. 8; 517. 1; 518. 3; 518. 5; 668. 00 -668. 04; 997. 3; 112. 4; 136. 3; 055. 1 35 Copyright 2006. Susan D. Horn. All rights reserved. Do not quote, copy or cite without permission.
Stroke Criteria Set page 1 430 -438. 9, 648. 6 -648. 64, 671. 5 -671. 54, 674 -674. 04 Category Indicator 1 Digestive Nausea/Vomiting No nausea or Neurology Neurological Status No unresponsiveness or confusion GCS vomiting 2 4 N/A Chronic confusion Persistent Vomiting Acute confusion GCS>=12 GCS=9 -11 GCS=6 -8 GCS<=5 Seizures Focal tremors/Seizures NOS Focal/Petit Mal Seizures Grand Mal/Status Epilepticus NOS Status Epilepticus w/ >2 Hrs Drug therapy Pupil Reaction Normal Pupil Reaction Unilateral Pupil Dilation Coordination/Balance Severe Ataxia N/A Aphasia Unsteady on Feet/Clumsiness Dizziness/Mild to Moderate NOS Ataxia Sensation Alteration NOS Complete loss of Sensation/Parathesia or Dysesthesia No Aphasia Mild Aphasia Bilateral Pupil Dilation N/A Dysarthria No/mild Dysarthria Dysphonia Dysarthria Incomprehensible Speech No Speech Dysphagia NOS Unable to swallow liquids Unable to swallow solids N/A Headache NOS, No Headache Moderate/severe headache Intense headache N/A Sensation alteration Pain Copyright Vomiting 3 2006. Susan D. Horn. All rights reserved. Do not quote, copy or cite without permission. Unresponsive Moderate/Severe Aphasia Global Aphasia 36
Stroke Criteria Set page 2 430 -438. 9, 648. 6 -648. 64, 671. 5 -671. 54, 674 -674. 04 Category Indicator Respiratory Dyspenea Breathing Difficulties NOS Dyspnea on exertion Dyspnea at Rest N/A Rales No Rales <=50% /<3 Lobes Rales >=50% />=3 lobes N/A Breath Sounds No decreased breath sounds Decreased Breath in <=50% / <3 lobes Decreased Breath in >=50% Absent Breath sounds />=3 lobes in >50% />=3 lobes Apnea No Apnea N/A Senses Perceptual impairment No perceptual impairment Acute Decline in perceptual N/A impairment Vitals Highest Systolic BP <=180 mm Hg Highest Diastolic BP <=99 mm Hg Chronic Perceptual impairment requiring external/internal cues 181 -219 mm Hg 100 -109 mm Hg >=220 mm Hg >=110 mm Hg N/A Lowest Systolic BP >=90 mm Hg 80 -89 mm Hg 61 -79 mm Hg <=60 mm Hg Highest Pulse rate <=99 Beats/min 100 -129 Beats/min >=130 Beats/min N/A Lowest Pulse Rate >=51 Beats/min 41 -50 Beats/min 31 -40 Beats/min <=30 Beats/min Highest Temp <=100. 4 Oral F >=100. 5 Oral F N/A Lowest Temp >=96. 8 Oral F <=96. 7 Oral F N/A O 2 Saturation No supplemental Oxygen, O 2 Oxygen 22 -50%, able to obtain Oxygen> 50% and able to Sat >= 90 O 2 Sat >=90 Obtain O 2 Sat >= 90% or Oxygen 22 -50% and unable to obtain O 2 sat >=90% No EKG Ectopy, Non Bigeminy/ 6 PVCs/min, SVT sustained Ventricular trigeminy/Quadrigeminy/Atrial Junctional ectopic Tachycardia fibrillation tachycardia EKG Rhythm Copyright 1 2 2006. Susan D. Horn. All rights reserved. Do not quote, copy or cite without permission. 3 4 Apnea Oxygen >50% and unable to obtain O 2 Sat >=90 Runs of ventricular tachycardia 37
Post-Stroke Rehabilitation Study 2001 – 2003; 1, 161 patients Study Objectives PBE study designed to discover what combinations of medical devices, therapies, medications, feeding approaches, and their interactions worked best for specific types of stroke patients treated in realworld practices. 38
Outcome: Discharge Motor FIM Severe Stroke – Full Stay General Assessment – Age PT Interventions – Formal assessment – Bed mobility + Mild motor impairment + Gait + Admission Motor FIM + Advanced gait + Admission Cognitive FIM – Black race OT Interventions + Home management SLP Interventions – Swallowing – Orientation + Reading comprehension Medications General Interventions – Days onset to rehab + Enteral feeding – Anti-Parkinsons – Modafinil – Old SSRIs + Atypical antipsychotics 39
Outcome: Discharge Motor FIM Severe Stroke– 1 st 3 hour Therapy block only General Assessment PT Interventions OT Interventions SLP Interventions – Age – Bed mobility + Home management st 3 hrs – Severe motor impairment time in 1 + Gait time in 1 st 3 + Admission Motor FIM hrs + Admission Cog. FIM + Advanced gait + No Dysphagia time in 1 st 3 hrs + Neurotropic Impairments Medications General treated with meds Interventions – Other Antidepressant – Days onset to rehab – Old SSRIs + LOS + Atypical antipsychotics Horn et al. , Arch Phys Med Rehabil 2005; 86(12 Supplement + Enteral feeding 40 2): S 101 -S 114
Policy Changes from Stroke PBE Study Early Rehabilitation Admission – get patient into rehabilitation as soon as possible after stroke onset; possibly start in Neuro ICU Early gait in PT – start gait as soon as possible after rehab admission; put patient in harness on treadmill for safety Early Feeding – continue or start enteral nutrition at rehab admission if patient is not able to eat full meals Use Opioids for Pain – continue or start opioids at rehab 41 admission if patient misses therapy due to pain
Traumatic Brain Injury What treatments are associated with better outcomes at rehabilitation discharge for patients with traumatic brain injury? 42
TBI Admission FIM Cognitive Subgroups (N=2130) Ns decrease due to non-consent for follow up, death, or incarceration Adm FIM Cognitive N during Rehab N at 3 months N at 9 months Score <=6 339 286 262 Score 7 -10 374 312 302 Score 11 -15 495 411 394 Score 16 -20 408 326 311 Score >=21 504 401 373 Total 2120* 1742** 1649*** *N=10, **N=6, ***N=7 Missing Admission FIM cognitive score 43 43
Point-of-Care (POC) documentation- Physical Therapy 44
Discharge Rasch-Adjusted FIM Motor Regression: Variance Explained Step R² Adm Cog Adm Cog <=6 7 -10 11 -15 16 -20 >=21 Pat/Inj 0. 38 0. 48 0. 54 0. 71 Pat/Inj + POC total 0. 41 0. 49 0. 48 0. 56 0. 72 Pat/Inj + POC act/ LOE 0. 74 0. 70 0. 62 0. 76 Pat/Inj + meds 0. 41 0. 48 0. 55 0. 72 0. 74 0. 70 0. 63 0. 76 0. 74 0. 71 0. 63 0. 65 0. 78 Pat/Inj + POC act/ LOE + meds + sites 45 45
All Regressions Summary: Patient and Injury Significant Covariates Red = negative coeff, p<. 05; Green = positive coeff, p<. 05 Covariate FIM Cog FIM Motor Age Avg LOE Inj to Adm HS, No Dplma BI CSI non-BI CSI Black Race Rehab Length of Stay (Lo. S) Discharge to Home (dc. H) Cog <=6 Cog 7 -10 dc. M dc. C dc. H fu. M fu. C Lo. S dc. M dc. C fu. M dc. M dc. H fu. C dc. M dc. H fu. M fu. C dc. M dc. C dc. H dc. M dc. C fu. M fu. C Cog 11 -15 Cog 16 -20 dc. C Lo. S dc. C Cog >=21 dc. C Lo. S dc. M dc. C dc. H fu. M fu. C dc. M dc. C dc. H fu. M Lo. S dc. M fu. M fu. C Lo. S dc. M fu. M dc. H fu. M Lo. S dc. M dc. C dc. H fu. M fu. C Lo. S fu. M Lo. S dc. M dc. C dc. H fu. M fu. C dc. C fu. C Lo. S fu. M fu. C fu. M fu. C Lo. S dc. H dc. M Lo. S dc. H Lo. S dc. M fu. M Lo. S fu. M Lo. S fu. M Lo. S dc. M Discharge FIM Motor (dc. M) 9 -Month FIM Motor (fu. M) Lo. S dc. M dc. C Discharge FIM Cognitive (dc. C) 9 -Month FIM Cognitive (fu. C) Lo. S dc. M 46
All Regressions Summary: Treatment Significant Covariates Red = negative coeff, p<. 05; Green = positive coeff, p<. 05 Covariate Adm Cog <=6 OT Education/Sexuality Min/Wk dc. H fu. M fu. C dc. H Lo. S OT Home IADLs Min/Wk dc. M dc. H dc. M OT Physical Impairments Min/Wk dc. M fu. M dc. M Lo. S PT Advanced Gait, Community Mobility, Stairs Min/Wk dc. M fu. M dc. H PT Equip Mgmt, WC/Bed Mobility, Casting, Sitting, Trnsfrs, Develop Seq Min/Wk dc. M fu. C Lo. S dc. M dc. C dc. M dc. H dc. M dc. C PT Formal Assessment Min/Wk Lo. S fu. M Lo. S dc. H fu. M Lo. S ST Education Min/Wk dc. H Lo. S dc. H fu. C Lo. S dc. H dc. C dc. H ST Basic Motor/Speech Min/Wk dc. M Lo. S dc. M dc. C fu. M dc. M fu. M dc. M dc. C fu. C Adm Cog 7 -10 Adm Cog 11 -15 Adm Cog 16 -20 Adm Cog >=21 ST Problem Solving, Math, Money, Memory, Orientation Min/Wk dc. M dc. C fu. M fu. C % Stay Atypical Antipsychotics Lo. S dc. M dc. C Rehab Length of Stay (LOS) Discharge FIM Motor (dc. M) Discharge FIM Cognitive (dc. C) Discharge to Home (dc. H) 9 -Month FIM Motor (fu. M) 9 -Month FIM Cognitive (fu. C) Red = negative coeff, p<. 05 Green = positive coeff, p<. 05 47
Children Hospitalized with RSV What treatments are associated with better outcomes for children hospitalized with RSV infections, controlling for child differences? 48
Pediatric Bronchiolitis Study Outcome = Cost n=722; R 2=0. 73 Assessment - Age in months (. 0001) + MCSIC (. 0001) Procedures + Admitted to PICU (. 0001) + Arterial line (. 04) + Central line (. 003) + Continuous nebulization (. 0002) + Interaction: chest pt & atelectasis (. 005) + Intubation (. 0001) + Ipratropium bromide (. 005) + Lasix (. 0001) + Ribavirin (. 0001) + Steroids (. 0003) Willson, et al. PEDIATRICS 2001; 108(4): 851 -855. 49
Prematurity and RSV Hospital Outcomes Significant Differences by Gestational Age Groups 33 -35 week GA infants had highest hospital resource use < 32 wks 33 -35 wks 36 wks > 37 wks p-value Intubation 21. 4% 38. 7% 20% 12. 1% 0. 002 ICU LOS 5. 8 days 7. 7 days 4. 2 days 3. 8 days 0. 021 Hospital LOS 6. 8 days 8. 4 days 4. 9 days 4. 1 days <0. 0001 Admitted to ICU 39. 3% 48. 4% 30. 0% 27. 9% 0. 101 HX of Hosp. for RSV /Bronchiolitis 14. 3% 16. 1% 6. 7% 6. 1% 0. 137 50
RSV Hospital Outcomes and Policy Changes Conclusions • 33 -35 week GA infants had highest hospital resource use • 36 week infants have risk similar to full term infants • Changed guidelines for immunoprophylaxis for 33 -35 week infants • Changed guidelines for intubation – try ‘stimulating’ first 51
Summary • PBE methodology provides a structured way to design studies of patients in routine care settings • PBE studies develop comprehensive databases of patient, treatment, and outcome differences • PBE findings associated with better outcomes are easily transferable for use in other sites because all patients with a condition can be included in a PBE study 52
Скачать презентацию Department of Biomedical Informatics University of Utah Practice-Based
afe17df78ba994b0ba17e8cb8e021119.ppt