QII in the Practice Partner Research Network Group

QII in the Practice Partner Research Network: Group Randomized Trials and Other Designs Steven Ornstein, MD Professor of Family Medicine and Director Practice Partner Research Network Medical University of SC Charleston, SC, USA Ornstesm@musc. edu

Presentation Goals Introduce PPRNet Present 2 PPRNet QII studies: TRIP-II (GRT) A-TRIP (Time-series) Discuss strengths and weaknesses of each project and lessons learned from them that may be applicable to other QII in primary care

PPRNet is… … “a practice-based learning and research organization designed to improve health care in its member practices and elsewhere in the United States. ”

PPRNet is… Vendor, PMSI Seattle Consultants, Collaborators Research Offices MUSC, Charleston Practices

PPRNet Practice Membership Interested users of Physician Micro Systems Inc (PMSI) Practice Partner Patient Records EMR 101 practices (mostly FP, IM) 502 clinicians 37 U. S. States

Practice Size # of Clinicians % of practices 1 or 2 25 3 or 4 35 5 -9 30 10+ 11

PPRNet Data Collection/Analyses ctice Pra Q 3 months Practice R eport PMSI PRNet P

PPRNet’s Motto “Blurring the distinction between quality improvement and research”

PPRNet Quality Improvement Studies Project Funder Dates Design Practices Conditions TRIP-II AHRQ 2000 -03 GRT A-TRIP AHRQ 2003 -06 TS AA-TRIP NIAAA 2004 -07 GRT 22 Alcohol C-TRIP NCI 2006 -09 GRT 30 CRC screening 20 CVD >100 Many

TRIP-II: Primary and Secondary Prevention of Cardiovascular Disease and Stroke in Small Primary Care Practices Funded by AHRQ as part of TRIP-II RFA (Grant No. 1 U 18 HS 11132 -01. ) Project period 9/30/00 to 2/28/04

TRIP-II Methods Group randomized trial 2 years (1/01 to 1/03) 20 non-academic FP and IM practices in 14 States 61 total HCP (44 MD, 17 mid-level) 87, 291 patients >= 18 y. o. visit in 3 years

TRIP II Methods: 10 Control Practices Quarterly practice-level performance reports on 21 indicators of care based on guidelines from JNCVI, NCEP ATP-III, ACC/AHA, ADA Each practice had EMR and organized approaches to improvement on their own

TRIP II Methods: 10 Intervention Practices Practice performance reports 6 -7 practice site visits for academic detailing & QI facilitation with participatory planning, use of EMR tools, complexity science, best practice approaches 2 network meetings to share “best practices”

TRIP II Analyses Practice-level analyses: randomization test comparing improvement in targets reached between groups, Wilcoxon signed-rank tests to assess improvement in each group Patient- level analyses: generalized mixedregression models with random practice effects Qualitative methods (interviews, review of archival documents to identify improvement model)

TRIP II Results--Quantitative Practice-level analyses: improvement in both intervention and control group practices in % of targets reached Intervention group had greater improvement in 18 of 21 indicators Patient-level analyses: improvement in intervention group > control group for 2 of 21 measures

TRIP II Results--Qualitative The PPRNet QI model that emerged emphasized the importance of “Prioritizing Performance”, “Involving All Staff”, “Delivery System Redesign”, “Patient Activation”, and “EMR Tools” Another model focused on “Leading a Practice through QII”

Nemeth, 2005

TRIP-II Challenges and Resolutions

Dealing with “Buy-in” Some providers in larger practices did not “buy” in to study (denial, competing obligations, lack of self-efficacy, etc. ) Response was to focus on more amenable members and have them model changes for others

Misplaced Focus of Site Visits Initial emphases on academic detailing and EMR optimization met reality that providers knew guidelines and wanted to use EMR in their idiosyncratic way Response was to change focus of site visits changed focus to QI approach at microsystem level

Importance of Non-Provider Staff Insufficiently recognized at study initiation. Response was to encourage these individuals to play greater role in participatory planning, implementation at later visits. Also, nonprovider staff from each practice participated in 2 nd network meeting.

TRIP-II QII Lessons/Conclusions

Clinicians will Participate PCP with EMR systems that facilitate QI reporting and interventions and that volunteer to participate in QII generally enjoy the work and will participate in an intervention that they deem beneficial to their patients

The EMR is not the Panacea A more robust QII model was needed. Simply giving practices the information (academic detailing) and the tool (EMR) is insufficient

One-Size Does Not Fit All Given the constraints of practices, intervention approaches and emphases have to be customized at the microsystem-level Study sections accustomed to specific protocols that require rigid adherence to assure this requirement may need to appreciate this reality

Accelerating TRIP (A-TRIP) in a Practice-based Research Network Funded by AHRQ as part of Partnerships for Quality RFA (Grant 1 U 18 HS 13716) Project period 9/30/02 to 9/29/06

A-TRIP Goals Recruit 100 primary care practices Enhance PPRNet practice performance reports to include ~80 indicators in 8 discrete clinical areas from based on guidelines from JNCVI, NCEP ATPIII, ACC/AHA, ADA, USPSTF, Beer’s criteria. Disseminate PPRNet-TRIP QI improvement methods/model to participating practices and assess impact on practice-level performance on ~80 indicators and 3 summary measures

A-TRIP Methods Demonstration project with descriptive and timeseries evaluation components 2. 75 year intervention (10 -03 to 6 -06)with enrollment until the final six months 101 non-academic FP and IM practices in 37 States 502 total HCP ~500, 000 patients >= 18 y. o.

PPRNet TRIP Intervention Methods Practice Performance Reports (all practices) Practice Site Visits q 6 months (62 practices to date) Network Meetings (~1/2 of practices at one of 3 meetings to date) © PPRNet, 2003

Clinical Practice Guideline Measures in PPRNet Practice Reports Diabetes mellitus (13) Heart disease and stroke (21) Cancer screening (12) Adult immunization (14) Respiratory disease/infectious disease (6) Mental health/substance abuse (8) Nutrition/obesity (3) Inappropriate Rx prescribing in the elderly (2)

Practice Performance Reports Distributed Quarterly 84 indicators SPC methodology (SAS) Comparison with PPRNet benchmark (ABC) and median Comparison with national benchmarks (where available

Site Visit Activities Review of practice report Presentation/discussion/local adaptation of multi-faceted PPRNet-TRIP QI model developed in TRIP-II

Summer Network Meetings One clinician and one staff member invited from each practice Best “practice presentations” Small group workshops, among participants with similar roles in their practices (e. g. , clinicians, nurses, back office staff)

A-TRIP Results Recruitment and reporting goals met Greater than expected participation in practice-site visits; less at network meetings Impact on practice-level performance reports will be assessed at study conclusion — 6 -06, preliminary data encouraging

A-TRIP Challenges and Resolutions

Need for Enhanced Reporting Participants wanted patient-level in addition to practice-level reports to better identify those in need of specific interventions; EMR did not have sufficient native reporting functions Response was to develop/enhance this tool…

Analyses Challenges I Practice attrition of ~10% annually (due to turnover in leadership, retirement, relocation, etc. ) and continued recruitment of new practices create challenges in time series analyses Plan is to include “duration of exposure” as variable in analyses

Analyses Challenges II The absence of a control group will compromise the validity of the findings. Using a “delayed intervention” approach was not feasible in this setting it was not possible to recruit the entire sample of practices at the beginning of the project Looking at a broad set of indicators and for a large effect size as evidence of efficacy may mitigate this limitation

A-TRIP QII Lessons/Conclusions

Clinicians will Participate A great number of PCP with EMR systems that facilitate QI reporting and interventions will participate in QII, particularly when they receive a tangible benefit (e. g. , free practice reports, CME, support to attend meetings), believe that the project is in the best interests of their patients, and can titrate their level of involvement to suit their particular needs/level of interest

Non-Provider Staff Are Key Non-provider staff in primary care settings span the spectrum from RNs to individuals whose only training is provided by the practice. These individuals need proper supervision, focused training, and inclusion as equally respected team members in QI planning activities in the practice setting Practice leaders need to be developed so as to better incorporate these individuals in QII; research is needed on best approaches

Other Relevant Outcomes In addition to clinical outcome measures studies need to look for the behavioral/organizational changes that take place as a result of different intervention strategies so approaches can be tailored efficiently to needs of specific practices

QII in the Practice Partner Research Network: Group Randomized Trials and Other Designs Steven Ornstein, MD Professor of Family Medicine and Director Practice Partner Research Network Medical University of SC Charleston, SC, USA Ornstesm@musc. edu