The BIDMC Physician Performance Evaluation Process Standard

The BIDMC Physician Performance Evaluation Process

Standard MS. 08. 01. 03 • Ongoing professional practice evaluation is required for privileging • Required elements: – Clearly defined process that facilitates evaluation of every practitioner – Process must include department-specific data being collected (must be approved by MEC ) – Information is used to determine whether privileges are continued, limited or revoked.

What are we required to do?

Current process BIDMC

How do we possibly gather information to make valued assessments?

Current process B & W

How do they possibly gather information to make valued assessments?

“The Personal Radiologist Performance Scorecard” Personal Radiologist Scorecard Medical knowledge Professionalism Interpersonal skills Practice-based learning Patient care Systems-based practice Peer review From a drop-down menu, each radiologist selects 2 -3 defined and approved metrics from each category. Target goals are defined by section and by modality. Target is 5% of annual volume. Data also includes peer review scores for specific radiologist.

“The Personal Radiologist Performance Scorecard” Personal Radiologist Scorecard Medical knowledge Professionalism Interpersonal skills Practice-based learning Patient care Systems-based practice All metrics autopopulate the database from our QI dashboard. Peer review 352 other staff cases reviewed (target 450) 261 of my cases reviewed: 248/7/3/1 29 QA cases submitted to Online Reporting System 8 hours category 1 Risk Management credits

Competency 1: Patient Care Evaluation metrics q q q Compliance testing up to date: HIPAA examination Human Research examination Moderate sedation training course TB testing Timely completion/approval of reports (median <12 hrs) Procedural complication rate (major cx rate: target <1% of cases) Compliance with universal policy (target >95%) Participation in Rad. Peer (>5% prior year volume) Rad. Peer scores 360 o evaluation (from nurses, technologists, residents, patients etc)

Competency 2: Medical Knowledge Evaluation metrics q q q ABR certification (and CAQ if appropriate) Massachusetts medical license Resident teaching scores Category 1 CME credits Risk management credit requirements Didactic lectures to residents and fellows (target >3/year) Board review sessions to residents Presentations at national academic meeting Educational exhibits at national meetings Peer reviewed manuscripts published in current year Invited lectures 360 degree evaluation

Competency 3: Practice-based Learning Evaluation metrics q q q q Submissions to online QA reporting system MCQ questions for resident board preparations log Attend QA Rounds Mentor resident in QA elective project Publish peer reviewed paper Document use of QA tools for problem solving 360 o evaluation (and demonstrate response to feedback from trainees)

Competency 4: Interpersonal & Communication Skills Evaluation metrics q 360 o evaluation Direct observation and feedback during interdisciplinary meetings, lecture presentations, teaching conferences, patient scanning and interventional procedures. q q Complete Online Error Disclosure module (? do we have one) Customer complaints Resident and fellow evaluations (beyond teaching score alone) Compliance with online Critical Results Communication Policy (QA elective project) q Evaluation of dictated reports (future project)

Competency 5: Professionalism Evaluation metrics q q q q q Timely completion of admin support review Timely completion of academic progress form (data from Andrea Baxter) Compliance testing: HIPAA, human research etc, JCAHO preparedness = QA elective project. RTAT: median <18 hrs Ethics grand rounds (research and clinical): attend or powerpoint (QA elective project) Universal protocol compliance: NPSG powerpoint Attend annual JCAHO preparedness lecture CRICO/RMF online modules 360 degree evaluation (including procedural competency)

Competency 6: Systems-based Practice Evaluation metrics q Attendance at Departmental QA and regular Grand Rounds q Peer review cases - 5% of prior year annual case volume q Attendance at section M & M meetings q Complete compliance testing HIPAA compliance module, Safe Conduct of Human Research module, JCAHO Preparation module, ACLS q Maintain ABR certification (including PQI project) q Mentor resident in QA project

Citizenship Evaluation metrics q q q q q RTAT median <16 hrs Up to date with BIDMC compliance testing 360 o evaluation Complete HIPAA compliance module Complete Safe Conduct of Human Research module Complete JCAHO Preparation module Mentor resident in quality improvement project Participate in ABR-mandated PQI project for MOC Attend 75% (9 of 12) of monthly Departmental QA Grand Rounds

Professional Assessment • • Peer review M & M process Chart reviews Personal scorecard

• • • Huge amount of work Some data tough to extract Time consuming Impractical Requires management Reactive rather than proactive

Lean Approach • Decide on useful relevant data – Get rid of waste (muda) – Only use data that adds value • Easily accessible data – Workplace design for efficiency – Minimize frustrations obtaining data (muri) • Make process consistent – Must include mandatory data – Department specific – Standardize process (mura)

Basics • Web- based system • Department specific • Many parameters will auto populate the database (licensing, board certification, compliance tests etc) – Include mandatory metrics (dept specific) – From staff category and privileges • Departments can select criteria to be used • These criteria can be physician or division or department specific

Suggested strategy • Relevant department-specific metrics • As much relevant info as possible should auto populate database • Build in auto reminders • Tie into general competencies

Radiologist Performance Evaluation • Clinical Performance and good citizenship metrics • Academic metrics • Multi source feedback

Suggested strategy Hospital specific compliance tests Up to date with Compliance testing: Y or N HIPAA general training exam - 14 Median Report Turnaround time Bloodborne pathogens - 11 employee comprehensive education - 20 responsible conduct for human studies - 1 Procedural success/cx rate (major/minor) Rad. Review participation (>5% prior yr volume) Rad. Review scores (vs national benchmark) what physicians need to know about JCAHO 23 critical results communication compliance moderate sedation training - 21 Online QA cases (category) HMFP compliance - 22 epipen location and activation - 102 general infection control employee education 136 Requires MEC approval radiology timeout procedure - 117 radiology patient fall prevention quiz - 88 universal protocol outside of OR - 134 Department specific compliance tests

Annual Academic Planning Review ABR certification: yes or no Mass license current: Y or N category 1 CME credits in last 24 months risk management credits last 24 months resident board review sessions academic rank: Ins, Assis, Assoc, full year of last promotion presentations at national meetings educational exhibits at national meetings QA exhibits at Silverman day peer reviewed manuscripts published/in press grants submitted/awarded during past year awards and honors received invited lectures - local invited lectures - national invited lectures - international mentor QA elective resident and title of QA project resident teaching scores customer complaints/compliments leadership roles w. RVU's w. RVU percentile vs AAARAD data

Multisource Feedback adherence to department's code of conduct customer service usefulness of reports section meetings attendance Universal policy compliance Availability to staff, technologists and trainees punctuality response to requests procedural skills major strengths and compliments how much do you enjoy working with…. suggestions for improvement

Current BIDMC Status of Compliance Testing

Specific to privileges

MEC: why have we not mandated this?

Compliance with Submissions

Section personalities Variable compliance Variable grading How many cases? Where all the category 3 and 4 cases? Who is taking ownership of peer review?

Summary: comprehensive performance evaluation Clinical performance metrics ABR certification: yes or no Up to date with Compliance testing: Y or N Mass license current: Y or N category 1 CME credits in last 24 months Median Report Turnaround time risk management credits last 24 months resident board review sessions Procedural success/cx rate (major/minor) academic rank: Ins, Assis, Assoc, full Rad. Review participation (>5% prior yr volume) year of last promotion presentations at national meetings Rad. Review scores (vs national benchmark) educational exhibits at national meetings Online QA cases (category) QA exhibits at Silverman day peer reviewed manuscripts published/in press adherence to department's code of conduct customer service awards and honors received usefulness of reports invited lectures - local section meetings attendance invited lectures - national Universal policy compliance invited lectures - international Availability to staff, technologists and trainees punctuality response to requests 360 procedural skills major strengths and compliments how much do you enjoy working with…. suggestions for improvement grants submitted/awarded during past year mentor QA elective resident and title of QA project resident teaching scores customer complaints/compliments leadership roles w. RVU's w. RVU percentile vs AAARAD data Academic Teaching research

Evaluation process • More than annual evaluation by chief • Signed form in practitioners folder • How should we handle problems? – Remedial actions – Working on 360° career development component (Sloan fellowship project – Ellen Volpe)

The Ask • Ongoing MEC agenda item • Share departmental “best” processes – 360 from anesthesia etc • MEC leadership so we can get this done • ? Central process – grant availability