Скачать презентацию S-START Evaluation Team Evaluator Nancy Amodei Ph Скачать презентацию S-START Evaluation Team Evaluator Nancy Amodei Ph

c7a729ba30c2b08f4f56a7cfe30b2a61.ppt

  • Количество слайдов: 63

S-START Evaluation Team • Evaluator: Nancy Amodei, Ph. D. – Dept Pediatrics • Evaluation S-START Evaluation Team • Evaluator: Nancy Amodei, Ph. D. – Dept Pediatrics • Evaluation Coordinator: Danielle Dunlap, M. S. – Dept Pediatrics • Data Manager: Kyle Kozlovsky, M. S. – Dept Pediatrics • Qualitative Expert: Suyen Schneegans, M. A. – Dept Pediatrics Special thanks to: Rasheem Battle Alejandro Bocanegra Meghan Crabtree Merced Doria Destiny Ramos Drew Russell ®

Process Evaluation Process Evaluation

Process Evaluation: How is the program being implemented? S-START Process Goals 1. Train UTHSCSA Process Evaluation: How is the program being implemented? S-START Process Goals 1. Train UTHSCSA medical residents and residents from other participating South Texas programs to use evidence based SBIRT procedures for patients who have or are at risk of substance abuse disorders. 2. Promote systems change in targeted residency programs by integrating the SBIRT model into the curriculum on a long-term basis.

Process Goal #1: Train UTHSCSA medical residents and residents from other participating South Texas Process Goal #1: Train UTHSCSA medical residents and residents from other participating South Texas programs Key Activities and Measures 1) Develop/implement a comprehensive curriculum 2) Train UTHSCSA and other Faculty Demographics, type of training, satisfaction (GRPA & qualitative findings) 3)Train UTHSCSA residents & residents from other programs Demographics, type of training, satisfaction (GPRA)

SBIRT Curricular Strategies by Specialty Large Group Didactic Lecture Small Group Discussion Skill-Building Workshops SBIRT Curricular Strategies by Specialty Large Group Didactic Lecture Small Group Discussion Skill-Building Workshops Reading Assignments Screening Questions In Medical Records Reminder Pocket Cards Screening Assignments* Inpatient Clinical Supervision Outpatient Clinical Supervision Independent Study Module (Blackboard) OSCE (Observed Standardized Clinical Exams) Pediatrics Psychiatry + OBGYN + + + Internal Medicine + IM - ERAHC + FCM + FM – Mc. Allen + + FM -Fort FM -Santa Surgery Hood Rosa + + + (likely but not in place yet) + + + + + + + + + + + + + + (child psych only) + + + + +

What is the core SBIRT event? Department Cohort Training type # Events Pediatrics 1 What is the core SBIRT event? Department Cohort Training type # Events Pediatrics 1 -3 Small group discussion, reading assignments Family and Community Medicine 1 -4 2 -hour didactic 4 Three 1½ -hour didactics 1 Internal Medicine (Cohort 1) 1 3* Internal Medicine (Cohorts 2 & 3) 2 -3 4 hours total-Two 1 -hour didactics, 1 -hour SP, & 1 -hour SP video review 2* OB/GYN 1 -2 1 -hour didactic 2 Psychiatry (Cohort 1) 1 Two 2 -hour didactics 1 Psychiatry (Cohort 2) 2 1 -hour didactic 1 Surgery 1 1 -hour didactic 1 Nurse practitioners 1 3 -hour didactic 1 Mc. Allen Family Medicine 1 -3 2 -hour didactic 3 Santa Rosa Family Medicine 1 -2 1 -hour didactic 2 Fort Hood Family Medicine 1 -2 1 -hour didactic 2 ERAHC Internal Medicine 1 -2 -hour didactic 2

 Faculty Training Who? How Many? Method/Approach Pediatrics, UTHSCSA 8 Train-the-trainer Family & Community Faculty Training Who? How Many? Method/Approach Pediatrics, UTHSCSA 8 Train-the-trainer Family & Community Medicine, UTHSCSA 9 Train-the-trainer Family Medicine, CHRISTUS Santa Rosa 9 Train-the-trainer Family Medicine, Fort Hood 5 Train-the-trainer Internal Medicine, UTHSCSA 1 Chief Residency Immersion Training (CRIT) Internal Medicine, ERAHC 1 Train-the-trainer; Chief Residency Immersion Training (CRIT) Other UTHSCSA Departments and external Departments 18 Personal consultation, Sharing resources via website, Newsletter, Email

Demographics of Faculty Completing GPRAs Demographics or Respondents at Baseline UTHSCSA Family and Community Demographics of Faculty Completing GPRAs Demographics or Respondents at Baseline UTHSCSA Family and Community Medicine Pediatrics N = 17 (%) 8 (47%) 9 (53%) Male 9(52. 9%) Hispanic/Latino 7(41. 2%) White 12 (70. 6%) Asian 3(17. 6%) African-American 0(0%)

 Baseline vs. 30 -day Faculty GPRA ratings (N=17) GPRA Item Mean 30 Baseline Baseline vs. 30 -day Faculty GPRA ratings (N=17) GPRA Item Mean 30 Baseline -Day Rating Z * P (2 - tailed) Overall how satisfied with your training experience? 2. 12 1. 93 -. 53 NS Material presented useful to me in dealing with substance abuse 1. 82 1. 94 -. 51 NS Training enhanced my skills in topic area 2. 00 1. 94 . 00 NS 1. 76 1. 88 -. 63 NS * Wilcoxen Signed Ranks Test Training relevant to my career * Wilcoxon Signed Ranks Test

Qualitative Study of Faculty Perceptions of S-START Purpose: Gain an in-depth understanding of the Qualitative Study of Faculty Perceptions of S-START Purpose: Gain an in-depth understanding of the experience and perceptions of S-START faculty Methods 16 training faculty from 5 specialties invited 15 accepted (12 from UTHSCSA; 1 FM program in Mc. Allen, 1 FM CHRISTUS Santa Rosa, 1 FM from Fort Hood) Mean age (43. 93 years); 72% female; 73% MDs, 1 Ph. D. , 1 Psy. D, 1 M. A. Average yrs of experience = 15

Qualitative Study of Faculty Perceptions of S-START Data Collection Data collection: ≈ 22 months Qualitative Study of Faculty Perceptions of S-START Data Collection Data collection: ≈ 22 months after S-START began 45 to 60 minute interviews using scripted but open-ended questions 14 of the interviews taped to facilitate transcription Topics: How S-START implemented in the program, barriers and challenges, impact of potential clinical service reimbursement in facilitating program; suggestions for improvement

Qualitative Study of Faculty Perceptions of S-START Data Analysis Evaluation team hand-coded transcribed interviews Qualitative Study of Faculty Perceptions of S-START Data Analysis Evaluation team hand-coded transcribed interviews Thematically coded them to correspond to each question Collapsed materials thematically into 10 emergent or preset categories

Qualitative Study of Faculty Perceptions of S-START Results 3 Thematic categories accounted for > Qualitative Study of Faculty Perceptions of S-START Results 3 Thematic categories accounted for > 50% of interview responses Critical components Barriers Motivation Critical Components Faculty training Barriers Lack of Leadership Motivation Buy-in from faculty and residents

 Resident Training Department Trained to date Expected trained by Total Expected Year 05 Resident Training Department Trained to date Expected trained by Total Expected Year 05 Trained by Year 05 Old estimates Pediatrics - UTHSCSA 80 107 112 Family & Community Medicine • UTHSCSA • Mc. Allen (South Texas) • CHRISTUS Santa Rosa • Fort Hood (Military) 71 29 25 17 84 24 --- 95 41 40 31 Internal Medicine • UTHSCSA • ERAHC (South Texas) 113 16 163 -- 163 26 OB-GYN -UTHSCSA 29 41 41 Psychiatry - UTHSCSA 71 120 107 Surgery- UTHSCSA 58 --- 96 Nurse Practitioners-UTHSCSA 39* --- 39 Total 548 539 679

 Demographics of Residents Completing GPRAs Demographics or Respondents at Baseline N = 409 Demographics of Residents Completing GPRAs Demographics or Respondents at Baseline N = 409 (%) Family Medicine Pediatrics Internal Medicine OB/GYN Psychiatry 138 (33. 7%) 80 (19. 6%) 121 (29. 6%) 27 (6. 6%) 43 (10. 5%) Male 155(37. 9%) Hispanic/Latino 133(33. 8%) White 259 (63. 3%) Asian 81(19. 8%) African-American 17(4. 2%)

 Baseline vs. 30 -day Resident GPRA ratings (N=409) Item Mean 30 Baseline -day Baseline vs. 30 -day Resident GPRA ratings (N=409) Item Mean 30 Baseline -day Rating Z * P (2 - tailed) Overall how satisfied with your training experience? 1. 75 1. 71 -4. 11 . 00 Material presented useful to me in dealing with substance abuse 1. 68 1. 64 -3. 98 . 00 Training enhanced my skills in topic area 1. 75 1. 67 -3. 09 . 01 Training relevant to my career 1. 59 1. 50 -2. 92 . 01 * Wilcoxon Signed Ranks Test

Process Goal #2: Promote Systems Change in Targeted Residency Programs by integrating SBIRT model Process Goal #2: Promote Systems Change in Targeted Residency Programs by integrating SBIRT model into curriculum on long-term basis Key Activities Council of Residency SBIRT Trainers Meetings Elicit support of key personnel Changes to Electronic Medical Record Pocket Cards SBIRT resources (including key modules) on the S-START website i. Pad Project

Progress towards Goal 2: Council of Residency SBIRT Trainers Meetings Date Pediatrics BAMC FCM Progress towards Goal 2: Council of Residency SBIRT Trainers Meetings Date Pediatrics BAMC FCM Internal Medicine Psychiatry OB-Gyn 1. 29. 09 2 1 1 - 2 1 1. 30. 09 2 - 2 2 - 9. 29. 09 4 2 12. 08. 09 5 1 2 3. 09. 10 6 1 8. 17. 10 4 2. 15. 11 FM – Fort Hood FM – Santa Rosa Trauma Nursing - - 1 - - - 2 - - - 3 - 1 1 1 - - - 1 - 1 2 2 2 1 1 - - 5 - 2 1 - - 1 - 6. 28. 11 2 2 - - - 1 - - 7. 19. 11 5 - 2 1 1 - - - 3 -

Process Goal #2: Promote Systems Change in Targeted Residency Programs by integrating SPIRT model Process Goal #2: Promote Systems Change in Targeted Residency Programs by integrating SPIRT model into curriculum on long-term basis Progress re other Activities Support of change leaders- e. g. UTHSCSA President, Residency Program Directors; PD and Co-PD have high profile positions Changes to Electronic Medical Record- UTHSCSA – DFCM, Peds; (Psychiatry and Surgery planned) Pocket cards Mc. Allen FM: Part of every patient visit paperwork SBIRT resources (including core modules, resource directory) on the S-START website i. Pad Project-proposed for UTHSCSA Pediatrics

Outcome Evaluation Outcome Evaluation

What is the program’s impact? S-START Outcome Goals: 1. Enhance residents’ knowledge of evidence-based What is the program’s impact? S-START Outcome Goals: 1. Enhance residents’ knowledge of evidence-based SBIRT practices. 2. Enhance residents’ readiness and perceived confidence to implement SBIRT with their patients 3. Increase residents’ implementation of SBIRT practices with their patients 4. Enhance Faculty Participants’ knowledge and confidence in ability to teach SBIRT practices to future physicians

Outcome Design 3 x 2 Repeated Measures Three data collection methods Measurement Occasions for Outcome Design 3 x 2 Repeated Measures Three data collection methods Measurement Occasions for Surveys: Pre-Test 30 -Day Follow-Up Annually up to 36 -month follow-up Measurement Occasions for Pocket Cards Varies by department Measurement Occasions for Chart Reviews 12 -month period prior to first core SBIRT module implementation 12 -month period following the first year of SBIRT module implementation 12 -month period following the third year of SBIRT module implementation

Evaluation Measures DOMAIN Measure Description Source Knowledge SBIRT knowledge every resident should know Local Evaluation Measures DOMAIN Measure Description Source Knowledge SBIRT knowledge every resident should know Local Core Residency-specific Department-specific SBIRT knowledge Attitudes Readiness to screen patients for use, assess readiness to change, perform intervention/referral, & documentation Confidence to use SBIRT Current practice Readiness to use SBIRT Alcohol Education Survey (D’Onofrio et al. , 2002) Confidence to screen patients for use, assess readiness to change, perform intervention/referral, & documentation Self-reported current use of SBIRT Self-reported current practice of screening patients for use, assessing readiness to change, performing intervention/referral, & documentation Alcohol Education Survey (D’Onofrio et al. , 2002) Pocket cards Family Medicine programs documenting SBIRT behavior w/patients ASSIST (WHO, 2002) Chart review Review of charts in Ped. & FCM inpatient clinics Local to see change in use & documentation of SBIRT

Timeline of Self-Administered Instruments & Incentives Tool Pre-test 30 -Day F/U X GPRA Baseline Timeline of Self-Administered Instruments & Incentives Tool Pre-test 30 -Day F/U X GPRA Baseline 12 - mos F/U 24 - mos F/U 36 –mos F/U X Alcohol Education Survey X X X Brief Substance Abuse Attitude Survey X X X Core knowledge X X X Residency-specific knowledge X X X $20 $10 $20 Incentive

Methods of Survey Data Collection Web-based surveys (i. e. , Survey. Monkey) Emails to Methods of Survey Data Collection Web-based surveys (i. e. , Survey. Monkey) Emails to UTHSCSA and private email addresses Unique web links provided to residency coordinators Hard copy surveys Pass out at grand rounds and conference periods Intra-office mail for fellows, faculty Mail to home and clinic physical address

Strategies for survey follow-up Collected contact information using a comprehensive tracking form Text reminders Strategies for survey follow-up Collected contact information using a comprehensive tracking form Text reminders to cell phone numbers Phone calls to (1) cell, (2) home, (3) significant others, (4) clinic Contact residency coordinators for updated contact information Enlist authoritative support of faculty Look up information using White Pages, AMA Doctor. Finder, respective state medical board websites (usually Texas) Peer-to-peer contact updates

Future strategies for survey follow-up Reminder postcards sent twice before each annual survey Bring Future strategies for survey follow-up Reminder postcards sent twice before each annual survey Bring surveys to end-of-year gatherings for graduating residents Include surveys in residents’ exit processing before graduation I pity the fool who doesn’t take the survey.

Resident survey rates Pre-test** Total residents & students Baseline** 30 -day 12 -month 24 Resident survey rates Pre-test** Total residents & students Baseline** 30 -day 12 -month 24 -month 153. 4% 128. 7% 89. 9% 56. 6% 67. 0% Response rates for similar populations (e. g. , students, medical professionals) tend be 60% or lower on follow-up surveys (Asch et al. , 1997; Kaplowitz et al. , 2004; Kaspryzyk et al. , 2001; Mc. Mahon et al. , 2003; Porter & Whitcomb, 2007)

Analyses of resident survey data Demographic data (pre-test) Measured changes from pre-test to 12 Analyses of resident survey data Demographic data (pre-test) Measured changes from pre-test to 12 -month followup in: Confidence to use SBIRT (residents only) Readiness to use SBIRT (residents only) Current SBIRT practice (residents only) SBIRT knowledge (residents & faculty) Confidence to teach SBIRT (faculty only) Selected departments for analysis: Pediatrics, Family and Community Medicine, Internal Medicine

Resident Demographics Pre-test results Total N=465 UT Ped n= 97 UT OB n = Resident Demographics Pre-test results Total N=465 UT Ped n= 97 UT OB n = 29 UT Psy n = 83 UT Sur n = 49 a FM n = 137 IM n = 130 38. 3% 20. 6% 10. 7% 38. 6% - - - 44. 9% 51. 6% White 44. 1% 55. 2% 67. 9% 61. 3% - - - 32. 6% 31. 5% Black 3. 7% 2. 1% 3. 6% 7. 5% - - - 1. 5% 4. 8% Hispanic 31. 1% 28. 1% 21. 4% 18. 8% - - - 35. 6% 38. 7% Asian 17. 9% 12. 4% 7. 1% 10. 0% - - - 27. 4% 19. 4% Other 3. 2% 2. 1% - - - 2. 5% - - - 3. 0% 5. 6% 30. 3(4. 8) 28. 2(2. 6) 28. 8(2. 4) 31. 4(5. 9) - - - 32. 5(5. 7) 29. 1(3. 6) Gender (freq. male) Race (freq. ) Age (mean; sd) Note. UT=University of Texas Health Science Center at San Antonio; Ped. =Pediatrics; FM=Family Medicine; IM=Internal Medicine; OB=Obstetrics/Gynecology; Psy. =Psychiatry (adult & child); Sur=Surgery. a. Surgery residents began the SBIRT curriculum on August 15, 2011.

Resident Demographics (cont. ) Pre-test results UT FCM n = 72 UT Mc. A Resident Demographics (cont. ) Pre-test results UT FCM n = 72 UT Mc. A n = 23 SR n = 24 FH n = 18 UT IM n = 109 UT ERAHC n = 21 40. 0% 47. 8% 33. 3% 90. 9% 48. 6% 66. 7% White 22. 5% 9. 1% 54. 2% 72. 2% 35. 0% 14. 3% Black 2. 8% 0% 0% 0% 1. 9% 19. 0% Hispanic 28. 2% 77. 3% 45. 8% 0% 36. 9% 47. 6% Asian 42. 3% 13. 6% 0% 22. 2% 19. 4% 19. 0% Other 4. 2% 0% 0% 5. 6% 6. 8% 0% 33. 5(6. 2) 35. 3(4. 6) 28. 2(2. 4) 30. 5(4. 4) 28. 8(3. 5) 30. 8(3. 9) Gender (freq. male) Race (freq. ) Age (mean; sd) Note. UT=University of Texas Health Science Center at San Antonio; FCM=Family and Community Medicine; Mc. A. =Mc. Allen Family Medicine; SR=CHRISTUS Santa Rosa Family Medicine; FH=Fort Hood Family Medicine; IM=Internal Medicine; ERAHC=Edinburgh Regional Academic Health Center Internal Medicine.

Outcome goal 1 Enhance residents’ knowledge of evidence-based SBIRT practices. Core SBIRT knowledge 12 Outcome goal 1 Enhance residents’ knowledge of evidence-based SBIRT practices. Core SBIRT knowledge 12 items developed locally by the SBIRT project directors Knowledge that residents across all departments should know after training Residency-specific SBIRT knowledge 7 -17 items developed locally by the SBIRT faculty in the respective programs Items designed for specific residency program SBIRT knowledge and patient populations

Outcome goal 1 Enhance residents’ knowledge of evidence-based SBIRT practices. Sample core knowledge item: Outcome goal 1 Enhance residents’ knowledge of evidence-based SBIRT practices. Sample core knowledge item: “How many ‘standard drinks’ are considered at-risk alcohol use by a healthy 40 -year-old man? ” Sample Pediatrics knowledge item: “____ exposure is the leading known preventable cause of mental retardation. ” Sample Family and Community Medicine knowledge item: “Hepatitis B, hepatitis C, HIV and AIDS are strongly associated with abuse of…” Sample Internal Medicine item: “Alcohol withdrawal treatment on the inpatient medical service is best accomplished by…”

Outcome goal 1 cont. Enhance residents’ knowledge of evidence-based SBIRT practices. Core SBIRT knowledge Outcome goal 1 cont. Enhance residents’ knowledge of evidence-based SBIRT practices. Core SBIRT knowledge All residents increased SBIRT knowledge, F(1, 167) = 32. 1, p <. 001. No differences found between residency programs

Outcome goal 1 cont. Enhance residents’ knowledge of evidence-based SBIRT practices. Residency-specific SBIRT knowledge Outcome goal 1 cont. Enhance residents’ knowledge of evidence-based SBIRT practices. Residency-specific SBIRT knowledge Pediatrics increased FCM maintained IM maintained SBIRT knowledge, F(1, 61) =. 20, 59) = 4. 53, p =. 038. 46) = 2. 2, p =. 149. p =. 659.

Outcome goal 2 Enhance residents’ readiness & perceived confidence to implement SBIRT with their Outcome goal 2 Enhance residents’ readiness & perceived confidence to implement SBIRT with their patients. Readiness to use SBIRT (D’Onofrio et al. , 2002) Subscale of AES comprised of 7 10 -point Likert scale items Range: 0 -100 Sample item: “How ready are you to change your practice behavior to ask patients about quantity and frequency of their alcohol use? ” Confidence to use SBIRT (D’Onofrio et al. , 2002) Subscale of AES comprised of 7 5 -point Likert scale items Range: 0 -100 Sample item: “I am confident in my ability to discuss/advise patients to change their drinking behavior. ”

Outcome goal 2 cont. Enhance residents’ readiness & perceived confidence to implement SBIRT with Outcome goal 2 cont. Enhance residents’ readiness & perceived confidence to implement SBIRT with their patients. Readiness to use SBIRT: No significant change in readiness from pre-training to 12 months post-training, F(1, 161) =. 87, p =. 353. FCM reported higher readiness than IM overall, F(2, 161) = 4. 7, p =. . 010. Pediatrics was not significantly different than the other two programs

Outcome goal 2 cont. Enhance residents’ readiness & perceived confidence to implement SBIRT with Outcome goal 2 cont. Enhance residents’ readiness & perceived confidence to implement SBIRT with their patients. Confidence to use SBIRT: Residents overall reported higher confidence at 12 month, F(1, 161) = 27. 3, p <. 001. FCM reported higher confidence overall than IM and Pediatrics, F(2, 161) = 8. 1, p < . 001. Pediatrics was not significantly different than the other programs

Outcome goal 3 Increase residents’ implementation of SBIRT practices with their patients. Self-report of Outcome goal 3 Increase residents’ implementation of SBIRT practices with their patients. Self-report of current SBIRT practice (D’Onofrio et al. , 2002) Subscale of AES comprised of 7 5 -point Likert scale items Range: 0 -100 Sample item: “How often do you formally screen patients for alcohol problems using brief screening tools (e. g. , T-ACE, AUDIT, CAGE)? ” Pocket cards Chart reviews

Outcome goal 3 cont. Increase residents’ implementation of SBIRT practices with their patients. Current Outcome goal 3 cont. Increase residents’ implementation of SBIRT practices with their patients. Current practice of SBIRT skills: Residents overall reported higher current SBIRT practice at 12 -month, F(1, 161) = 35. 2, p <. 001. Significant interaction, F(2, 161) = 19. 7, p <. 001. Both Pediatrics and FCM improved self-reported current practice. Internal Medicine declined in self-reported current practice.

Summary of Resident Survey Data Findings SBIRT core knowledge improved from pre-test to 12 Summary of Resident Survey Data Findings SBIRT core knowledge improved from pre-test to 12 month follow-up Readiness to implement SBIRT did not change, but was high at pre-test Confidence to use SBIRT improved from pre-test to 12 -month follow-up For self-report of SBIRT practice, residents overall improved from pre-test to follow-up However, when departments were analyzed separately, Internal Medicine decreased from pre-test to 12 -month

Outcome goal 3 cont. UTHSCSA Family Medicine Pocket Cards Settings: Family Medicine inpatient service Outcome goal 3 cont. UTHSCSA Family Medicine Pocket Cards Settings: Family Medicine inpatient service at University Hospital in San Antonio, Texas Subjects: 285 adult patients, from July 2009 to May 2011. Average Age: 47 Gender Distribution: 71. 3% Male

UTHSCSA Family Medicine Procedures Patients were interviewed with a 4 -step pocket card Step UTHSCSA Family Medicine Procedures Patients were interviewed with a 4 -step pocket card Step 1: Pre-screening questions for substance use Step 2: WHO ASSIST (Alcohol, Smoking, and Substance Involvement Screening Test) Step 3: ASSIST score to assess the level of risk and determine need for intervention Step 4: checklist describing the intervention, patient response, and future plan. Residents were asked to complete 12 per year 26 out of 26 trained residents participated Residents completed 11 total on average

Step 1: Pre-screening Results Step 1: Pre-screening Results

Step 2 -3: ASSIST Results 95. 8% of patients screened positive for at least Step 2 -3: ASSIST Results 95. 8% of patients screened positive for at least 1 substance Avg. ASSIST Score was 19 indicating a moderate risk of substance abuse

Brief Interventions When the ASSIST Score recommended a brief intervention, residents reported some form Brief Interventions When the ASSIST Score recommended a brief intervention, residents reported some form of brief intervention 69. 4%(over two thirds ) of the time Residents most likely to discuss consequences of use if ASSIST Score recommended brief intervention (79% of the time) 8% of patients declined to discuss their response to screening

Brief Intervention Actions Taken Brief Intervention Actions Taken

Referrals to Treatment When the ASSIST Score suggests a referral to treatment, residents referred Referrals to Treatment When the ASSIST Score suggests a referral to treatment, residents referred a patient to treatment 71. 8% of the time Residents were most likely to contact an LCDC (Licensed chemical dependency counselor) when ASSIST Score recommended a referral to treatment (46. 5% of the time) Referrals to Tx

Referrals to Treatment Actions Taken Referrals to Treatment Actions Taken

Step 4: Patient Plans Patients were more likely to report some sort of action Step 4: Patient Plans Patients were more likely to report some sort of action (cut back, quit or seek outside help) when the resident documented a brief intervention (80% of the time compared to 71. 4%) Patients were more likely to report some sort of action when the resident documented a referral to treatment (80% of the time compared to 68. 8%)

Participation in Other Departments The ASSIST Pocket Card was distributed to other departments with Participation in Other Departments The ASSIST Pocket Card was distributed to other departments with some participation Residency # of Cards Completed # of Residents Participating # of Residents Trained UTHSCSA Mc. Allen Family Medicine 103 13 29 Fort Hood Family Medicine 25 6 11

Outcome goal 3 cont. Overall chart review study design Location 12 -month period prior Outcome goal 3 cont. Overall chart review study design Location 12 -month period prior to first core SBIRT module implementation 12 -month period following the first year of SBIRT module implementation 12 -month period following the third year of SBIRT module implementation Pediatrics May 4, 2008 May 4, 2009 May 4, 2010 May 4, 2011 May 4, 2012 May 4, 2013 Family & Community Medicine April 22, 2008 April 22, 2009 April 22, 2010 April 22, 2011 April 22, 2012 April 22, 2013

Goal 3 cont. Overall Chart Review Study Design 1. 1. 1 st Chart Review* Goal 3 cont. Overall Chart Review Study Design 1. 1. 1 st Chart Review* Pediatrics outpatient continuity clinic Family & Community Medicine outpatient continuity clinic SUBTOTAL 2 nd Chart Review^ Pediatrics outpatient continuity clinic Family & Community Medicine outpatient continuity clinic SUBTOTAL 3 rd Chart Review Pediatrics outpatient continuity clinic Family & Community Medicine outpatient continuity clinic SUBTOTAL *Completed ^In Progress # of cases for sample 400 400 800 # of cases for sample 400 800 2400

Pediatrics Chart Review Baseline findings Setting: Pediatrics Outpatient Continuity Clinic Chart selection criteria: (1) Pediatrics Chart Review Baseline findings Setting: Pediatrics Outpatient Continuity Clinic Chart selection criteria: (1) seen at least once within the review period by a resident (2) at least one visit within the appropriate review period (e. g. , one year prior to implementation of core SBIRT intervention and medical record changes, one year after, and three years after) (3) age of patient 11 and up Visits included in the analysis are acute visits and annual well child visits

Patient Demographics Mean age was 13 (1. 98) 49. 5% Female Ethnicity known to Patient Demographics Mean age was 13 (1. 98) 49. 5% Female Ethnicity known to be majority Hispanic, but documentation in charts is rare

Screening Out of 967 visits, 511 (just over half) screenings were documented Visits included Screening Out of 967 visits, 511 (just over half) screenings were documented Visits included Annual Check ups and Acute Visits Some acute visits were sports physicals

Screening Cont. 9 positive screenings documented for tobacco 0 positive screenings documented for alcohol Screening Cont. 9 positive screenings documented for tobacco 0 positive screenings documented for alcohol or other drugs The HEADSS was documented as a screening tool in 149 visits Only one CRAFFT screening was documented

Brief Interventions and Referrals to Treatment Of 967 visits, 204 brief interventions were documented Brief Interventions and Referrals to Treatment Of 967 visits, 204 brief interventions were documented 4 of 9 positive tobacco screenings were followed by a documented brief intervention “Anticipatory guidance” for drug use is often used with pediatric patients and was considered a BI in our design Only 2 referrals to treatment were documented

Conclusions from Pediatric Chart Review Documentation is a likely contributor to the lack of Conclusions from Pediatric Chart Review Documentation is a likely contributor to the lack of SBIRT practices found Ongoing changes to the medical record and routine SBIRT training will likely increase SBIRT practices An increase in screening might lead to an increase in positive screenings (and, in turn, increase opportunities for brief interventions and referrals to treatment)

Outcome goal 4 Enhance faculty participants’ SBIRT knowledge and confidence in ability to teach Outcome goal 4 Enhance faculty participants’ SBIRT knowledge and confidence in ability to teach SBIRT practices to future physicians. SBIRT knowledge Developed locally by the S-START project directors Same core knowledge that the residents receive Confidence to teach SBIRT to residents Adaptation to the resident scale (D’Onofrio et al. , 2002) Range: 0 -100 Sample item: “I am confident in my ability to train residents in advising patients to change drinking behavior. “

Outcome goal 4 cont. Enhance faculty participants’ SBIRT knowledge and confidence in ability to Outcome goal 4 cont. Enhance faculty participants’ SBIRT knowledge and confidence in ability to teach SBIRT practices to future physicians. Faculty significantly increased their confidence increased their SBIRT at 12 -month (Z =- -2. 76, p knowledge scores at 12 month (Z = -2. 12, p =. 034) =. 006)

Questions? Questions?