The Mpowerment Project Turning Research Into Practice Greg

The Mpowerment Project Turning Research Into Practice Greg Rebchook, Ph. D, Susan Kegeles, Ph. D, Scott Tebbetts, John Hamiga, Robert Williams, Lance Pollack, Ph. D, David Sweeney UCSF-Center for AIDS Prevention Studies Dave Huebner, Ph. D University of Utah, SLC

Background • The Mpowerment Project (MP) is an evidence-based HIV prevention intervention (EBI) for young gay/bisexual men • Funders encourage CBOs to implement EBIs • Little research has been conducted about how to help CBOs implement EBIs with fidelity to the original models • We developed a technology exchange system to help CBOs effectively replicate MP

Agenda Overview • Background of Mpowerment Project • Description of our TRIP Study • Who stopped implementing MP and why? • Fidelity - What is it and how to predict it? • Next steps - How to improve our MP technology exchange system

Expectations? • Are there specific issues you want us to cover? • Specific questions you want us to address? • What are your expectations from the workshop?

Introductions • Brief introduction of presenters • Audience questions • How many from CBOs implementing MP? • How many from CBOs considering MP? • How many funders? • How many capacity-building providers? • Others?

Experience? • How long have you been involved in HIV prevention? • Less than 1 year? • 1 -5 years? • 5 -10 years? • More than 10 years?

Section 2: MP Background • Kiosk Video • Guiding Principles • Core Elements and examples of adaptations

Guiding Principles • Social Focus • Empowerment • Peer Influence • Diffusion of Innovations • Multi-Level • Sex Positive & Affirming of all Orientations • Community Building

Social Focus • Need to be able to reach young gay men, many of whom are not interested in HIV/AIDS Austin Men’s Project 1999 • Address young gay and bi men’s social needs • Link HIV prevention to the fulfillment of these and other unmet needs • Infuse HIV prevention into all social activities

Empowerment • Behavior change is most lasting when we’re actively involved in creating and implementing solutions to our own problems • Young men are the decision makers of the Project • It is their own Project MPowerment Detroit 2007

Peer Influence • Peer-influence is very powerful • The Mpowerment Project mobilizes peers to act as agents of change within their social networks MPowerment Detroit 2007

Diffusion of Innovations • Young MSM talk with and encourage their friends to be safe • They spread this message throughout the community • A norm of safer sex is established and reinforced MPower - Albuquerque, NM 1997

Multi-level Young men engage in high risk sex for a variety of reasons • individual factors (e. g. , perception that safer sex is dull) • interpersonal factors (e. g. unable to negotiate safer sex w/partner) • social factors (e. g. , lack of support for safer sex) • environmental factors (e. g. , “risky” venues, no targeted programs for young gay/bi men) The Mpowerment Project operates at all these levels and addresses multiple predictors of risk

Sex Positive & Affirming of All Sexual Orientations • Enrich and strengthen young men’s sexual identity and pride about their sexual orientation • We avoid fear-based or shame inducing messages • Eroticize safer sex--not just focused on condoms • Images reflect diversity of the community • Promotional events are uplifting, fun and erotic the Mu Crew - Dayton Ohio 2006

Community Building • Creates a healthy community • Promote supportive friendship networks • Disseminate a norm of safer sex throughout a community MPowerment Detroit 2007

Mpowerment Core Elements Program Structure • Core Group & Volunteers • Coordinators & Supervision • CAB FO IO Program Components • Formal Outreach • Informal Outreach • M-Groups • Project Space • Publicity Campaign CG & V PS MG

Informal Outreach Project Space Core Group M Groups Formal Outreach

Coordinators • Maintains the vision of the project • Mobilizes Core Group & Volunteers • Focused on process • Makes sure things happen • Guardians of the project MPowerment Detroit 2007

Core Group & Volunteers • The decision making body. 10 - 20 YGM • Changes over time • Seek the diversity of your priority population • Carefully analyzes what they are doing and why MPowerment Detroit 2007 • Coordinators create the agenda for CG • Adaptation: MP Detroit added an internship program for potential CG members

Project Space • Space for most events and activities • Young gay/bi men’s community center Mpowerment Detroit 2007 FUSION Wilton Manors FL 2008 • Resources and referrals • Volunteers decorate and maintain space • Home-like, safe, gay- positive atmosphere • Adaptation: dedicated space within agency building, sharing space w/other agencies or universities FUSION - Wilton Manors Florida 2008

Outreach Model Formal Outreach • Outreach Events (Large, Medium, Small) • Outreach Team Performances • Adaptable to all communities Informal Outreach • Taught and encouraged in M-groups • 1 on 1 Conversations • Adaptation: Number of IO encounters recorded on display board MPOWER OC - Orange County, CA 2007

M-Group - Small Group • Peer-led • 8 -10 young gay/bi men • 1 time group • About 3 hours long • Promoted as a way of meeting other young gay/bi men, fun, discussion of important issues (NOT a “safer sex workshop”) • Adaptation: new role play scenarios added

Publicity Campaign • Articles/ads in gay and alternative press • Posters, fliers, brochures • Web-site (with calendar) • E-mail distribution lists • Word of mouth • Generally avoided “mainstream” press • Adaptation: More emphasis on new media and technologies (texting, facebook, myspace)

TRIP Methods • Longitudinal study of 72 CBOs implementing the Mpowerment Project • Assessed relationship between use of the MP technology exchange system (MPTES) and fidelity • 532 semi-structured telephone interviews with 1 -5 staff (N=329) at each CBO at baseline, 6, 12, and 24 month follow up intervals

Key variables measured • Agency characteristics • Community characteristic • Program characteristics • Funder characteristics • MPTES utilization

Agency Characteristics • Annual budget • # FTE • # Volunteers • Mission • ED experience • ED turn-over • PM turn-over • # Coordinators • Agency functioning • Agency efficacy • HIV prevention budget • Trust in ED • Staff turn-over • Agency history

Community Characteristics • Size: population & # of counties • Location • Gay community cohesion • Target population cohesion • # gay bars • # gay publications • # pride events

Funder Characteristics • Funding efficacy • Health department issues

MPTES Utilization • Use of materials • Use of TA • # TA contacts with coordinators • Topics of each TA episode • % of staff trained by CAPS

Implementation Variables • Length of time implementing • Whether each core element was implemented, dropped, or modified • Adherence to MP guiding principles • Adherence to MP core elements

MPTES Components • Written materials (training manuals, facilitator guides) • Videos (program overview and facilitator training) • Interactive TA (proactively delivered by former MP coordinators) • 3 day trainings • Web-based resources (e. g. , online materials, photos, listserv, chats)

Manual and videos 928 Distributed

Technical Assistance • 69 agencies received proactive technical assistance (every two weeks) • 1425 Technical Assistance Events (phone calls, emails, web-chats, mail & in person) • TA conducted by individuals who have run the intervention in CBOs

44 Trainings - 1031 Participants

Mpowerment. org

youtube. com/mpowermentprojects Mpowerment myspace. com/mpowermentprojects

TRi. P Cities 2003 - 2007

Location of Implementing CBOs Community Size < 100 k 13% 100 k – 200 k 13% 200 k – 500 k 35% 500 k – 1 m 17% 1 m – 2 m 17% > 2 m 13% Geographic Region West 30% Northeast 17% Southeast 16% Midwest 16% South 10% Southwest 9% Puerto Rico 1%

Project Priority Populations

Projects by Age Breakdown Ages % of programs Youth oriented 12/13 - 23/24 year olds 10% Teens/young men 14/16 - 21/29 year olds 19% Original project 18 - 29 year olds 31% Young men 18/19 - 30/35 year olds 17% Older men 15/25 to no upper limit 23%

Funding Sources • CDC pass-through 44 • State - 34 • Private foundations/big pharma - 25 • Private fund-raising - 18 • Direct from CDC -10 • County - 10 • City - 1 • Other federal - 5 • Don’t know - 9

Budgets for MP *Did not begin asking this question until part-way into the project

CBOs and Mpowerment Staff

Discontinued Agencies • 21 agencies stopped implementing MP • Most stopped within first year (mean number of months implementing equals 9)

Why was MP discontinued? • Smaller HIV prevention budget • Low number of Coordinators • Not following MP as described in model • Low levels of trust in ED • Low agency Efficacy • Low TA Utilization • Not having a Community Advisory Board + + +

What had no effect on discontinuation? • CBO Annual budget (#FTEs, #volunteers) • Funding source • Agency mission • Agency experience with HIV prevention • Community characteristics • Funder characteristics • Materials utilization

Discontinuation Interviews N=19 Discontinuing an intervention isn’t always a bad thing • One agency tried MP for >1 year • Budget = $4, 000 • Less than. 5 FTE Coordinator • Service area of 4, 000 square miles • Largest towns were 3 -5 k people Important to determine if an agency is really ready for the Mpowerment Project before investing in it

Readiness Important theme that came up during the interviews Not necessarily about money, number of staff, or community size: “There needs to be a serious inventory of the assets and liabilities that an agency has before moving forward with any plans to implement…[one of] the dangers of the EBIs is that it looks like a cookbook whereby you can buy the ingredients and you’re done…. but if you don’t do an honest assessment to see if you can do the physical space thing and if you can find the right people to lead it, then you’re setting yourself up for failure…assessing the agency readiness is totally critical…. does the staff know what it means to do this? Are there a lot of naysayers in the room who want to do their own thing and don’t think it will fit for them? . . . if the staff doesn’t believe in a science-based project, then it’s doomed…”

Discussion Question What do you think are some of the characteristics of an agency that is ready for the Mpowerment Project? (large or small group discussion)

Fidelity • Fidelity: the extent to which intervention is implemented using same methods as in the original model • 2 approaches to rating fidelity • Externally-rated fidelity scores: • TA specialists kept extensive notes on TA episodes, issues arising in each organization • TA specialists and evaluator read all TA notes and all Interview notes per organization • Rated fidelity score: ranged 1 - 10 (1 = no fidelity, 10 = high fidelity) • Extensive work conducted to reach shared understanding of fidelity ratings

We randomly selected 26 rating periods and found that the 3 raters were within 1 point of each other 73% of the time and within 2 points of each other 100% of the time Self-rated fidelity scores • Participants were asked if they were implementing each Core Element as described in materials, if they modified the Element, or if they dropped the Element • Numeric values assigned to each response • Overall fidelity and fidelity to each Core Element were calculated

Mean fidelity scores n=48 implementers

MPTES Utilization • We reviewed our records and asked respondents how often they used each MPTES component • Factor analysis yielded two strong, uncorrelated factors • Materials Utilization Scale (use of all replication materials) • Interactive TA Scale (all TA activities)

Both fidelity scores were correlated to MPTES utilization BASELINE 6 MONTHS 12 MONTHS Fidelityexternal rating Fidelityself report Materials. 30* Utilization . 40** . 32* . 41** 0. 16 0. 00 Interactive 0. 03 TA -0. 05 0. 11 0. 25 . 31* . 29* Fidelityself report Fidelityexternal rating *p <. 05; **p <. 01

R (external) R (self) df Annual HIV prevention budget 0. 24 . 46** 43 Annual MP budget 15 . 30+ 40 # Coordinators . 34* . 56*** 48 Organizational Functioning 0. 19 . 26+ 48 Agency efficacy to implement MP 47** . 49*** 48 Replication materials use . 32* . 39** 48 Staff turn over 0. 15 . 29* 48 Adherence to guiding principles 0. 19 . 27+ 46 Adherence to Core Elements . 48*** . 62*** 44 TA: M-groups . 34* . 31* 48 TA: Outreach events . 32* . 31* 48 TA: Outreach implementation . 39** . 36* 48 TA: Addressing safer sex during OR . 32* . 39** 48 TA: Informal OR . 34* . 39** 48 TA: Publicity . 30* . 32* 48 TA: Project Space . 25+ . 38** 48 TA: Volunteers 0. 24 . 27+ 48 TA: Coordinators . 25+ . 27+ 48 TA: Core Group . 27+ . 26+ 48 TA: Supervision . 25+ . 26+ 48 TA: Evaluation . 37** . 25* 48 TA: Community Issues . 24+ 0. 20 48 Fidelity Predictors +p<. 10 *p<. 05 **p<. 01 ***p<. 001

Agency factors and fidelity Related to fidelity • • Annual HIV prevention budget MP budget Not related to fidelity # Coordinators • • • Organizational functioning Agency efficacy to implement MP More staff turn over Adherence to guiding principles Adherence to core elements Staff’s trust in ED Size of agency Overall annual CBO budget

Implementation Examples Low fidelity implementation • • Mid-size, Southern city Well funded; >2 Coordinators Not interested in Core Elements Little belief in guiding principles No clear implementation plans Publicity not planned High fidelity implementation • • • No accountability • Administrative barriers within agency • Isolated, rural town in conservative, red state No local ASO or gay community orgs. Poorly funded; 1 PT coordinator Energetic coordinator with strong belief in MP model Created fun, social events, strong publicity targeted to community Wrote his own funding applications Many successful events (small and large)

Low Fidelity Agency “They don’t plan anything as a group…they don’t plan activities at all…she says that it’s a joke to expect they would plan anything…the curriculum is ideal but unrealistic for real life as they know it…people are busy and they are not going to go and do your job for you…so the coordinators do the events…she thinks that part of the program, the core group planning stuff, is a joke and needs to be changed…maybe in a place where there is nothing to do but pick your nose, but in XXX, it isn’t going to happen. ” - Program Coordinator

Lack of buy-in to Guiding Principles • Young men are very concerned with social and self-esteem issues • Empowerment promotes more lasting changes in behavior

High fidelity agency “The Core Group meetings start at 7, and they plan ‘til about 8: 30…then have an agenda for each meeting…they hand out pens so guys can make notes…they talk about questions, issues…talk about past events…they always start the meetings with pros and cons of the past event…if it was a dance or swimming party…and they get to voice what they felt and they can learn from what they did…and he and his co-coordinator co-facilitate but the core group members run it…right now the guys are working on scheduling issues…since they are new, they are trying to figure out what events work will and which ones don’t. ”

MPTES predictors and fidelity Related to fidelity • • Materials use TA on specific core elements TA on community issues TA on supervision and evaluation Not related to fidelity • • Total # of TA contacts Total TA contacts with coordinators Length of TA calls TA on project start-up TA on agency issues TA on funding TA on fidelity/adaptation

TA Experiences We had many excellent TA relationships with agencies • We checked in with them regularly • They could call us as necessary We were frustrated with agencies that we began calling “MINOs” Mpowerment in Name Only • Sometimes, the MINO agencies had a long history of HIV prevention, often with youth • Generally, they received MP funding because parts of the model matched what they were already doing • Little desire to change their current programming • Often little accountability from agency to funder, or from line-staff to supervisors

Other factors did not predict fidelity • Length of time implementing MP • Community level variables (e. g. , size, gay community cohesion, number of gay bars, pride events, gay publications)

Stakeholder Exercise • Divide into 4 groups: Coordinators, Supervisors, Funders, CBA/TA providers • Scenario 1: There’s 1 full-time coordinator working in a small Southern town. His other responsibilities include HIV 101 presentations and HIV testing. The coordinator has pulled together a Core Group of 4 people, one is his boyfriend another is his best friend. He doesn’t like the M-groups and he wants to change them into 3 multi-sessions that resemble what he used to do at a different agency. His supervisor think that’s a great idea. • From your group’s perspective, what could you do to make this program more successful?

Next Steps • Based on our findings, we realize that many issues affecting implementation are not under the control of the agency’s front-line staff (Coordinators). • We need a new approach that addresses entire systems and today’s funding realities • We are developing new materials (re-writing our manual, developing online audio-slideshows that address specific implementation issues) • Audioslide show demonstration if time allows • We don’t have resources for limitless TA • We have developed a matrix that has more realistic deliverables based on funding

Funding Matrix 3 FTE 1 1 1 na na na 1 1 1 2 3 na na na 2 3 4 4 na na na 3 4 4 4 na na na 4 4 FTE <25 K 25 -50 K 50 -75 K 75 -100 K 100 -125 K 125 -150 K >150 K . 25 FTE. 50 FTE. 75 FTE 1. 5 FTE 2. 5 FTE

Implementation Stages Stage-based approach to determine realistic deliverables based on funding and staffing • Stage 1: Marginal implementation • Stage 2: Partial implementation • Stage 3: Mostly full implementation • Stage 4: Full implementation Future fidelity ratings will take stages into consideration

Ideas for enhanced MPTES • How best to work with funders? • How best to work with agency EDs? • What additional materials are needed? What format? • How to help CBOs network with each other? • Other suggestions?