A Randomized Controlled Trial to Improve Wage Outcomes

A Randomized Controlled Trial to Improve Wage Outcomes of SSDI Customers of VR: Experiences of Minnesota and Kentucky VR Presentation Sept 8, 2016 Richmond, VA

Presenters • Susan Foley, Institute for Community Inclusion • Allan Lunz, Minnesota DEED-VRS • Kellie Scott, Kentucky OVR • Todd Honeycutt, Mathematica Policy Research • Joseph Marrone, ICI

What is the SGA Project? • Model Demonstration • SSDI only non-blind VR customers • Improve earnings outcomes at closure • Innovation must be within the control of the VR agency.

Funding from the Rehabilitation Services Administration Rob Groenendaal Jerry Elliott Project Officers

Partners • Mathematica Policy Research • Michael D’Alto • VR Development Group • Vendors in KY and MN providing financial specialist services

The Innovation! Early coordinated team approach (CTA) to rapidly engaging SSDI only applicants in the VR process to improve earnings at closure of VR services. • CTA includes job placement, VR counselor and financial specialist with customer Pre-IPE and then throughout VR tenure. • Number of days specified for each rehab process step.

How did we get there? • Intensive site visits with 8 state VR agencies • RSA 911 Data Analysis • Expert opinion through Delphi Panel • Document review • Literature review

Few key findings of early process of creating innovation • Agencies that had above average wage outcomes for SSDI only customers had: – Fewer counseling staff and more counseling support staff as a % of all staff – Had fewer customers on wait lists – Provided more services to customers through VR personnel and purchased fewer services in aggregate from vendors – Had fewer average days to IPE – Had fewer SSDI only customers closed because of refused services/uncooperative – Had lower average cost of purchased services for SSDI only customers who received services.

Case Study Findings • Multiple SVRAs emphasized early and progressively detailed access to financial services • Rehabilitation counseling strategies with a sense of urgency and rapid engagement. Momentum is good customer service. • A business relations focus to job development rather than supply side job placement. • Active attempts to align office level procedures and policies for responsive client services.

Recruiting the SVRAs • Identified SVRAs with sufficient numbers to host project (n = 19) • Had to eliminate those SVRAs that: – Too few number of offices – Were closed to all categories (OOS) or had lengthy weight lists – Had intense political, budgetary, union or reorganization issues – Purchased most of its services or moved to use of “virtual” employees – Some states elected to go for PROMISE grants – Made site visits to six SVRAs

SGA Project-Minnesota Allan Lunz DEED: Vocational Rehabilitation Services

Why Minnesota said “Yes” to the SGA Project • Rapid Pacing • Teaming Approach • Engagement • Additional Program Income

Preparations for the Future • Workforce Innovation Opportunities Act(WIOA) • Minnesota’s Olmstead Plan • Ongoing Relevance of the State VR Program

Keys to Successful Implementation • Top level leadership support . • Buy in at local level from RAMs and VRCs • Added resources available to SSDI clients on treatment teams

With Challenge comes Success! Immediate Feedback Value of: • Team Approach • Financial Education • Up Front Resources & Support • Pacing

Kentucky’s SGA Project Kellie Scott, Program Evaluator Kentucky Office of Vocational Rehabilitation

Why Was KY OVR Interested in the SGA Project? • Building capacity in work incentive counseling; • Fit in with recent asset development initiatives; • Leadership interested in enhanced pacing; • Possibility of increasing positive employment outcomes and SSA reimbursements; • Previous positive experience with ICI; • Progressive Leadership (David Beach was Director).

SGA PROJECT KWICs Florence-#8 Ft. Wright*-#16 Carol Leonhart Ryan Henson Brad Mills Donna Mundy – Addis Megun – Evangeline Johnson - Middletown- #15 Greg Willett Elizabethtown-#6 Owensboro*-#3 Carroll Mason Oldham Harrison Henry Franklin Shelby Bourbon Todd Simpson Barren Metcalfe Allen y Graves Trigg er Fulton Marshall m go Hickman Taylor Perry Clay Laurel Vacant Bowling Green*-#4 Missy Wheeler Leslie Knox Wayne Mc. Creary Whitley Harlan Bell Danville*-#7 Ruth Dyer Barb Pugh Pike Knott Letcher Whitesburg-#13 Russell Cumberland Monroe Clinton CDPVTC Martin Floyd Breathitt Jackson Owsley Calloway Madisonville-#2 Magoffin Casey Pulaski Lawrence t on Warren Logan Wolfe Estill Tony Perry Morgan Johnson Lee Rockcastle Green Edmonson Garrard Lincoln Adair Christian Carlisle Larue Elliott Powell Madison Boyle Menifee Marion Caldwell Lyon Mc. Cracken Grayson Butler Mercer Washington Hart Muhlenberg ton Ballard Ohio Hopkins s ng i Liv Crittenden Mc. Lean Clark Jessamine Hardin Rowan West Liberty-#10 Boyd Carter Bath Fayette Anderson Nelson Breckinridge Daviess Spencer Greenup Fleming M Henderson Bullitt Meade Hancock Webster Nicholas Scott Jefferson Lewis Robertson Woodford Union Jonathan White Owen Jim Pike Brent Sturgill Gallatin Grant. Pendleton Bracken Trimble Jason Cole Lisa Allen l be mp l n Kento Boone Ashland*-#12 Lexington*-#9 Ca Jenny Lampton Louisville* -#5 Paducah-#1 Bluegrass-#14 Viva Anderson

What were the barriers? • • • Lack of local staff buy-in; Bureaucratic inertia; Changes in staff and administration; Lack of work incentive capacity; Distractions for leadership (WIOA implementation, budget issues, etc); • One more thing for VRC’s to “get their head around (e. g. large caseloads); • Lack of infrastructure (should have spent 1 st year updating PC’s, online assessments, DB 101). 20/20 hindsight

Successes • Leadership and staff see value in participation of KWIC in team, early involvement of job placement and enhanced pacing; • Quick and early identification of SSA recipients; • Able to build capacity in several areas; specifically Job Placement; • Preliminary indications of increased employment rate among SSDI recipients

SGA Project Evaluation Todd Honeycutt Mathematica Policy Research Center for Studying Disability Policy thoneycutt@mathematica-mpr. com

SGA Project Evaluation Design • Examine KY & MN projects separately, based on state’s own implementation model • Implementation evaluation – To what extent did offices implement their assigned conditions with fidelity to their design? – What are the key lessons learned from implementation experiences? • Outcome evaluation – What impact did the innovations have on VR service delivery and client employment outcomes?

Preliminary Evaluation • Assess processes and outcomes for SSDIonly clients who applied through January 2016 (KY – 521 clients; MN – 645 clients) • Data include – RSA-911 data – Site visits – Interviews conducted with staff involved in the demonstration – Agency administrative data • Preliminary results shown

Kentucky Qualitative Themes • SGA Project model represented a substantial shift in priorities and operating procedures – Challenging for counselors to change practice patterns related to presumptive eligibility • Counselors enjoyed increasing the pace and expressed a positive attitude toward the team approach • Difficulty recruiting enough financial counselors • Difficulty with follow-up team meetings

Kentucky Enhanced Services Relative to Usual Services • Higher eligibility rates (98% vs. 94%) • Shorter median time to eligibility (3 days vs. 30 days) • Higher proportions obtaining IPEs (71% vs. 60%) • Shorter median time to obtaining IPEs (42 days vs. 77 days) • At six months after application, no difference in the proportion receiving services (50% vs. 55%) – But more likely to have exited with employment (17% vs. 8%)

Minnesota Qualitative Themes • Adopting the new models was initially a challenge • Increased pace of service could be adopted more broadly with the right supports • Greater access to financial and benefit information enhanced the VR experience • Perceived as less effective and harder to implement in sites serving large geographic and mostly rural areas

Minnesota Implementation Fidelity • Enhanced services group typically did not achieve program milestones – 78 percent received IPE, but only 37 percent received IPE within 30 days – 14 percent received a benefits analysis, with almost all occurring within 8 weeks – Over half received a team meeting, and only 23 percent of meetings happened within 7 days • Implementation varied across offices

Minnesota Enhanced Services Relative to Usual Services • Shorter median time to eligibility (3 days vs. 7 days) • Higher proportions obtained IPEs (78% vs. 68%) • Shorter median time to obtaining IPEs (32 days vs. 46 days) • At six months after application, less likely to be still receiving services (55% vs. 64%) and more likely to have exited with employment (16% vs. 10%)

Evaluation Next Steps • Release preliminary evaluation reports (Fall 2016) • Conduct full evaluation for all enrollees, including early earnings impacts (Fall 2017)

LESSONS LEARNED • LEADERSHIP SUPPORT AS SGA AS AN “IMPORTANT” PIECE FOR VR AGENCY QI, NOT JUST AN EXTRANEOUS PROJECT • PACING DID NOT SCARE AWAY CLIENTS NOR DISRUPT THE AGENCY • EARLY INVOLVEMENT OF FINANCIAL SPECIALISTS AND EMPLOYMENT STAFF VERY HELPFUL – ESPECIALLY THE FINANCIAL DISCUSSION (NOT JUST BENEFITS COUNSELING) • LISTEN TO CLIENTS ON HOW THE TEAM SHOULD BE ROLLED OUT- THEIR TIME IMPORTANT TOO

LESSONS LEARNED • STAYING FLEXIBLE WHILE ADHERING TO CORE INTERVENTION DESIGN • ICI TECHNICAL ASSISTANCE USEFUL BUT LOCAL TA (E. G. , MN DSHS) JUST AS IMPORTANT • ISSUES THAT DID OCCUR OFTEN TRACED BACK TO BROADER MGMT QUESTIONS MORE THAN THE SGA PROJECT DESIGN OR IMPLEMENTATION ITSELF • MAKE IT FUN, NOT JUST A CHORE