Encouraging Fidelity Ontario s Assessor Training and Data Quality

Encouraging Fidelity: Ontario’s Assessor Training and Data Quality Assurance Program June 9, 2011 Monica Neitzert Ministry of Community and Social Services Government of Ontario (Canada)

Ontario facts p Ontario is Canada’s most populous province and the second largest in total area (about 350, 000 sq mi) p Total population according to the 2006 census stands at about 12 million, of whom 1% are estimated to have developmental disabilities p Ontario includes the biggest city in Canada, Toronto, as well as the nation’s capital, Ottawa p The Province provides income support for about 50, 000 adults with developmental disabilities; about 15, 000 also receive residential and community participation support and live in small group home settings p A range of support options are funded, but waiting lists are increasing p Developmental services are delivered through 380 not-for-profit community service agencies, ranging from small to medium-sized enterprises, with about 18, 500 full-time equivalent direct support staff p The last three government-operated institutions for adults with developmental disability were closed by March 2009 Ministry of Community and Social Services, Ontario 2

Developmental services transformation in Ontario p In 2004 the Government of Ontario announced that the province would be transforming supports for adults with developmental disabilities to create an accessible, fair, and sustainable system of communitybased supports p The government has worked with stakeholders to create a plan that will support adults with a developmental disability to live as independently as possible in the community and promote their full inclusion in all aspects of society p New legislation was passed in 2008 to facilitate the transformation of the developmental services system, replacing an Act that was put in place over 35 years ago Ministry of Community and Social Services, Ontario 3

Developmental services transformation (cont’d) Moving From: To: Different ways of accessing services Single points of access Inconsistent eligibility decisions Clear eligibility criteria defined in legislation Inconsistent assessment of individual’s support needs A standard provincial assessment method Inconsistent funding for people with similar support needs Needs-based funding allocation Limited choice for individualized supports More options for direct funding or partnering with agencies to create individualized supports Fragmented services and supports Person-directed planning for better coordination and continuity of supports Little information about service population needs for planning Individual information system to support individual and system planning Inconsistent quality assurance standards for ministry-funded services Provincial quality assurance measures Ministry of Community and Social Services, Ontario 4

Objectives for implementing SIS in Ontario p Ease burden on individuals and families – consistent, single process for support planning p Improve the quality of services and supports for individuals – use person-centred planning process to tailor supports to individual needs, strengths, priority goals p Improve equity in service decision making and allocation of resources p Improve system sustainability, accountability and cost-efficiency p Gather consistent and accurate data to monitor and track individual outcomes and for system planning and forecasting Ministry of Community and Social Services, Ontario 5

SIS implementation milestones p SIS identified, evaluated and recommended for implementation, along with a home-grown Application for Developmental Services and Supports (ADSS) by a broad stakeholder reference group in 2006 p SIS and ADSS were piloted and tested for validity as comprehensive measures of support needs in Ontario in 2007 -2008 n n n p 1911 assessments completed through the pilot 45 assessors trained by AAIDD Very positive participant evaluation and evidence for validity documented Phased implementation of the tools began in October 2009. Phase I focussed on using SIS and ADSS for individual service planning for people who urgently require residential services n n Another 20 assessors were trained by AAIDD Approximately 1, 500 assessments completed during Phase I p Target population expanded to include all new applicants April 1, 2011 p New single points of access and consolidated information system will be launched July 1, 2011 Ministry of Community and Social Services, Ontario 6

Developing a training program for Ontario p In preparation for province-wide implementation, Ministry reviewed training experience from the pilot and other jurisdictions that had implemented SIS p During consultations on new Ontario legislation, stakeholders provided a convincing case for assessment, service decisions and service provision to be independent functions of the system p It is very important to the Ministry that assessment data is 100% accurate and consistent p To establish a framework for this, the Ministry issued a policy directive, based in the new legislation that: n n n p Mandates SIS/ADSS as the sole assessment system for accessing all Ministry-funded developmental services and supports Establishes qualifications for SIS/ADSS assessors and service standards for SIS/ADSS assessments in Ontario Requires SIS/ADSS assessors to be independent from service decision-making and direct service provision Supporting the policy directive is the Ontario Assessor Training and Data Quality Assurance Program Ministry of Community and Social Services, Ontario 7

Assessor training and data quality assurance p A central trainer position was created in the policy division of the Ministry to develop and provide all assessor training and data quality assurance p The trainer developed a fully integrated training program for SIS, ADSS and a tool that provides a synthesized summary of the results for the individual and service provider (Assessor Summary Report) n n n p Four-day foundations training on SIS and ADSS, orientation and observed assessment; also includes public information on ADSS/SIS to establish a common language for public information about the tools Five weeks for practicing to complete 4 -6 assessments, including some coached interviews Observation and interviewer reliability reviews (IRR); reliability reviews to be completed every 18 months Two-day follow-up training on synthesis of information and Assessor Summary Report, includes cultural competencies for special populations (Aboriginal origins, FASD, new immigrants, Francophone, Hindi, Cantonese/Mandarin, Arabic) Ongoing support and upgrading through the central trainer Ongoing data quality reviews (to be automated) The program began implementation in the fall of 2010 to prepare for the opening of the new single points of access n n Upgrade training for previous assessors who would migrate to the new system: Nov/10–Jul/11 Training for new assessors: Feb/11 -Oct/11 (50+ trained assessors in the system) Ministry of Community and Social Services, Ontario 8

Unique features of Ontario assessor training p Central trainer in government, with assessors in external agencies n n Single points of access have responsibility for administration of direct funding Assessors cannot be employees of service-providing agencies p Implementation coincides with a new consolidated information system p Data reviews/follow-up reliability reviews mandated by the policy directive to manage scope drift p Independent “funding entity” established by the legislation separates assessment from service decision function p Training program is comprehensive and supportive n n p Training, IRR, coaching, observation, practice Support/orientation to single points of access to enable hiring based on competencies and best-practice qualifications After foundations training, most assessors are linked with an experienced coach to observe/shadow and provide feedback on practice sessions Ministry of Community and Social Services, Ontario 9

Observations/initial “lessons” from Ontario p Having one central trainer has some benefits n n n p Avoids potential conflicts of interest regarding who “passes” IRR Helps to maintain objectivity in assessment data Enables a consistent process for dealing with testing, maintaining standards and monitoring Resolving some issues that arose in Ontario n n Regions with dispersed and remote populations – considering the use of video-conferencing for assessments Special populations are concentrated – encouraging the formation of linked networks for support/mentoring Training across a large territory – bring assessors together for initial training and practice (hosting) Difficult situations that arise – role-playing at follow-up sessions to demonstrate and problem-solve Ministry of Community and Social Services, Ontario 10

Next steps p Continue implementation p Refine/upgrade information collection system p Finalize resource allocation and prioritization process Ministry of Community and Social Services, Ontario 11

Questions and Feedback p Questions, comments and feedback are welcome: Monica Neitzert Ministry of Community and Social Services monica. neitzert@ontario. ca 416 -327 -4572 Ministry of Community and Social Services, Ontario 12