Quality Matters Building Capacity and Investment in Youth

Quality Matters: Building Capacity and Investment in Youth Program Quality The Center for Youth Program Quality 1

Objectives • Understand how quality defined and measured by the Youth Program Quality Assessment and other quality assessment tools • Learn about quality improvement systems currently implemented in numerous statewide and place-based networks 2

Systems for Quality Leveraging existing “change” resources Professional Development Local Evaluators Accountability Measurement POS Quality Access to key developmental and learning experiences 3

Quality & Reach… Looking into the developmental white space Outcome Areas Developmental “White Space” Ages school afterschool Times of Day/Year At best, schools fill only a portion of developmental “white space. ” Who fills the rest? And what is the “locally appropriate mechanism for monitoring the availability, accessibility, and quality of programs…” in school and out? Saturating communities with “ample programs” requires improving the quality and reach of all the systems, settings and programs that touch young people’s lives. Selective replication and improvement are not sufficient. 4

The Landscape of Youth Programs Children’s Services in LA County 5 SOURCE: Margaret Dunkle

The Ready by 21™ Quality Counts Framework Domains Elements Strong Policy / Leadership Horsepower Decision-maker engagement & coordinating structures Aligned policies for quality accountability and improvement Shared vision, strong demand, active family/youth involvement Strong, Stable Program Base Healthy program landscape (distribution & focus) Cross-system program data base/info source Cross-system convening/coordination mechanisms Capacity to Assess & Improve Programs Buy in re definitions, quality standards, accountability requirements Widely adopted assessment and monitoring procedures Adequate assessment and improvement training/TA capacity Capacity to Recruit, Train, Retain Workforce Cross-system provider networks and communications Accurate data on workforce (skills, supports, recruitment, retention) Professional development opportunities/incentives 6

What is program quality? inputs ? ? ? outcomes inputs youth program Another way to say it: • What do we want to see in high quality youth programs? 7

Defining Quality Several ways to organize: • Readin’ + ‘Ritin + ‘Rithmatic (old-school) • Affect + Active Learning + Metacognition (Education) • Relatedness + Autonomy + Competence (Psychology SDT) What kids need… Relationship + Task + Increasing Complexity Content Therapeutic process What adults should do…

Our Quality Construct: The Pyramid of Program Quality Plan Make choices Engagement Reflect Lead and mentor Be in small groups Partner with adults Experience belonging Interaction Encouragement Reframing conflict Supportive Skill building Session flow Active engagement Welcoming atmosphere Environment Psychological and emotional safety Program space and furniture Emergency procedures Safe Healthy food and drinks Physically safe environment Environment 9

Viewing Quality in a Systems Context Engage Interaction POS Point Of Service Support Safety PLC Professional Learning Community SAE System Accountability Environment 10

Defining Quality What can management do? Community Continuity Daily supervision & support by site coordinator Pre-session planning Frequent staff meetings Learning Features Data & Information Performance Feedback External monitoring Lesson & curriculum review Focusing Features Management Priorities/Values Youth engagement & content relevance Purposeful relationships Youth voice structures 11

Defining quality at the system level Current Regulatory Models Miss the POS Source: (1998). The NSACA Standards for Quality School-Age Care. There are thirty-six keys of quality and 144 total standards Engagement 6 standards Interaction 4 standards Supportive Environment 33 standards Safe Environment 49 standards Youth Organizational Voice 1 standard Professional Learning Community 19 standards Other 32 standards 12

The Youth Program Quality Assessment (YPQA) – Validated observational assessment tool – Measures quality at the point of service – Assesses frequency and access to key developmental experiences – Can be used to assess progress over time – Content-neutral for use across settings, ages, systems 13

Structure of the Youth PQA Form B Organizational Interview Form A Observation Ask questions, write, score (2 hours) Watch, write, score (3 hours) Program Offering 1 Program Offering 2 Organization Program Offering 3 Program Offering 4 14

Sample Item from the Youth PQA “Domain” III. Interaction “scale” III-L. Youth have opportunities to develop a sense of belonging. Note: Structured refers to the quality of being intentional, planned, and/or named; it does not refer to informal conversation. Indicators 1 Youth have no opportunities to get to know each other (beyond selfselected pairs or small cliques). Supporting Evidence 3 Youth have informal opportunities to get to know each other (e. g. , youth engage in informal conversations before, during, or after session. 5 Youth have structured opportunities to get to know each other (e. g. , there are team -building activities, introductions, personal updates, welcomes of new group members, icebreakers, and a variety of groupings for activities) “item row” The Youth PQA consists of 7 Domains (4 in A, 3 in B) 30 Scales (18 in A, 12 in B); 103 item rows (60 in A, 43 in B) 15

What POS Quality Looks Like on the Ground Occurred For All Occurred For Some Did Not Occur • Sample of nearly 600 different youth workers • Parallel findings in schools research 8/13/2008 The Center for Youth Program Quality 16

Does it Work? Findings from Several Samples • POS quality-outcomes findings: – Supportive environment related to: Attendance – Interaction related to: Interest in program – Engagement related to: Sense of challenge, sense of growth, school-day reading, school-day suspension – Note: No offerings get to high engagement without high support and high interaction • Quality Improvement (YPQI) Findings – Scores increase from pre to post – Scores increase in the targeted areas more – Management practices are related to quality change (Vision, Feedback, Continuity)

How we think about DDCI - People change not programs Prochaska, J. O. , & Di. Clemente, C. C. (1982). Transtheoretical therapy toward a more integrative model of change. Psychotherapy: Theory, Research and Practice, 19(3), 276 -287. Maintenance Repeat cycle Action POS Point Of Service Implementation & coaching PLC Professional Learning Community Preparation Contemplation Planning with data Quality assessment Individual Change Model 8/13/2008 The Center for Youth Program Quality SAE System Accountability Environment Organizational Context 18

Competencies for Youth Development Practitioners Emergence of POS quality culture Spend time planning & prepping activities Improve practice systematically "Team focused on youth experience" mentality Intentionality in POS quality practice Icebreakers and inclusion Cooperative grouping strategies Targeting learning edge – scaffolding, engaging youth in planning & reflection, active learning Providing opportunities for youth voice and leadership 19

Competencies for Management – Knowledge/understanding of positive youth development methods individually and integrated as the High/Scope participatory learning approach; – Conduct reliable performance assessment based on observation and validated measurement rubrics; – Conduct performance coaching based on strengths based transmission of performance data; – Lead a staff team through a data driven quality improvement process 20

System Level Outputs – Accountability policy with high returns in staff buy-in and learning; – Common language and terminology supporting focus on quality at the point of service; – An integrated professional development investment that is integrated over time as a sequence of PD experiences, integrated across levels of organization and professional roles, and integrated with ultimate program purposes in positive youth development; 21

Outcomes – High quality programs – Better staff retention and retention of better staff – Greater impact on youth development and learning 22

Systems for Quality Accountability Policies in Places • YPQA is part of state and county accountability policies: – Cross sector (DHS& DOE) snapshots: Iowa, Washington, Arkansas – Statewide 21 st Century: Michigan, Maine, Minnesota, Rhode Island, New Mexico, – Cities and Counties: Rochester, Detroit, Grand Rapids, Palm Beach Rochester Minneapolis Washington* Grand Rapids Chicago New etroit York Iowa Rhode Island ` Indianapolis mbus Georgetown Divide Columbus St. Louis Oklahoma ` ` Kentucky Nashville l Austin West Palm Beach County 23

Designing Quality Improvement Systems (QIS) Mostly Mangers System External quality Capacity assessment (a) Managers and direct staff Use of on-line dashboards and training (d) External Quality Report with Norms Quality Advisor (e) TOTs for quality assess, coaching, and youth work methods (f, g) Coaching & Training Self-Assessed Quality Report Program Self. Staff Skill assessment & Knowl of Quality (b) Use of on-line dashboards and training (d) Planning with Data (c) Phase 1: Readiness & Capacity Targeted youth Quality work methods coaching by training for direct managers (i) staff (h) Phase 2: Impact & Sustainability 24

Defining the Purpose of Your QIS Lower Stakes Higher Stakes (the creative middle) Program Self-Assessment External Assessment Rough data to get staff thinking and discussing program quality in the context of best practice Precise data for internal and external audiences for evaluation, monitoring, accountability, improvement, reporting Less time Less money Impact internal audiences More time More money Impact internal and external audiences 25

Columbus Indiana Phase 1: Building Local Capacity POS Point Of Service SAE System Accountability Environment STEP 1 Decide to build system PLC STEP 2 a Selfassessment STEP 2 b Professional Learning Community STEP 3 STEP 4 STEP 5 Plan for improvement Carry out plan Measure change External assessment August 25 Quality Matters Presentation August 26 Youth PQA Basics January 27 -28 Ext Assessment Annually October 8 Planning with Data Improvement Plan Opt Phase 2 Method Workshops Quality Coaching Program SA Ext Assessment Observe-Reflection Planning with Data

Questions about… Purpose? Process? Pilot? Next Steps… 27