Скачать презентацию ACA s Research on Youth Outcomes and Program Improvement Скачать презентацию ACA s Research on Youth Outcomes and Program Improvement

055e16b2bc820eb4db40cd8c631ed77d.ppt

  • Количество слайдов: 31

ACA’s Research on Youth Outcomes and Program Improvement Karla A. Henderson, Ph. D. Professor ACA’s Research on Youth Outcomes and Program Improvement Karla A. Henderson, Ph. D. Professor North Carolina State University Deb Bialeschki, Ph. D. Senior Researcher American Camp Association

Overview National Youth • ACA National Outcomes Study Development • ACA National Program Improvement Overview National Youth • ACA National Outcomes Study Development • ACA National Program Improvement Project (with Research national Benchmark component) Projects Educational Materials • “Workbooks” • Web-based tools and courses

American Camp Association (ACA) ACA addresses human development for children, youth, and adults ACA American Camp Association (ACA) ACA addresses human development for children, youth, and adults ACA is a community of camp professionals dedicated to assuring quality in organized camping At least 10 million individuals are associated with camp programs each year ACA accredits over 2300 camps representing different sponsorships, activity foci, and diverse campers from across the United States

National Outcomes Study Camper Growth Instruments (CGI) • Pre-, post-, 6 month surveys with National Outcomes Study Camper Growth Instruments (CGI) • Pre-, post-, 6 month surveys with >5200 campers and their parents • Pre- and post- counselors’ observation checklists • Directors’ survey Major Findings (Directions) • Significant growth in all 4 domains (Positive Identity, Social Skills, Physical & Thinking Skills, and Positive Values & Spirituality) • Much of the growth was maintained over 6 months • Parents indicated significant growth for their children in all 4 domains as a result of the camp experience as did the children’s counselors

Needs That Emerged From The National Outcomes Study Need additional outcomes tools • • Needs That Emerged From The National Outcomes Study Need additional outcomes tools • • Outcomes measures that are “modular” Age-appropriate choices Easy to administer and analyze surveys Reliable and valid instruments Need a “workbook” suitable for seasonal staff training • Help staff connect their job responsibilities to outcomes • “Recipe for success” (logic model) • Facilitator guides Need to look at how to provide an environment that supports positive youth development

ACA Camp Outcomes Surveys* Friendship Skills Responsibility Independence Family Citizenship Teamwork Competence Exploration *Reliable ACA Camp Outcomes Surveys* Friendship Skills Responsibility Independence Family Citizenship Teamwork Competence Exploration *Reliable and valid measures

Camper Learning Camper Learning questionnaire (Interview Questions) 14 questions Measures camp learning based on Camper Learning Camper Learning questionnaire (Interview Questions) 14 questions Measures camp learning based on 7 domains Examples: 1. At camp, did you learn how to be better at making friends? 2. At camp, did you learn to feel good about things that you do well? 3. At camp, did you learn that you can be good at some things that you didn’t know you were good at? 4 point scale: I didn’t learn anything about this I’m not sure I learned a little about this I learned a lot about this

Older Camper: Basic Version Basic “Increase Format” Available for 7 outcomes 6 -14 items Older Camper: Basic Version Basic “Increase Format” Available for 7 outcomes 6 -14 items per outcome 5 point scales How much, if any, has your experience as a camper in this camp changed you in each of the following ways? Decrease d 1. Placing group goals above things that I want. 2. Working well with others. Did not increase or decrease Increased a little bit, maybe Increased some, I am sure Increased a lot, I am sure O O O O O

Older Camper: Detailed Version “Status + change format” Available for 7 outcomes 6 -13 Older Camper: Detailed Version “Status + change format” Available for 7 outcomes 6 -13 items/outcome (status and change) 6 point scales 1. I accept responsibility for my actions False Somewhat False A little True Somewhat True Is the above statement more or less true today than before camp? A lot less Somewhat less A little more Somewhat more A lot more False Somewhat False A little True Somewhat True A lot less Somewhat less A little more Somewhat more A lot more 2. I own up to my mistakes Is the above statement more or less true today than before camp?

How do I know which one(s) to use? ? Young Camper (Camper Learning) Best How do I know which one(s) to use? ? Young Camper (Camper Learning) Best for campers 7 -9, possibly up to 10 or 11 Shortest & fastest Unidimensional (camp learning) Basic Version Campers are 10+ Interested in gains through camp Less burdensome than status + change Multidimensional (7 outcomes) Detailed Version Campers are 10+ Interested in gains through camp Interested in status after camp (correlations with other indices) Most burdensome Multidimensional (7 outcomes)

How Sherwood Forest Camps used the Outcomes Measures During staff orientation Work by camper How Sherwood Forest Camps used the Outcomes Measures During staff orientation Work by camper age groups Choose two outcomes for each village Measure those outcomes for at least two summers With the campers In village meetings In cabin groups At teachable moments

Additional ACA Evaluation Resource: Creating Positive Youth Outcomes Tool Kit A 7 -step process Additional ACA Evaluation Resource: Creating Positive Youth Outcomes Tool Kit A 7 -step process for program evaluation to help you increase positive youth outcomes. A book and resource CD include: the evaluation outline, examples of goals and objectives you might target, a facilitator's guide for staff training, copy-ready checklists and worksheets, forms for refining targeted outcomes, and youth survey tools designed and used by ACA for youth outcomes research.

Benchmark & PIP Studies Phase 1: Benchmarked supports and opportunities perceptions of 7645 campers Benchmark & PIP Studies Phase 1: Benchmarked supports and opportunities perceptions of 7645 campers at 80 ACA accredited camps with YDSI survey Phase 2: Program improvement year long process with 23 of the 80 camps

Community Action Framework for Youth Development © Connell & Gambone 1998 Build Community Capacity Community Action Framework for Youth Development © Connell & Gambone 1998 Build Community Capacity and Conditions for Change (E) Increase Supports and Opportunities for Youth (C) Improve Youth Development Outcomes (B) --Adequate nutrition, health and shelter --Multiple supportive relationships Implement Community Strategies to Enhance Supports and Opportunities for Youth (D) --Meaningful involvement --Challenging and engaging skill building activities --Safety Improve Long. Term Outcomes in Adulthood (A)

A Word About Supports & Opportunities Supportive Relationships Safety Youth Involvement Skill-Building • Guidance, A Word About Supports & Opportunities Supportive Relationships Safety Youth Involvement Skill-Building • Guidance, emotional & practical support • Adult & peers know who they are & what’s important to them • Physical & emotional • Leadership • Decision-making • Belonging • Challenging and interesting experiences that build a wide array of skills • Experience sense of growth and progress

Overall Supports and Opportunities (Optimal) (Insufficient) (Optimal) Overall Supports and Opportunities (Optimal) (Insufficient) (Optimal)

Camp Characteristics Dimension Overall Agency Religious 62% 71% 78% 67% 47% 72% Safety 28 Camp Characteristics Dimension Overall Agency Religious 62% 71% 78% 67% 47% 72% Safety 28 35 34 25 22 31 Youth Involvement 4 5 6 6 3 6 Skill-Building 35 41 51 36 27 42 2584 1338 1974 1756 883 6771 Supportive Relationships Sample Size IFP INFP Day Resident

Camp Characteristics Dimension Overall 1 week 2 -3 weeks 61% 66% 77% 66% 72% Camp Characteristics Dimension Overall 1 week 2 -3 weeks 61% 66% 77% 66% 72% Safety 27 28 34 27 30 37 Youth Involvement 4 5 7 5 6 6 Skill-Building 35 39 47 37 54 43 2410 2302 2923 4773 814 2101 Supportive Relationships Sample Size 4+ weeks Co-ed Boys Girls

Camper Characteristics Dimension Overall 12 -13 yrs old 14 -15 yrs old 16 -18 Camper Characteristics Dimension Overall 12 -13 yrs old 14 -15 yrs old 16 -18 yrs old 62% 69% 75% 82% 64% 72% Safety 25 29 34 44 22 35 Youth Involvement 3 4 7 15 4 6 Skill-Building 42 40 39 46 39 42 2366 3079 1749 451 2968 4586 Supportive Relationships Sample Size 10 -11 yrs old Male Female

Camper Characteristics Dimension Overall White Non-white 1 summer 2 -3 summers 4+ summers 72% Camper Characteristics Dimension Overall White Non-white 1 summer 2 -3 summers 4+ summers 72% 56% 59% 70% 76% Safety 33 20 20 30 38 Youth Involvement 6 3 3 5 7 Skill-Building 43 33 37 42 43 5946 1607 2263 2647 2644 Supportive Relationships Sample Size

Phase 2: Program Improvement 23 camps from original 80 benchmark camps 6 month process: Phase 2: Program Improvement 23 camps from original 80 benchmark camps 6 month process: identify and set targets gather input from campers and staff conduct organizational assessment develop action plan set first year priorities Re-surveyed second summer

Eight Steps to PPI Ask: Campers Ask: Staff Assess: Organizational Practices Brainstorm: Strategies Choose: Eight Steps to PPI Ask: Campers Ask: Staff Assess: Organizational Practices Brainstorm: Strategies Choose: Strategies and Develop Action Plan Do: Take Action Share: Action Plan Evaluate: Take Stock

Organizational Process Improvement Organizational structures (S): • Low youth to staff/volunteer ratios • Safe, Organizational Process Improvement Organizational structures (S): • Low youth to staff/volunteer ratios • Safe, reliable, and accessible activities and spaces • Continuity and consistency of care Organizational policies (P): • Ongoing, results-based staff and organizational improvement process • Flexibility in allocating available resources • Community engagement Organizational activities (A): • Range of diverse, interesting, and skill-building activities • Youth engagement in organizational decision making • High, clear, and fair standards

Findings About Strategies Grouped individual strategies into 7 categories: Mission- related Camp activities Programming/scheduling Findings About Strategies Grouped individual strategies into 7 categories: Mission- related Camp activities Programming/scheduling changes Staffing patterns Camper input & involvement in planning and decision-making Staff training Facility

Major Findings- Phase 2 (PIP) (Innovations, ACA, 2006) Significant improvement occurred in all areas Major Findings- Phase 2 (PIP) (Innovations, ACA, 2006) Significant improvement occurred in all areas Youth Involvement and Skill Building (83% in one or both areas) More than 33% also strengthened in Supportive Relationships and Safety Intentional, camper-centered assessment and planning was important to positive change Camps with changes across all three organizational practices (structures, policies, activities) were twice as successful at producing change than if only 1 -2 areas were addressed.

Percent ACA PIP Camps with Positive Change in 1 or More Dimensions Percent ACA PIP Camps with Positive Change in 1 or More Dimensions

Learned When Planning For Change… 1. Start with organization specific data 2. Use a Learned When Planning For Change… 1. Start with organization specific data 2. Use a youth development framework 3. Focus on intentionality and accountability

Resources to help… Research-based (ACA’s youth development “supports and opportunities” study and Program Improvement Resources to help… Research-based (ACA’s youth development “supports and opportunities” study and Program Improvement Project) Focuses on a step-by-step process for organizational improvement (across structures, policies, and activities) On-line web-based support course suitable for staff training

Future Research Ideas… Staff training around quality improvement (age and seasonality of staff) Parent Future Research Ideas… Staff training around quality improvement (age and seasonality of staff) Parent understanding/perceptions about camp programs and their children’s growth Issues of intensity and dosage of the camp experience Effect of the nature-based setting Issues related to age/stage & optimal developmental supports Relationships of camp experience on work force skills development “Culture of camp” within changing societal demographics Role of camp experience within mental/physical health perspective (child stress, physical activity, healthy lifestyles, etc)

Questions? Contact Deb Bialeschki at dbialeschki@acacamps. org Go to www. acacamps. org/research Questions? Contact Deb Bialeschki at [email protected] org Go to www. acacamps. org/research