Programming Tools and Techniques Making Your Advanced Master

Programming Tools and Techniques Making Your Advanced Master Gardener Program Work VCE-Advanced Master Gardener Water Stewards Master Gardener College June 20, 2013 Pat Reilly

Name, unit Are you joining a program already established or creating a new program? Introductions

Understand why Advanced VCE-MG programs need to address strategic goals and how to do this 2. Provide logical and do-able steps 1. a. Planning a new program b. Evaluating / improving an existing program Objectives

Value judged on relevance Strategic goals best represent social needs Funding based on worth Accountability Why Address Strategic Goals?

Government Performance & Results Act Government Accounting Office of Management and Budget Council of Economic Advisors Internal agency evaluators Federal Level

Virginia, county, city budgets Effectiveness and efficiency Elimination of duplication of effort & reduction in funding required State and Local Level

Business partners ◦ Bottom line ◦ Build good will Extension & Master Gardener leadership ◦ Volunteer time ◦ Financial resources Public, Private Level

Stewards’ Role in VCE Programming 2. VCE Programming Process 1. ◦ ◦ ◦ Situation Analysis Program Design and Implementation Program Evaluation and Reporting Situation Analysis – Organization 4. Situation Analysis – Assess Community Needs 3. Programming Manual

Comprehensive Planning 6. Planning Level 3 or 4 Programs 7. Evaluation and Reporting 8. Community – Volunteer Management 9. Publicizing Your Program, Resource Development 10. Presentation and Final Planning Meeting 5. Programming Manual

To appraise, value, judge the worth of It involves collecting, analyzing, interpreting and communicating information about the effectiveness of policies and social programs for the purpose of improvement Evaluation easiest when designed into the program Program Evaluation Defined

Effectiveness and efficiency Program outcome/impact Process and implementation Program design and theory Need for the program Evaluation Hierarchy

Assess the need for the program Develop theories for why and how your program will meet needs; design the program Create the steps or processes and implement the program Monitor for outcomes and impacts Tell your story/report findings Continuously improve the program Steps for Program Evaluation

Identify who cares, and why 2. Define the target population 3. Identify the needs, outcomes and solutions 4. Prioritize the needs 1. Needs Assessment

1. a. b. c. d. Identify who cares and why Resources Who wants the issue addressed Who are all the “customers”? Develop a needs statement Needs Assessment – Step 1

1. Define the target population a. Who are these people? b. Do the numbers support the scope? c. Do you have the ability to reach them? 2. Service delivery environment a. Level of programming b. Where will you offer your service? Needs Assessment – Step 2

Level 1 – awareness; information distribution (reactive) Level 2 – awareness, personal commitment, and increase in knowledge; limited educational programs (reactive with some outreach) Level 3 – increasing knowledge and skills; more dynamic, one-time events (proactive) Level 4 – behavioral change; on-going program; community is beginning to assume responsibility (long-term structured/proactive) Level 5 – spin off the program (reduction in VCE MG leadership of program) Programming Levels - Goals

Identify the needs 2. Describe expected and current outcomes 3. Generate possible solutions, plans, activities 4. Chart your results 1. Needs Assessment – Step 3

Set priorities for decision-makers (discrepancy, or “gap” model) 2. Needed 3. Achievable 1. Needs Assessment – Step 4

Assess the need for the program Develop theories for why and how your program will meet needs; design the program Create the steps or processes and implement the program Monitor for outcomes and impacts Tell your story/report findings Continuously improve the program Steps for Program Evaluation

What is the reasoning behind the cause of the problem? Will program as designed address the condition? ◦ Assumptions are made when creating activities ◦ Examining the program theories will ensure that the activities will have the desired effect Program Design and Theory

1. Methods for Arriving at the Cause a. Ishikawa or “fishbone” diagram b. Social marketing technique Program Design and Theory

Sample Cause and Effect Diagram People Policies Lack knowledge Don’t know it can be easy Don’t know how Don’t have stuff Do n inc e orr ect ly To thi o ma Piles stink ng s t ny od o Don’t know it can be easy Don’t have time Th i bad nk pil es a re Zoning or HOA rules Don’t have equipment W on ’t do Mix Don’t know what’s green or brown bro gree na e wn nd ak rd ffo t m. Won’t buy Do ’ ta “How to” complicated on n’ D val esn’t W on’ ue un Ca de be ‘ t know rst just it ca an pile n d it’ “How to” complicated County doesn’t require Trash haulers pick up trimmings Equipment/ Supplies Procedures fill Landfills up to w tly ne s Co eate cr ng t to en ss i Tr i mm d lan Environment Citizens do not compost

Cause &Effect Diagram – Root Causes People Policies Lack knowledge Don’t know it can be easy Don’t know how Don’t have stuff Do n inc e orr ect ly To thi o ma Piles stink ng s t ny od o Don’t know it can be easy Don’t have time Th i bad nk pil es a re Zoning or HOA rules Don’t have equipment W on ’t do Mix Don’t know what’s green or brown bro gree na e wn nd ak rd ffo t m. Won’t buy Do ’ ta “How to” complicated on n’ D val esn’t W on’ ue un Ca de be ‘ t know rst just it ca an pile n d it’ “How to” complicated County doesn’t require Trash haulers pick up trimmings Equipment/ Supplies Procedures fill Landfills up to w tly ne s Co eate cr ng t to en ss i Tr i mm d lan Environment Citizens do not compost

Social marketing technique New Behavior Competing Behavior #1 Behavior #2 Perceived Benefits Perceived Barriers Program Design and Theory

Activities and outcomes 1. Why you think that the activities you have picked for your program will achieve the expected outcomes? 2. Graphically display the program from idea through outcomes a. Flow chart b. Logic model Program Design and Theory

Precautions! 1. Regulation is contingent on people’s willingness to be regulated 2. Education alone has little or no effect on sustainable changes in behavior 3. Economic self-interest has limited success in changing behavior 4. Community-based social marketing has had documented success Program Design and Theory

Basic Flow Chart Client requests composting information MG Steward recruits for program Client attends classes Client receives composter Steward conducts survey

Detailed Flow Chart Client requests composting information MG Steward Provides program information Client completes enrollment form A B Client provided class schedule Client no completes program 6 mos Steward contacts client Steward conducts survey yes Form processed A Client receives composter B Client is active composter

Sample Logic Model Long term outcomes Intermediate outcomes Initial outcomes Outputs Neighborhoods are safer and healthier; surface and groundwater cleaner Residents implement BMPs Fewer system failures; less Health Dept resources are used/consumed Residents are knowledgeable on septic BMPs Data on implementation of septic BMPs Classes are offered; residents attend Pre-survey, post-assistance evals; septic pump-outs Activities Program provides residents with 2 classes two weeks apart in classroom on septic system operation, individual site plan assistance and surveys before classes and 2 months after site plan assistance Neighborhood provides meeting space, marketing support, residents to participate Inputs Health Department identifies communities needing septic program Stewards provide program management, instruction, printed materials, site assessments

Assess the need for the program Develop theories for why and how your program will meet needs; design the program Create the steps or processes and implement the program Monitor for outcomes and impacts Tell your story/report findings Continuously improve the program Steps for Program Evaluation

Has the program been implemented as designed? 1. Reaching the target population 2. Service delivery consistent with design 3. Support functions providing assistance …appropriate, adequate, intended, sufficient, satisfactory, reasonable… Process Theory

Regular review 1. Compare logic model or flow chart to actual operation 2. Write a procedure of actual operation and have program designers review 3. Ask program support if their role(s) and activities are as described Process Theory

Assess the need for the program Develop theories for why and how your program will meet needs; design the program Create the steps or processes and implement the program Monitor for outcomes and impacts Tell your story/report findings Continuously improve the program Steps for Program Evaluation

Data collection at regular intervals 2. Data analysis a. Data relevance, reliability, repeatability b. Coverage of the target population c. Bias I. Self selection II. Taking the cream off the top 1. Monitoring

Outputs 2. Outcomes 3. Caution! When making claims… 1. a. Attrition of your client base b. Long-term trends which occur without your program Outcomes and Impacts

The measures everyone wants (expects) to hear 2. Most difficult to determine 3. Requires the most information 4. Relies on the other layers of the evaluation hierarchy 1. Effectiveness and Efficiency

1. Cost-benefit analysis a. Outcomes expressed in monetary terms b. Dollars on water quality program and dollars on water treatment 2. Cost-effectiveness analysis a. Outcomes expressed in substantive terms b. Dollars on converting polluters Effectiveness, Efficiency Models

Assess the need for the program Develop theories for why and how your program will meet needs; design the program Create the steps or processes and implement the program Monitor for outcomes and impacts Tell your story/report findings Continuously improve the program Steps for Program Evaluation

Appropriate to the stakeholders 2. Methods a. Reports b. Presentations c. Personal interaction 3. Content a. Mix data and stories 1. Report Findings

Map your intended or current program 2. Determine what impact you want to report 3. Collect and analyze data 4. Tell your story 1. VCE Impact Statements

“Participants apply appropriate best management practices” “Participants identify pathogen before applying a chemical control” “Participants build successful coalitions and networks with stakeholders” “Participants share their awareness of community issues with others” VCE Behavior Outcome Examples

Assess the need for the program Develop theories for why and how your program will meet needs; design the program Create the steps or processes and implement the program Monitor for outcomes and impacts Tell your story/report findings Continuously improve the program Steps for Program Evaluation

Establish a schedule for program evaluations every year or two 2. New people, new ideas and new external forces require reviews 3. Consider external evaluators 1. a. Different perspective b. New audience aware of program(s) Continuous Improvement

Questions?