Writing a Smart CTCP Proposal Hosted by

Writing a Smart CTCP Proposal Hosted by CTCP/CBN March 18, 2010 10: 00 – 11: 30 a. m.

Agenda n 10: 00 – 10: 05 Gathering n 10: 05 – 10: 15 Setting the Stage (Applicant Capability) Shirley Shelton & La Roux Pendleton n 10: 15 – 11: 00 Body & Soul (SOW/Evaluation) April Roeseler & Caroline Kurtz n 11: 00 – 11: 15 Conclusion (Budget) Linda Ono n 11: 15 – 11: 30 Questions & Discussion

Homework n Know who you are – don’t wait for RFA/RFP n Clear, Concise Existing n n n TCP Mission n n Mission Statement Agency Goal Agency Objectives Project Description Values Improve the health of all Californians by reducing illness and premature death attributable to the use of tobacco products. TCP Goal n Prevent tobacco-related disease and death in California by reducing tobacco use across the state.

Stay Focused

Resources n n www. calendow. org/grant_guide/ www. tobaccopreventionnetworks. or g/site/c. ks. JPKXPFJp. H/b. 2580071 http: //www. tobaccofreekids. org/campaign /state/ http: //www. federalgrantswire. com/centers -for-disease-control-andpreventioninvestigations-and-technicalassistance. html

Applicant Capability Section

Purpose of the Applicant Capability Section Agency’s demonstration of their capacity and ability to carry out requirements of the RFA (based on the scoring criteria)

Applicant Capability Sections in RFA 10 -100 Program Experience Organizational Start-Up Administrative/Fiscal Experience Equipment Letters of Reference

Program Experience Questions Agency Mission Program Staffing Demonstrated Results Demonstrated access to priority populations and cultural competency Non-traditional and traditional partners

Administrative/Fiscal Experience Questions Describe the Primary Agency’s (and subcontractors’ and/or consultants’, if applicable) performance within the last three years with the management of government and/or non-government funds and activities, including administrative, fiscal, program, and evaluation functions such as: timely and accurate completion of deliverables; submission of fiscal, program, and evaluation documentation; subcontract/consultant monitoring; compliance with government requirements; and fiscal ability to manage payments in arrears. Describe the agency’s fiscal and programmatic audit history within the last three years. Information to include: frequency of audits, dates of the most recent audits, and a summary of the major audit findings. Indicate if the Proposer has been audited by a governmental agency within the last three years. If yes, include: the name of the agency, the agency’s contact person and phone number; the year the audit was conducted; and the outcome of the audit. Negative audit findings should be thoroughly explained. Describe the Primary Agency’s current administrative staffing pattern for activities such as contract management and oversight, payroll, bookkeeping, invoicing, and tracking of contractual, administrative, and fiscal controls. Describe the qualifications of key administrative staff; including their educational background and experience with monitoring government grant funds, overseeing, and managing the administrative and contractual functions of subcontractors and/or consultants.

Letters of Reference Questions Solicit, scan, and upload three letters of reference into OTIS. Letters of reference must be written to the Primary Agency within the last two years. No more than three letters will be accepted. If the Primary Agency has in the past or is currently receiving or within the last two years has received funding from a local, state, or federal agency, other than CTCP, one of the references must be from one of these agencies. The letters are to be from three separate agencies or businesses. Each letter must be on the reference provider’s letterhead and include: n n The address, telephone number, e-mail, name, and title of the letter’s author; A description of the positive relationship and capacity in which the reference provider worked with the Primary Agency; The Primary Agency’s fiscal and administrative ability to manage government grant funds, including satisfactory performance with administering and managing government and/or non-government funds through timely and accurate submission of fiscal, program, and evaluation documents; A description of the Primary Agency’s ability to change social norms with regard to the interventions or programs mentioned.

Building a Smart Tobacco Control Plan April Roeseler California Tobacco Control Program California Department of Public Health

Narrative Adventure in Travel • The CX narrative describes the pre-trip research conducted that describes why you are going to this particular destination (aka your objective)

Where do you want to go? • Indicators and assets reflect possible travel destinations to investigate – Smoke-free outdoor dining – Smoke-free multi-unit housing – Tobacco retail licensing – Cessation services – Increased coalition diversity – Increased level of community activism

Benefits to Pre-Trip Research • Avoid getting lost • Pack the right clothes for the weather and activities • Avoid fights with traveling companions over the quality of the hotels or food • Avoid ruining relationships over unmet expectations • Avoid wasting time and money

Benefits to Pre-Trip Research • There’s a vast number of tobacco control destinations and you have budget, people resources, and time • Research a few possible destinations (indicators and assets)

“I wish my bags had been lighter” Benefits to Using Data in Your Pre- trip Research • Arrive at a destination that is worth going to • Helps you defend to others that this is an important destination (especially when it is a controversial destination) • Keeps you focuses on spending your limited resources on changing tangible community norms, (e. g. , How will my community be different as a result of this trip? ) • Helps you figure out what items you need to pack for the trip (e. g. , paid advertising, spokesperson training for coalition members, cameras to document the trip)

CX an Adventure in Travel • Your scope of work is your travel itinerary which details where you will stay, sites you will visit, and the timeline for various portions of your trip

CX an Adventure in Travel • Your evaluation documents your trip so you can share your experiences with others and help have an even better trip next time

Money Matters • Your budget guides whether you will be staying in youth hostels or 4 -star hotels, whether you will be eating at nice restaurants or from street vendors, how much you have to spend on souvenirs

Budgeting Things to Think About • The budget needs to support/sustain the SOW – Do you have the right staffing for program, materials development, evaluation, administrative activities? – Did you budget for educational materials, behavior modification and advocacy campaign materials, media, data collectors, etc. ? – Does the quantity of work support the budget request?

Advice on Traveling Companions Have a Great Trip • Get buy-in for travel destinations • Set expectations about the trip • Picking people who know the language spoken or have familiarity with the destination can make for a smoother trip

Be a SMART Planner

Objectives • Who or what is expected to change? • What and how much change will change? (do not use ranges) • Where will the change occur? • When will the change occur?

Example When Who By June 30, 2010, the City of Vista will adopt and implement a policy that prohibits the smoking of cigarettes, cigars, and hookahs in outdoor dining, bar and patio areas What Outcome is Expected from the Intervention

What’s wrong with this objective? By June 30, 2010, Data from 250 -500 retailers will be analyzed to determine communities with 90% compliance rates with policies that prohibit the sale of tobacco sales to minors as assessed by enforcement agencies. Answer Who/what is to change? Communities? Uses a range. How much change is expected? 90% compliance rate, but it is not clear that this is a change vs. analyzing existing data. Where is the change to occur? No information provided. When is the change expected to occur? 6/30/2010

What’s wrong with this objective? By June 30, 2013: Based on the results of a needs assessment, and in coordination with local projects, increase youth involvement in local smoke-free housing policy efforts resulting in at least 20% of participating local CDPH/TCP-funded projects successfully working with multi-unit housing to adopt indoor and/or outdoor smoke-free policies. Answer Who is to change? Not clear if it is youth changing their involvement in MUH policies or are communities changing by adopting smoke-free housing policies. Are these multi-unit housing policies or is something else meant by smoke-free housing policy effort? How much change is expected? Not clear—is youth involvement to increase or is there to be 20% increase in CDPH/TCP projects working successfully on MUH? Where is the change to occur? No information provided. When is the change expected to occur? 6/30/2013. Age appropriateness of this objective? If you had one objective to target towards and engage high school age youth in the activities would MUH be the one that you would select?

Design a Comprehensive Approach • Influence individual behavior change through community norm change approach – Beyond education/information dissemination – Engage the community (letters to the editor, endorsements, public forums, petitions, development of relationships) – Use PR and advertising – Use data strategically to demonstrate support for policy

Demonstrate Coordination/Collaboration • Tobacco control work is a team effort • CTCP is looking for evidence of coordination and collaboration with statewide technical assistance projects

Examples

Obsession with Quantification • Helps support your budget request for personnel and operating expenses • Helps justify the % deliverable assigned • May demonstrate that the activity is too ambitious • It’s okay to use ranges in activities

What’s wrong with this activity? Conduct merchant education trainings with local law enforcement agencies. The training will provide an in depth definition of The Stake Act, including scope, exceptions, enforcement, and penalties associated with violation of law. Answer Lacks quantification. How many trainings will be conducted? How many will be trained? How frequently will the trainings be conducted? What is the format for the training—in person, webinar, self-taught course? Lacks clarity. What does it mean to conduct merchant education training with law enforcement agencies? Merchant education implies this addresses how to check ID and not sell to underage youth. Would you expect that law enforcement agencies would benefit from that or would they benefit from legal training on how to conduct a compliance check, methods to collect evidence, information on support with recruiting youth decoys for compliance checks, how to deal with telephone trees, etc. ?

Behavior Modification vs. Campaign Advocacy Materials • Behavior Modification Materials – Motivate/reinforce behavior – Identify the material and the behavior – Behavior may be cessation, policy adoption, helped collected data – No cash. May not be >$50 person/year – Food may be used

Behavior Modification vs. Campaign Advocacy Materials • Advocacy Campaign Materials – Give-aways that generate attention for your program or “give legs” to your message • Separate behavior modification and campaign advocacy materials in your SOW and Budget, even if the same key chain is being used as both an behavior modification material and an advocacy campaign material

Sponsorship • A form of advertising in which your project gets something in return, (e. g. your name prominently promoted at an event through a program, signage, announcements, advertising) • It is NOT – Giving money to community groups to put on an event – A “reward” for adopting a policy

What’s wrong with this activity? Give 2 -4 $100 -$500 media sponsorships to community groups associated with the target groups annually to advertise the Helplines, give culturally appropriate quit kits and conduct other event appropriate media. Answer Inappropriate use of the term sponsorship. Sponsorship is one of the four arms of marketing (advertising, sales promotion, public relations, and sponsorship). It It provides access to a live audience, on-site sampling (e. g. of anti-tobacco use materials), and opportunities to survey audiences about tobacco control issues. A better term to use would have been “stipend. ”

Materials Development Form • Completed for: – Substantive, original works – Have the potential for statewide distribution – High quality development process • • Factually correct, free of grammar/punctuation errors Pilot/field testing Releases for artwork, photos, models obtained Professional production values • Includes: – Educational materials, – Behavior modification and Advocacy campaign material s – Advertising

Copyright • Copyright denotes ownership • Assign it to: – Original materials developed – Works for hire (logos, artwork) – Program and Evaluation products – Creative work developed as part of a contest • Don’t assign it to things you want others to freely use: – Sample policy – Press release – Fact sheet

Percent Deliverables • Assign to tangible products or services • Assign in 0. 5% increments • The % deliverable is inclusive of all the collaborative/planning activities leading up to the product or service • Used to assess portion of budget directed towards various indicators/assets • Used by CTCP at close-out to reduce final payment if deliverables were not completed

What’s wrong with this Percent Deliverable? By 06/30/2010, 50 tobacco retailers in XXXXX County will be in compliance with STAKE Act signage. Total % for all activities = 25% Budget =$823, 975/3 years 25% x $823, 975 = $205, 994/50 stores = $4, 000/store to post signs

Target Populations • This is not a contest to check the most boxes! • Identification facilitates OTIS searches for networking/collaboration • Only mark those that the intervention directly touches – General Audience Groups – General Ethnic Population Groups – Specific Ethnic Population Groups*

Intervention Topics • These summarize the narrative activity descriptions • This is not a contest to check the most boxes! • Identification facilitates OTIS searches for networking/collaboration – Promote California Smokers’ Helpline – Provide adult cessation classes – Train health care providers • Recommend that you go-back & look at this after completing the narrative activity descriptions in the SOW

Responsible Parties • Budgeted – Make sure the title given in the SOW matches the title used in the budget • Non-Budgeted – Collaborative partners you have control over – Okay to budget a stipend

Tracking Measures • These are the souvenirs that you will collect and save from the trip (e. g. , press releases, training materials, survey instruments, and sign-in sheets) – Used to verify completion of activity – No more than 2 per activity • Some of these things you will share with your friends at CTCP in progress reports, others you will keep in your office and only share if your CTCP friends come visit your office to hear about your travels.

Tracking Measures • Submitted – Sent or uploaded to CTCP – Substantial, original items (e. g. educational materials, agendas, data collection instruments) • On File: kept at your agency – Sign-in sheets – Meeting logs Do NOT Submit Brochures from TECC!

Evaluating your SMART Plan Caroline Kurtz, Ph. D.

Common problems l l l Not including an evaluator in plan Objective not written very well Evaluation not corresponding to objective or intervention Evaluation portions of OTIS not completed correctly Evaluation Activity written in Intervention Plan Not enough time spent consulting evaluator l l Lack of detail or specifics Evaluator does not examine OTIS Evaluation guide

Common problems l Disagreement between evaluator and project staff l l l project staff should NOT just turn over evaluation to consultant -- give/take and empowerment Not using the OTIS Evaluation guide Good evaluators are hard to find

Evaluation resources l Resource documents l Tell Your Story: Guidelines for Writing a High Quality Final Evaluation Report l Tips and Tools l OTIS Evaluation Guide l TC Evaluation Center l www. tobaccoeval. ucdavis. edu

Why is Evaluation important?

Evaluation is important because it can… Improve your intervention l Manage your resources effectively l Show your program’s success l Demonstrate accountability l

Evaluation requirements 10% Solution l Evaluation plan for each objective in plan l Designate a primary objective where a more in-depth evaluation plan is created l

Primary objectives Primary objective requires the bulk of your evaluation resources l Requires a final evaluation report l

Create your Evaluation Plan based on your objective

Evaluation Planning Process At-a-Glance If your plan type is: Legislated/Voluntary Policy or Legislated Resolution Policy or Resolution adoption the type of data collection will be: process policy implementation outcome Individual Behavior Change Other with Measurable Outcome Other without Measurable Outcome policy adoption and implementation process and outcome Outcome (can do Process data collection too) process

Process vs. Outcome Evaluation l Process evaluation l l l Monitors the progress of the intervention and helps to identify changes needed to improve service delivery Essential with objectives without a quantifiable outcome (e. g. policy adoption) Outcome evaluation l Assesses the extent to which an intervention affects participants (e. g. extent a policy was implemented) l Indicates changes in knowledge, attitudes, awareness or behavior that you expect to occur as a result of your intervention

Create your Evaluation Activity Plan

Outcome Evaluation: What outcome is being measured? Link to the objective! l If the objective is: By June 30, 2010, at least 65% of 120 restaurants in Gourmet County with an outdoor dining space will prohibit smoking in these areas. l The outcome is: The percentage of restaurants who prohibit smoking in their outdoor dining spaces.

Outcome data collection Study Design l Experimental l Quasi-experimental l Non-experimental Data Collection Instrument l Description l Instrument design Data Collection Process l Sample size, sampling, data collection method, data source,

Study Design Types l Experimental l Think POWER! l Involves at least one control group and one intervention group l Random assignment required l Can generalize results beyond your specific intervention l Allows you to begin attributing your efforts

Study Design Types l Quasi-Experimental l Requires a control group or multiple measurements (comparison over time and comparison with other groups) l Does NOT require random assignment to groups l May introduce bias

Study Design Types l Non-Experimental l Does not incorporate control groups, multiple measurements, or random assignment l Least likely to reflect or demonstrate the success of a program l Useful where an experimental design is either not applicable or is too expensive

Process Data Collection: What do you want to know about the progress of your intervention? What do you want to know about what worked and what didn’t work with your intervention?

Process Data Collection Forms l l l l l Data Collection Training Education Participant Survey Focus Group Key Informant Interviews Media Activity Record Observation Data Policy Record Public Opinion Poll/Public Intercept Survey Youth Tobacco Purchase Survey Other

Process Data Collection in OTIS l Education Participant Survey l l Data Collection Training l l Used to measure attitudes, knowledge or behavior after participants attend a training, class or were targets of an educational intervention Used to evaluate the process of training your data collectors Key Informant Interviews l Used to ascertain information from community members who are knowledgeable about specific topics

Evaluation Reporting l Resources l Tell Your Story: Guidelines for Preparing a Complete High Quality Final Evaluation Report l Tips and Tools l Writing and organizing interview questions l Conducting the interviews l Focus group interviews l Qualitative methods l Content Analysis l Record Reviews

Evaluation Reporting What type of analysis will be done? l How will the results be disseminated? l What if any limitations or challenges to do you foresee with the evaluation? l

Common questions What if my evaluation shows that the intervention was unsuccessful? l My evaluator says I can’t do this evaluation, can I? l

CTCP does not penalize grantees for not achieving the objective l CTCP looks to make sure the interventions/activities planned to achieve the objective were conducted l

The Four Standards There is no one “right” evaluation. l l Utility: Who needs the info from this evaluation and what info do they need? Feasibility: How much money, time, and effort can we put into this? Propriety: Who needs to be involved in the evaluation to be ethical? Accuracy: What design will lead to accurate information?

How do you know if your plan can get the job done without being overkill? l Ask yourself: l l l l Will this information be useful during the project? Next time? Does it address SMART? Will the data we plan to collect tell us if we will meet the objective? Work with your evaluator Consult with the TC Evaluation Center Read other final evaluation reports Use your previous experience

SMART Budget Justification

SMART Budget Justification Today we will review: Ø Most frequent reviewer comments. Ø Reviewer comments by budget category. Ø Tips for a SMART Budget Justification

SMART Budget Justification Top 5 Most Frequent Reviewer Comments and Topics 1) 2) 3) 4) 5) RFA Instructions Budget and SOW Activities Budget Narratives and Justifications Formula Calculations Personnel and Consultant Rates

SMART Budget Justification Budget Category Line Items 1. 2. 3. 4. 5. 6. 7. 8. Personnel Costs Fringe Benefits Operating Expenses Equipment Travel/Per Diem and Training Subcontracts and Consultants Other Costs Indirect Costs

SMART Budget Justification Personnel Costs • Salary formula calculations are incorrect or missing. ü Include salary range, percent of time, and pay period frequency. • Missing information on the lead staff responsible for overseeing evaluation activities. ü Identify the lead staff responsible for 5% FTE oversight or equivalent 4 hours/week. • Provide narrative descriptions for each position. ü Include a solid justification in the Duties Description. • Position titles should be consistent with the SOW Responsible Parties fields. ü Take a few moments to check the titles in the budget justification and SOW. • Personnel, subcontract, and consultant costs were budgeted at higher rates than Comparable State Civil Service classification levels. ü Provide sufficient justification.

Example of Personnel Cost • Title: Project Director (From the Responsible Party’s field in the SOW) Salary Calculation (i. e. , Formula): ($2, 875 -$3, 082 semi-monthly) x (100%) x 24 pay period per year Duties Description: Responsible for the overall planning, development and completion of project goals, objectives and SOW deliverables. Supervise agency staff, project consultants and subcontractors. Responsible for the fiscal accountability of project budget. Responsible for the general coordination of the day-to-day operations and reporting progress at scheduled intervals to CDPH/CTCP. Maintains liaison with CDPH/CTCP Program Consultant and Procurement Manager. Responsible for the budget, budget revisions, invoices, staff changes, consultant contracts, and ensures timely progress on contract obligations, and other duties as required. Designated lead and devotes 5% FTE of effort (or equivalent to 4 hours/week to oversee the implementation of project evaluation and supervision of the evaluation consultant.

SMART Budget Justification Fringe Benefits • Should be consistent with State policies. ü Refer to the Contract Documents section on TCFOR. • Should be adequately justified. ü Provide detail in the narrative-see following slide. • For additional information for this RFA: Tobacco Control Funding Opportunities and Resource (TCFOR), Contract Documents » Exhibit D(S), Provision 20 TCFOR Application Information » Section III, Application/Proposal Requirements and Instructions » Page 22

Example of Fringe Benefits • At approximately 20% - 40% of Total Personnel Costs, Fringe Benefits includes the following: Fringe Benefits are paid to eligible employees according to state and agency policies. They include FICA, FUTA, State Unemployment Insurance, Long Term Disability Insurance premiums, Worker's Compensation Insurance premiums, Health, Dental, Vision, Life (approximately 25%) and Pension (approximately 10%). Vacation and other leave are not expensed in fringe benefits.

SMART Budget Justification Operating Expenses: • Formulas are incorrect or missing. ü Show budgets were calculated. • Justifications are missing or inadequate. ü Provide detail for the justification to describe how these will support your proposed project. • Items need to be consistent with the SOW. ü Show the connection to the SOW. • For more information refer to the RFA: – TCFOR, Application Information » Section III, Application/Proposal Requirements and Instructions » Pages 22 -27

Example of Operating Expense For Space Rent/Lease • Title: Space Rent/Lease Description (Formula): FY 2010/11: July 2010 -June 2011: 3. 0 FTE x 150 sq. ft. + 100 sq. ft. (common area) x $1. 00/sq. ft. x 12 months = $6, 600.

SMART Budget Justification Equipment • Equipment budgets were not consistent with the applicant capability narrative. ü Take the time to check for consistency throughout your application. • For additional information refer to the RFA: – TCFOR, Application Information » Appendix: Computer Hardware/Software Minimum Specifications

SMART Budget Justification Subcontracts and Consultants • Formulas were missing or incorrect. ü Enter a formula to show the budget was calculated. • Consultant rates were high. ü Rate paid to consultants should be commensurate with the consultant’s education & experience and negotiated at the lowest possible cost. • Role of consultants were missing or incomplete. ü Provide more detail for a description of duties. • Did not see these titles in the SOW ü Provide consistency with what is listed in the Budget Justification and the SOW Responsible Parties fields.

SMART Budget Justification Travel/Per Diem and Training: • Travel should be budgeted in the correct subcategories. ü • Required Travel items were listed in the incorrect fiscal years. ü • Provide these to show the budget was calculated. Justification detail was missing or incomplete. ü • Check the RFA instructions, Section II, Work Statement. Formulas were missing or incorrect. ü • Read the RFA instructions and determine the appropriate subcategories for your budget. Show the travel supports the project. Did not match up with SOW activities. ü Show this connects with the SOW.

Example of Travel Expense • • Title: Project Travel/Training Description: Includes travel to local and regional coalition meetings, community organizing and campaign trainings and on-site technical assistance; participation in CDPH/CTCP statewide workgroup meetings and sponsored trainings; and, other local, regional or statewide stakeholder meetings. All in-state travel is approved by the Project Director. Travel is reimbursed according to Department of Personnel Administration policies; which includes airfare, meals, lodging, incidental expenses and mileage @ $0. 50 per mile. At approx. $500/month x 12 months = $6, 000 x 3 years = $18, 000 • For additional information refer to the RFA: – TCFOR, Application Information » Section II, Work Statement, page 16 » Section III, Application/Proposal Requirements and Instructions, » Pages 28 -30

SMART Budget Justification Other Costs: Subcategories: Educational Materials Advocacy Campaign Materials Behavior Modification Materials Media Sponsorships Other Agency-Defined Subcategories • Lacks detail in description. ü Provide a narrative to explain how this supports and connects with the SOW. • Use correct subcategories. ü Can review the definitions in the RFA Instructions, Section III.

SMART Budget Justification Indirect Expenses: • Narrative lacked detail. ü Provide a justification for each indirect cost item. • Personnel costs were listed here and under the Personnel Costs category. ü Review the RFA Instructions, Section III, for guidelines on what is allowed. • Indirect Costs were miscalculated. ü Indirect expenses are calculated at a maximum of 25% (twenty-five percent) of the total personnel expenses (salary expenses plus fringe benefits). • Audit costs listed in Operating Expense plus Indirect Costs calculation exceeded the maximum amount. ü The total of audit costs plus indirect costs cannot exceed the maximum percentage rate of 25%. Review the RFA Instructions, Section III, for guidelines on this. • For additional information regarding allowable expenses for this line item refer to the RFA: – TCFOR, Application Information – Section III, Application/Proposal Requirement and Instructions, page 36

Tips for a SMART Budget Justification

Tip #1 Carefully read and review all of the instructions in the RFA (see TCFOR, Application Information) Reference both of these RFA documents together: – Work Statement (TCFOR, Application Information, Section II). • Scoring criteria and specific requirements unique to this RFA. – Application/Proposal Requirements and Instructions (TCFOR, Application Information, Section III). • General requirements • Application/proposal submission and instruction

Tip #1, Continued Application Information

Tip #1, Continued TCFOR, Application Information

Tip #2 • Always be clear and concise. • Provide sufficient detail and justify proposed budget items. • Propose cost effective operating expenses that will directly support staff and satisfactorily complete all SOW activities.

Tip #3 • Budget justification supports expenses connected with the SOW. • Check mathematical formulas. • Proof your work.

Summary n Clarity is Everything – n n n Be concise Interesting Persuasive & Unique n Demonstrate project logic and outcome, impact of funds. n Be specific about goals, measurable objectives, and quantifiable outcomes n Follow exact RFA/P Instructions n Don’t forget to Deliver!