Process Evaluation of MISP Implementation in Kathmandu and

Process Evaluation of MISP Implementation in Kathmandu and Sindhupalchok, Nepal Training: Focus Group Discussion (FGD) September 10 -12, 2015 Samira Sami, MPH Dr. PH Candidate Johns Hopkins University Anna Myers, MPH Research Manager Women’s Refugee Commission Research. Rethink. Resolve. 1

Research. Rethink. Resolve. Agenda • Day 1: Training – Introductions – Staff roles and responsibilities – Overview of the MISP – Overview of the evaluation methodology – Ethical issues and safety protocols – Overview of focus group discussions (FGDs)

Research. Rethink. Resolve. Agenda (2) • Day 2: Training – Introduction to FGD Tools – Practice, practice • Day 3: Pilot – Conduct one male and female FGD – Practice translation/transcription – Debrief

Research. Rethink. Resolve. Introductions • • What is your name? Where are you from? What are your favorite activities to do? What is your research experience? – Experience in qualitative research – Experience in reproductive health research or other work

Research. Rethink. Resolve. Staff Roles and Responsibilities

Research. Rethink. Resolve. Structure of Evaluation Team § Site 1: Kathmandu § 1 Male Facilitator and 2 Male Notetakers § 1 Female Facilitator and 2 Female Notetakers § Site 2: Sindhupalchok § 1 Male Facilitator and 2 Male Notetakers § 1 Female Facilitator and 2 Female Notetakers

Research. Rethink. Resolve. Roles/Responsibilities of Teams § Roles § Facilitator § Notetakers § Interpreter § Responsibilities § Timeliness § Respect § Be prepared § Team player § Ask questions!

Research. Rethink. Resolve. Timeline of Activities Sunday Monday Tuesday Wednesday Thursday Friday Saturday 6 7 8 9 10 Training 11 Training 12 Pilot 13 14 15 Data Collection Data Day #1 Collection Day #2 16 Data Collection Day #3 17 Data Collection Day #4 18 Data Collection Day #5 19 Debrief Day 20 21 Debrief with stakeholders 23 24 25 26 22

Timeline 9

Timeline 10

Research. Rethink. Resolve. Communication Protocol § Security Issues § Administration § Hours and lateness § Housekeeping issues

Research. Rethink. Resolve. Planning FGD Schedule and Participant Recruitment § List of health workers § How to recruit participants and informed consent § Selecting the right environment/setting § Scheduling dates/times for your FGDs

Research. Rethink. Resolve. QUESTIONS? 13

Research. Rethink. Resolve. Overview of the MISP

Research. Rethink. Resolve. Reproductive Health in Crisis Settings • WHY do we need to pay attention to RH in a crisis? • 60% of preventable maternal deaths & 53% of child deaths occur during crisis • 4% of population pregnant at any time: – 15% of pregnant women will need emergency obstetric care – 9 -15% of newborns will need live-saving emergency care • Consequences of crisis can elevate mortality and sexual violence

Research. Rethink. Resolve. Reproductive Health in Crisis Settings (2) • How does crisis worsen reproductive health? – – – Destruction of health facilities Lack of trained staff Health facilities overwhelmed Breakdown of social structures, families Lack of supplies No access (displacement, security, cultural restrictions, knowledge/attitudes towards HC, economics) – Targeted and opportunistic sexual violence – Sexual exploitation

Research. Rethink. Resolve. What is the Minimum Initial Service Package (MISP) for Reproductive Health? • Minimum: – Ensure basic, limited reproductive health services • Initial: – For use in emergencies, without site-specific needs assessment • Service: – Health care for the population • Package: – Activities, supplies, coordination and planning

Research. Rethink. Resolve.

Research. Rethink. Resolve. Objective 1: Coordinating implementation of MISP priority RH services Ensure the health sector/cluster identifies an organization to lead implementation of the MISP.

Research. Rethink. Resolve. Objective 2: Preventing and managing the consequences of sexual violence • Putting in place measures to protect affected populations from sexual violence • Making clinical care available for survivors of rape • Ensuring the community is aware of the available clinical services

Research. Rethink. Resolve. Objective 3: Reducing the spread of HIV • Ensuring safe and rational blood transfusions • Enforcing respect for standard precautions • Guaranteeing the availability of free condoms

Research. Rethink. Resolve. Objective 4: Preventing maternal and newborn death and disability • Ensuring availability and accessibility of Em. OC and newborn care services at health facilities and referral hospitals • Establishing a referral system • Providing clean delivery kits

Research. Rethink. Resolve. Objective 5: Planning for Comprehensive RH Services • Collecting existing background data • Identifying suitable sites for future RH service delivery • Assessing staff capacity • Ordering equipment and supplies

Research. Rethink. Resolve. Additional MISP Priorities • Family Planning • Sexually Transmitted Infections • Prevention of Mother-to-Child Transmission of HIV/AIDS

Research. Rethink. Resolve. QUESTIONS? 25

Research. Rethink. Resolve. Overview of Evaluation

Research. Rethink. Resolve.

Research. Rethink. Resolve. Evaluation Methodology • Case study design – Kathmandu – Sindhupalchok • Purposive sampling • Population – Program staff – Health workers – Community • Methods: – Key informant interviews – Health facility assessments – Focus group discussions Method Sampling plan Data collection strategy Key informant interviews Program managers and staff • Semi-structured interviews Health facility assessments 25 health facilities • Structured questionnaire Focus group discussions 32 focus groups (male and female) • Open ended questions

Research. Rethink. Resolve. Evaluation of the Implementation of the MISP for RH among Crisis-Affected Persons in Kathmandu and Sindhupalchok, Nepal • Purpose of FGDs – To qualitatively assess the extent that the MISP has been implemented in Kathmandu and Sindhupalchok since the earthquake • FGD Objectives 1. To assess the extent that RH services are available and accessible 2. To explore how resources (human and material) relevant to the MISP were allocated, mobilized, and delivered by agencies in the humanitarian response 3. To explore awareness about and use of RH services by the crisisaffected population 4. To explore the factors that influence the implementation of the MISP

Research. Rethink. Resolve. Focus Group Discussions • Purpose: To understand the knowledge, attitudes, and access of RH services among the crisis-affected population • Group composition: no more than 10 participants per group Site 1 – Kathmandu Site 2 – Sindhupalchowk TOTAL No. of FGDs near health far from near health facilities No. of FGDs far from across all health sites facilities 2 2 8 Boys aged 15 -24 years 2 2 8 Women aged 25 -49 years Men aged 25 -49 years 2 2 2 2 8 32 Age-Gender Group Girls aged 15 -24 years

Research. Rethink. Resolve. Evaluation Collaborators § Evaluation Team § Women’s Refugee Commission § Boston University School of Public Health § Johns Hopkins Bloomberg School of Public Health § RIDA research staff § Evaluation Partners § Department of Health (DOH) Nepal § United Nations Population Fund (UNFPA) Nepal § FPAN - SPRINT (Sexual and Reproductive Health Programme in Crisis and Post-Crisis Situations) § Reproductive Health Sub-Cluster in Nepal

Research. Rethink. Resolve. Evaluation Outcomes • Share findings with evaluation partners to inform the development of program and policy recommendations • Improve reproductive health services for people affected by the earthquake in Nepal • Disseminate final report for local and international learning • Inform future emergencies by ensuring quality RH services are provided to affected populations in similar contexts

Research. Rethink. Resolve. QUESTIONS? 33

Research. Rethink. Resolve. Lunch!

Research. Rethink. Resolve. Ethical Issues and Safety Protocols

Research. Rethink. Resolve. Principles of Ethical Research • Respect integrity & minimize harm • Informed consent • Confidentiality • Privacy • Secure handling of data • Sharing results with relevant stakeholders • Understand the referral pathway for gender based violence

Research. Rethink. Resolve. Elements of Consent • * If non-literate, read slowly in appropriate language and ask for questions • Introduction • • • Purpose of Study Procedures Risks & benefits Questions or Concerns Confidentiality Voluntary Participation

Research. Rethink. Resolve. Practice: Informed Consent • Read out loud • Identify consent elements: – Purpose of Study – Procedures – Risks and benefits – Questions or concerns – Confidentiality – Voluntary Participation

Research. Rethink. Resolve. Eligible FGD Participants • 18 and older → regular consent • 15 to 17 years: must meet one of these – Ever Married – Pregnant – Parent of a child

Research. Rethink. Resolve. Referral Pathway for Gender Based Violence • See Handout Research. Rethink. Resolve.

Research. Rethink. Resolve. Overview of Focus Group Discussions

Quantitative vs. Qualitative Research • Qualitative: • Attempts to explain • Complex, changing (‘fluid’) • Describes meanings: • Perceptions, knowledge, beliefs, behaviors • Describe variation • Behaviors explained in context of what is happening around them • Quantitative: • Attempts to measure • Often can quantify an event but NOT explain why it occurred • Standardized so that larger generalizations can be made 42

Quantitative 43

Qualitative What happens in a house, how is a “home” defined, what influences what people do in their houses? 44

Research. Rethink. Resolve. What is qualitative research? • • • Qualitative research is flexible. Non-leading questions and statements are used. Information is probed as much as possible. Participants are treated as the expert. The participants lead the conversation while staying on topic. • How and why things happen are of most interest. • The words people use are analyzed. Research. Rethink. Resolve.

No ‘right’ or ‘wrong’ – we are trying to understand perceptions 46

Research. Rethink. Resolve. What is a focus group discussion? • A group interview that emphasizes communication among group members in order to generate information about a chosen topic

Research. Rethink. Resolve. Why Do Focus Groups? • To collect information about knowledge, behaviors, what people think, how they think and why they think that way • To discover variety within a homogenous population • To assess areas of agreement and disagreement between groups • To develop a consensus view on a product • To empower participants, to hear their voices * FGDs: NOT a way to interview a number of people at once

Roles in a FGD Research. Rethink. Resolve. • Facilitator - conducts the group interview – Must be able to ask and clarify a question – Hear the response – Determine if further clarity is needed regarding the responses • Notetaker- records what is being said or expressed – – – Responsible for ensuring detailed documentation of the FG Back-up to recording device Captures quotes and who is speaking Retain specific terms or phrases in local language Record body language and nonverbal signals of participants Remind facilitator of issues overlooked during the discussion • Both positions need to work together

THE FACILITATOR Research. Rethink. Resolve. 50

Research. Rethink. Resolve. What are characteristics of good facilitators? • • • Language and communication skills. Familiarity with the content and structure of the activity. Familiarity and comfort level with discussing reproductive health topics. Ability to respect the dignity and confidentiality of participants. Previous experience conducting focus groups or other qualitative activities. Ability to multi-task and be flexible with adjusting questions as needed. Research. Rethink. Resolve.

Research. Rethink. Resolve. Steps for Conducting a FGD 1. Prepare materials for FGD 2. Greet participants 3. Introduce yourself and the team – Explain purpose of interview – Describe link to local partners 4. Conduct informed consent – – Cannot promise other participants of the FGD will not share Encourage FGD participants not to reveal identities Avoid using participant’s names during the FGD If participants refuse to participate, thank them for their time and they can be excused from the group at any time 5. Give a few brief group rules – One speaker at a time – No side conversations – Everyone participating, no one dominating

Research. Rethink. Resolve. How to Facilitate a FGD (2) 6. Move to first specific topic – “Thank you. Let’s start with our first topic. ” – “One thing I’ve heard several people mention is _____. What else can you say about that? ” or “I’m surprised no one has mentioned ____. Does it matter or not? ” – Track connection between conversation and guide. 7. When closing, provide a summary of the discussion – Summarize key points of discussion with groups to make sure important points have been captured. – Ask if there’s anything else to add – Don’t give out cookies/drinks until the end. – Share the locations of existing services for participants to access additional information and services. 8. Prepare for debrief with research team

Research. Rethink. Resolve. What is probing? • • A way to encourage participants to say more and explain why or how to get as much information as possible. Types of probes are: – Why or what questions: “What do you mean when you say…” – Silent probe: Staying silent but supporting the participant to continue talking. – Echo probe: Repeating the last point raised by the participant. “I see. You said that people might talk to friends if they have a problem. Tell me more about that. What happens then? ” Be careful not to repeat something different than what was said, as this would be leading. – “Uh huh” probe : “oh, ” “Uh-huh, ” “I see”. • Other examples: – – – “Can you explain further? ” “Can you give me an example? ” “Can you say more? ” “I am not sure I understand x. Would you explain that to me? ” “How do you feel about…” “Is there anything else? ” Research. Rethink. Resolve.

Research. Rethink. Resolve. What are good ways to probe? • • “What do you mean when you say…” “Can you explain further? ” “Can you give me an example? ” “Can you say more? ” “I am not sure I understand x. Would you explain that to me? ” “How do you feel about…” “Is there anything else? ” Research. Rethink. Resolve.

Research. Rethink. Resolve. What is a leading question? • • • Leading questions are said in a way that suggests a particular answer or implies that one answer is expected or more correct. If leading questions are used, participants might be more likely to agree with what we ask or think we are looking for a particular answer. Participants are less likely to say what they really think. Research. Rethink. Resolve.

Research. Rethink. Resolve. What is a leading question? Leading questions: • “What fears do you have when your baby’s diarrhea does not stop? ” • “Why didn’t you take your baby to the health center for treatment? ” • “How good was the treatment at the health center? ” Non-leading questions: • “What do you think about when your baby’s diarrhea does not stop? ” • “What do you do when her/his diarrhea does not stop? ” • “How do you feel about the treatment your baby got at the health center? ” Research. Rethink. Resolve.

TROUBLESHOOTING Research. Rethink. Resolve. 58

Research. Rethink. Resolve. How should a facilitator facilitate? • • Listen to participants and concentrate on what they are saying. Ask questions one at a time. Show interest and understanding. Encourage participants to feel comfortable sharing as much information as possible. Remind participants to share information about the general situation and not their personal situation. Listen for causes and characteristics of an issue. Be silent when silence is needed. Research. Rethink. Resolve.

Research. Rethink. Resolve. What are things the facilitator should avoid? • • • Nodding your head, smiling, frowning or using other body movement that can show a biased or judgmental attitude. Giving answers to the questions and/or agreeing verbally with the answers. Being impatient. Judging the answers (e. g. , saying, “It is good”). Cutting off the participants’ answers in the middle of their sentences. Letting two or more participants talk at the same time. Sharing your personal opinions. Arguing with participants. Losing control of the discussion to participants that dominate the conversation. Skipping any participant during the discussions. Research. Rethink. Resolve.

Research. Rethink. Resolve. What are examples of participants you may need to manage? Challenge Suggested Response Talkative Thank them for their contribution Invite others to comment Encourage them to make one point at a time Asking lots of questions Write question on flip chart and return to them Prioritize which questions to focus on Tired Take a break Anger Acknowledge emotion or sensitivity Steer towards sensitive issue rather than person Shy Encourage to contribute by calling pseudo name Ask if agrees/disagrees with what has been shared Personalities Use pleasant and patient tone Add humor, where appropriate Full agreement Jot down your thoughts first, now let’s discuss Research. Rethink. Resolve.

Research. Rethink. Resolve. How can different participants be managed? • Participants that are “the expert, ” too talkative, rambling or politically inclined: o Use verbal and non-verbal communication, such as body motion and eye contact. o Repeat the topic question so that the participant understands the topic. • Participants that are scared, shy or confused: o Give more attention and encouragement through using eye contact and by being patient. Research. Rethink. Resolve.

THE NOTETAKER Research. Rethink. Resolve. 63

Research. Rethink. Resolve. What are characteristics of good notetakers? • • Excellent listening skills with the ability to listen while writing notes in an format without slowing the tempo of the discussion unnecessarily. Able to write verbal and non-verbal responses Ability to listen without making comments Ability to respect the dignity and confidentiality of respondents. Ability to multi-task and recognize when the facilitator has altered the sequence of the questions, which requires a strong familiarity with the study tool. Date must be recorded in a format that is readable, understandable, and complete Ability to concentrate and write notes for a period of up to 2 hours in length. Research. Rethink. Resolve.

Research. Rethink. Resolve. Expectations of the Notetakers • • Assign speaker identifiers and note who says what Retain specific terms or phrases in local language Record body language and nonverbal signals of participants Remind facilitator of issues overlooked during the discussion Research. Rethink. Resolve.

Research. Rethink. Resolve. How to Record a FGD 1. 2. 3. 4. 5. 6. 7. Prepare the audio recorder and environment for the discussions Document the ‘FGD Details’ on the guide. Obtain informed consent before recording the discussion Record participant identifiers Make sure you clearly indicate in your notes how you are recording nonverbal behavior and your own comments Record additional questions/probes by the facilitator Indicate symbols for facilitator – Place marks in margins that so that it is easily seen when you are scanning the notes for completeness – * could signify that a sentence or answer needed more clarity 8. 9. 10. 11. Speak up if pace of FGD is too fast for recording Coordinate with your team before closing the discussion Complete the ‘FGD Participant Profile’ before participants leave Stop and check the recording

Research. Rethink. Resolve. Recording Nonverbal Behavior • Guidelines: – First, does it add to the interpretation of the FGD? – If yes, • Write a short description of the behavior • Provide some interpretation about what it means

Research. Rethink. Resolve. Example: Nonverbal Behavior • “You know, we have hired assistants to work on projects at [Organization redacted] and some of our projects that I'm directly involved in-- some are involved as staff, but yeah. So, um, because, students like to do graduate assistant work and it, ah (smiling and rubbing fingers together to indicate money) get a little cash on the side. ”

Research. Rethink. Resolve. Tracking Participant Responses • • In your notebook, draw how participants are seated, and assign a number to each person to track who says what. If participants will move around, simply assign a number that you can remember for each person. 3 2 1 Facilitator Notetaker 4 5 6 Research. Rethink. Resolve.

Research. Rethink. Resolve. Real life example Question: What would you say are the barriers or challenges for women to begin using contraceptives? [2]: I know how to use methods (just pills), but there are challenges, such as lack of awareness among both women and men. Husbands prevent wives from using the pill. If I want to use it, I use it in a secret way. [4]: Using the pill will prevent women from having many children. Some people produce ten children and some lose three. They remain with seven children. If they take the pill, they may only have a few children, and if they die, that will be bad luck. If we produce more than 12 children, if six die, we still remain with six. [7]: Some forget to take the pill because of too much work. By the end of the day, the mind is too tired and we cannot remember to take the pill. Research. Rethink. Resolve.

Research. Rethink. Resolve. Example continued [1]: Fear of our husbands is another issue. [3]: Small girls fear their parents and families. There is a lack of awareness and incorrect information. There also no CBD [community-based distribution] agents in their area. The husband also opposes. [Facilitator]: Is there anything else? Can you think of other reasons? [5]: We need more information on how to be aware of the rest of the methods. For condoms, we do not know how to use it. Even men cannot accept. We can only use injections or pills. Many nod their heads in agreement. Research. Rethink. Resolve.

DEBRIEFING Research. Rethink. Resolve. 72

Research. Rethink. Resolve. What is a debriefing session? • Immediately following an activity, the facilitator and notetaker should come together to discuss the session and address areas of improvement. 1. Facilitator and notetaker review responses by having the notetaker verbally review each response that she or he has written for each question. 2. Facilitator can add any missed information. 3. Facilitator and notetaker discuss the responses for any noteworthy thoughts. Research. Rethink. Resolve.

Research. Rethink. Resolve. Debriefing at the End of the Day • Make sure you understand the definition of terms or phrases used in the local language: – Get agreement from your team – Keep a record of word in original language • Debriefing is the start of analysis: you look at the data and make sense of it • When possible, return to key persons and ask questions about things you do not understand

TRANSCRIPTION Research. Rethink. Resolve. 75

Research. Rethink. Resolve. Expectations of the Transcriber • Transcribe the FGD word for word (include every word as spoken), include non-verbal observations • Include facilitator’s questions and comments • Include pauses, interruptions, laughter, silences, and other unspoken dynamics in the transcript (use notes) • Include speaker identifiers where appropriate

Research. Rethink. Resolve. What happens during transcription? • Once the data collection is finished, the research team will share and discuss their notes. • One note taker will merge the two sets of notes and translate and type one transcript. • In the process: • Compare the transcript to the handwritten notes to match speakers and add any missing information. • Add any non-verbal observations that were noted by hand so that the transcripts ultimately contain all of the information. • Refrain from adding any identifying information (such as names of people) Research. Rethink. Resolve.

Research. Rethink. Resolve. Transcription Keys Key • [Text] Indicates speech by the person who was not the primary speaker at the moment, i. e. “yeah”, “Mmhmm”, or other short non-substantive affirmations, probes or interjections. • [Text] Description of body language or other environmental context. • … Indicates a pause. Length of pause indicated by number of dots: … vs. … … • Indicates that the sentence was cut short by an interjection by the other speaker. • — Indicates a quick transition in ideas, consistent with its typical grammatical use. • ? Indicates that the sentence was spoken with a questioning tone, consistent with its typical grammatical use. • , Indicates a natural transition pause while communicating a continuous thought, consistent with its typical grammatical use. • . Indicates the natural end of one thought, followed by the beginning of a new one or speech by the other speaker, consistent with its typical grammatical use. • “____” Indicates that the speaker is imitating the speech of a third party who is not present, or is in fact quoting a third party who is not present, consistent with its typical grammatical use. • XXXX Content redacted in order to de-identify the transcript.

Research. Rethink. Resolve. Important information about transcriptions: • Transcriptions have to be written word for word. • Summaries, missing pauses, shortening rambling statements, rephrasing, etc. , cannot be done. • Everything that is said, along with actions and gestures of importance, must be included in your transcript. Research. Rethink. Reslve.

Research. Rethink. Resolve. Introduce FGD Tools

Research. Rethink. Resolve. Wrap Up (End Day 1)

DAY 2 82

DEBRIEF FROM DAY 1 83

Research. Rethink. Resolve. Guiding principles • Speak slowly and clearly • Repeat when needed • Be courteous and attentive – Do not yawn – Take a break if they want – Start with a friendly, welcoming atmosphere

Research. Rethink. Resolve. Guiding principles (2) • Be neutral and accepting – Do not use judgmental language • Listen carefully to the answers – Do not assume you know the answer

Research. Rethink. Resolve. Guiding principles (3) • Establish and maintain boundaries – Let the respondent know you are listening but also focused on completing the interview – Listen – Maintain neutral role • Pause if needed • Stay focused on the respondent – Avoid personal discussions about yourself

Research. Rethink. Resolve. Guiding principles (4) • Treat the respondent with respect • Do not act as a counselor – We are not social workers – If respondent is concerned, provide a referral to services – If you cannot handle a situation, discuss with supervisor

REVIEW TOOLS 88

Research. Rethink. Resolve. Practicing with examples… • Facilitators o Practice obtaining informed consent. o Practice facilitating a short discussion or activity with participants. o Practice closing the discussion or group activity. • Notetakers o Practice taking notes and documenting observations. Research. Rethink. Resolve.

Research. Rethink. Resolve. Practicing with examples… • Participants o Play different types of participants, including those with different impairments, the talkative and quiet. • Debrief o Notetaker reviews group proceedings. o Participants to point out any missed information and offer suggestions to facilitators on ensuring inclusion and maximum participation. • Change roles Research. Rethink. Resolve.

Observe the process and answer the following questions: Does the interviewer introduce her/himself? Does the interviewer ask for informed consent? Does the interviewer try to make the interviewee comfortable? (i. e. , good eye contact, nodding of head, smiling, encouraging behaviors) Does the interviewer tell the interviewee what to expect and how long the interview might last? 91

Research. Rethink. Resolve. Practice Discussion • What went well? • What difficulties did you encounter – Facilitator – Notetakers – Participants

Research. Rethink. Resolve. Wrap Up and Prepare for Pilot

Research. Rethink. Resolve. womensrefugeecommission. org 94