Скачать презентацию Evaluating Human Resources for Health Systems Strengthening Experiences Скачать презентацию Evaluating Human Resources for Health Systems Strengthening Experiences

552073fd42fa46e22e71f97dd35bc610.ppt

  • Количество слайдов: 46

Evaluating Human Resources for Health Systems Strengthening: Experiences from USAID’s Capacity Project Laura Gibney, Evaluating Human Resources for Health Systems Strengthening: Experiences from USAID’s Capacity Project Laura Gibney, Director of M&E and & Research, Intra. Health International, Inc. Linda Fogarty, Director of Results and Knowledge Management, Capacity Project (Jhpiego) Danny De. Vries, Monitoring, Evaluation & Research Manager, Capacity Project (Intra. Health)

Why the Global Focus on HRH? • Insufficient # of qualified health workers • Why the Global Focus on HRH? • Insufficient # of qualified health workers • Maldistribution of health workers • Inadequate support systems for HCWs • Low access to health services • Poor quality health services • Poor health outcomes

Health Impact of Provider Shortage • There is a direct relationship between the ratio Health Impact of Provider Shortage • There is a direct relationship between the ratio of health workers to population and survival of women during childbirth and children in early infancy. As the number of health workers declines, survival declines proportionately. WHO 2006

Global Health Workforce Shortage • 57 countries with critical shortages • • WHO estimated Global Health Workforce Shortage • 57 countries with critical shortages • • WHO estimated at least 4, 250 000 health workers needed to fill the gap (providers, managers) Worst shortage is in Sub-Saharan Africa 11% of the world’s population • 24% of global disease burden • 3% of the world’s health workers • 1% of world health expenditure. WHO 2006 •

Demands are increasing with HIV • Demand for care is increasing – • e. Demands are increasing with HIV • Demand for care is increasing – • e. g. , from 1988 -1992 in Kenya, the number of HIV+ patients in hospitals more than doubled Supply of health care providers is dwindling – e. g. , Botswana lost 17% of its health workforce to AIDS between 1999 and 2005. Will reach 40% by 2010. – Sick workers are less productive • • In an HIV+ worker’s final year of work, likely to miss on average every other day of work. Capacity to supply health providers is decreasing – Faculty/trainers are dying of AIDS, so there are fewer to train new providers

The Capacity Project • USAID-funded project led by Intra. Health International, with IMA World The Capacity Project • USAID-funded project led by Intra. Health International, with IMA World Health, Johns Hopkins, Liverpool Associates for Tropical Health, Management Sciences for Health, PATH and Training Resources Group. • Technical leadership in global human resources for health (HRH) • Worked in 25 countries, with a focus on: Ø Improving workforce planning and leadership Ø Developing better education and training programs Ø Strengthening systems to support workforce performance

Intra. Health International’s HRH Capacity Building Projects • Human Resources Development Capacity Project (5 Intra. Health International’s HRH Capacity Building Projects • Human Resources Development Capacity Project (5 year project. End date: 12/09) • Human Resources for Health and Quality Services (HRHQS) Project (5 year project. Start date: 10/09) • The Southern Africa Human Capacity Development Coalition (Botswana, Lesotho, Malawi, Namibia, and Swaziland) • HRH-related Associate Awards: Uganda, Tanzania, Namibia, Kenya, G-CAP (Guatemala, Panama, El Salvador, Costa Rica, Belize)

Challenges in Evaluating HRH System Strengthening • Relatively new area of assessment. No compendium Challenges in Evaluating HRH System Strengthening • Relatively new area of assessment. No compendium of standardized indicators. • Dearth of data needed as most countries do not have functional, populated HRIS systems. • Increasing expectation that we demonstrate impact of HRH system strengthening on health service delivery & health outcomes

Multipaper Session Presentations 1. Context and Evaluating Human Resources for Health Capacity Building (Danny Multipaper Session Presentations 1. Context and Evaluating Human Resources for Health Capacity Building (Danny De. Vries) 2. Indicators for Measuring Human Resources for Health Capacity Building (Linda Fogarty)

Context and Evaluating Human Resources for Health Capacity Building Danny de Vries, Monitoring, Evaluation Context and Evaluating Human Resources for Health Capacity Building Danny de Vries, Monitoring, Evaluation and Research Manager, Intra. Health International, Inc. Linda Fogarty, Director, Results and Knowledge Management, Jhpiego Erik Reavely, Monitoring and Evaluation Advisor, Intra. Health International, Inc. Elizabeth Bunch, Manager, Knowledge Sharing, Intra. Health International, Inc.

Questions • How does the context of HRH capacity building influence the technical approach? Questions • How does the context of HRH capacity building influence the technical approach? • What evaluation strategies have been suggested to incorporate context in evaluation of program success? • What indicators are suggested to incorporate context, if needed?

Context in Capacity Building 1. Traditional “boxed” system approach where context is seen as Context in Capacity Building 1. Traditional “boxed” system approach where context is seen as a boundary in a closed system (the “external”). 2. Relational approach that emphasizes an “enabling environment. ” 3. Ecological theory of complex adaptive systems, seeing “context” as an internal dynamic landscape embedding the program.

HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006) HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006)

Health Workforce Outcome Indicators • % health facilities with staffing levels that meet the Health Workforce Outcome Indicators • % health facilities with staffing levels that meet the requirements of the HRH plan • Numbers of patients: professional staff numbers • Bed utilization: number of staff • Vacancy rate • Turnover rate • Retention rate • Absenteeism • Rural/urban distribution • Staff satisfaction (based on attitude surveys) • % of total budget spent on staff salaries and allowances. From: Hornby P, Forte P. Human Resource Indicators and Health Service Performance. Human Resource Development Journal. 1997; 1(2): 103 -118.

Intermediate Action Field Indicators Suggested Indicators Action Field HR Management Systems HRM units in Intermediate Action Field Indicators Suggested Indicators Action Field HR Management Systems HRM units in place and strategically located at national and local levels HR information system in place and used for HRH planning at all levels Performance management system in place. HRH policies in place Appropriate scopes of practice defined for all cadres. Leadership Partnership Education Mechanisms in place for coordination (e. g. , other key ministries, donors, service providers including private sector providers, professional associations, etc. ) Agreements in place between MOH and other health providers Mechanisms in place to involve communities in health services. Policy Finance Evidence of high-level advocacy to promote the implementation of the HRH plan Leadership development program established for managers at all levels Involvement of wide range of stakeholders in policy and decision-making processes (professional associations, researchers, unions, private sector, FBOs). Pre-service education tied to health needs In-service training (e. g. , distance and blended, continuing education) Capacity of training institutions Training of community health workers and non-formal care providers. Salaries and allowances competitive in local labor market Salaries and allowances equitable between cadres of health workers National health accounts routinely collect data on HRH expenditure. HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006)

Context Indicators? HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006) Context Indicators? HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006)

Context Indicators? HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006) Context Indicators? HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006)

Context Indicators? HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006) Context Indicators? HRH Action Framework, developed (WHO 2006; Dal Poz et al. 2006)

Study: Role of context in HRH M&E • Taking advantage of technical staff experience Study: Role of context in HRH M&E • Taking advantage of technical staff experience at end of the project to inform next steps for M&E in HRH systems strengthening • A semi-structured, open-ended interview to document project’s history and technical evolution and explore implications for M&E. • 29 key informants: senior Capacity Project leadership and technical staff, USAID/OPRH stakeholders, and country directors. • One hour interviews transcribed analyzed using qualitative data software and inductive coding strategy.

Findings: Contexts of HRH Capacity Building • Historical-Cultural Situation • Political & Leadership Context Findings: Contexts of HRH Capacity Building • Historical-Cultural Situation • Political & Leadership Context • Individual growth of HRH Champions • Health Systemic Context (Action Fields, physical capital and consumables) • Multi-Sectoral Context • Global Context • Temporal Context

Historical & Cultural Situation • Understanding what actually drives change within that context and Historical & Cultural Situation • Understanding what actually drives change within that context and what sorts of enabling environment exists or not exist. … It’s something new in the capacity building world. I think it’s really important because if we are able to have sufficient understanding of what we are trying to change, then we are likely to engage in those contexts better. • Participation and Partnership of Key Stakeholders in order to Enlist and Incorporate Local Interests and Needs • I think that one of our key approaches has been really to involve stake holders from the very beginning and throughout to really try to understand what the needs of our in-country partners, especially the Ministry of Health, what their needs were before we move forward.

Political and Leadership Context • Working with Key Central Leaders to ensure buy-in, policy Political and Leadership Context • Working with Key Central Leaders to ensure buy-in, policy change, and sustained support for regional and system-wide holistic HRH interventions • It matters where you start. The reason is that the center will always be in charge overall … If you start from down, if the center does not approve or does not buy in, then it will be unsustainable. • And also the issue is that the center gives you a framework or a platform on which to build your system… you must understand the policies, you must form partnerships with key people that are interested in those things happening down there… • Well it depends I think on how autonomously the districts or health centers hospitals work from the central level. In some countries each hospital is almost its own entity and they can make their own decision and others you go to the central first you introduce your ideas

Individual Context HRH Champions • Value of Coaching and Mentoring HRH champions in the Individual Context HRH Champions • Value of Coaching and Mentoring HRH champions in the field • • And so as I understand it and as I’ve experienced it, it’s very iterative, so you know, you find out what people need to know and what they think of themselves as leaders and then you work with that. Benefit of utilizing internal and in-country secondments as part of an overall strategy of direct involvement and support

Health Systemic Context • Systems level scope demands technical approach has had to adjust Health Systemic Context • Systems level scope demands technical approach has had to adjust towards a comprehensive approach • • We realized with training is that you can do training really, really well, but if you don’t have these other things in the working environment for health workers to go back and to support them then you know, it’s just a waste of money pouring it into training. HRH Action Framework Fields • But there’s a bigger framework back there that says if you’re going to change something, you’ve got to be looking at all these different action areas, kind of what we’ve called action fields

Multi-sectoral context • Adopting a Cross-Cutting & Multi-Sectored Approach in order to close the Multi-sectoral context • Adopting a Cross-Cutting & Multi-Sectored Approach in order to close the gaps, get ministries and other organizations working together on interrelated issues • To deal with a problem like workplace violence and discrimination these are things that cross-cut sectors so we adopted a multi-sectoral approach to it. And you need to have intervention at the level of the health sector, but also at the policy level, ministries of labor and gender. You also need to have intervention at the individual service provider level and you need to have different kinds of interventions. So it’s best to take a multi-sectoral approach.

Global Context • Health worker migration as a global issue • USAID donor global Global Context • Health worker migration as a global issue • USAID donor global context - PEPFAR “bean counting” • Project has international list of partners • Global ethical problems health worker distribution issues, including brain-drain and post-colonial power dynamics. (number of trainings says nothing about quality of trainings)

Temporal Context • Long-term, strategic framework demands “strategic patience” • Complexity of stakeholder relationships Temporal Context • Long-term, strategic framework demands “strategic patience” • Complexity of stakeholder relationships • The time it takes for structural change in an organizational or core level • Resistance inherent to HR • Delay in expression of that change in service outcomes. • Novelty of CB itself: took time for stakeholder interest to develop.

Expanding measurement boundaries Global Political & Leadership Individual Temporal Other System Components Expanding measurement boundaries Global Political & Leadership Individual Temporal Other System Components

Cultural Contexts of HRH Capacity Building Contextual influence Historical-Cultural Situation Political Context Technical Response Cultural Contexts of HRH Capacity Building Contextual influence Historical-Cultural Situation Political Context Technical Response Evaluation/Indicator needs Situational assessments Partnerships & Stakeholder groups Secondments In-country offices Partnerships Strategic patience Adaptive management Historically informed baseline studies Participatory evaluation to negotiate meaning “success” Measure quality of partnerships. Measure cultural sensitivity of technical approach, including adaptation of systems and products to fit local culture Evidence of ownership & commitment o Changes in discourse (adoption of terms) o Government actions taken such as policy changes o Integration HRH in routine government operations Measure quality of partnerships Illustrate changes in relationships and coordination mechanisms Measures of transparency & corruption

Cultural Contexts of HRH Capacity Building Contextual Individual context of HRH champions and key Cultural Contexts of HRH Capacity Building Contextual Individual context of HRH champions and key leaders Health Systemic & Non-Health Systemic (multisector) Contexts Technical Response Coaching Mentoring Training Evaluation/Indicator needs Holistic, systems thinking Adaptive management Measures of coaching, mentoring, training quality Evidence of Ownership o Champions take the lead o Changes in discourse Professionalization o Levels of advocacy o Operate HRH management functions without technical assistance o Increase in HRH system thinking o Increase in data driven decision making Measures of innovation Multiplier effects (TOT, ripple effects) Mixed-methods for data collection Measures of systemic resilience (crisis response) o Functional redundancy, diversity, reversibility o Level of internal learning through self-organization, innovation, etc. o Levels of internal communication & fragmentation of information Buy-in in program by other funders Acknowledge multi-causality

Cultural Contexts of HRH Capacity Building Contextual influence Technical Response Global Context Global partnerships Cultural Contexts of HRH Capacity Building Contextual influence Technical Response Global Context Global partnerships Managing expectations Evaluation/Indicator needs Temporal Context Benchmarking Adaptive Management Development of global HRH system indicators Ethical and moral indicators (e. g. fair regional distribution) o Level of ethical concerns and standards within morally conscious framework o Fairness in relationship to recruitment (transparency) o Equitable distribution of the workforce Measure quality of partnerships Building capacity of local M&E staff to balance and align global/local priorities Global cost-savings because of local adoption of tools Level to which model is replicated elsewhere Frequency / extent of technical assistance given to other projects Measurement of changes in system efficiency Development expectation of strategic patience for donor and clients Benchmark indicators that recognize key steps along the way Early warning indicators to know when intervention is going off track (e. g. loss of interest)

Conclusion: the enabling environment • In response to the complexity of the HRH system, Conclusion: the enabling environment • In response to the complexity of the HRH system, the Capacity Project technical approach has emphasized a flexible, adaptive management approach to explicitly incorporate “context” as strategy for long-term success. • Sustainability of the HRH capacity building approach seems strongly linked to seeing context as an enabling environment, or even dynamic landscape for HR strengthening. • A multiscalar HRH M&E system is recommended to include HRH Workforce Outcome Indicators, HAF Action Field indicators as approximate measures, and enabling environment indicators to monitor sustainability. • Special attention should be given to indicators that measure levels of ownership, the quality of relationships, and ethics as missing elements.

Indicators for Measuring Human Resources for Health Capacity Building Linda Fogarty, Director, Results and Indicators for Measuring Human Resources for Health Capacity Building Linda Fogarty, Director, Results and Knowledge Management, Jhpiego Danny de Vries, Monitoring, Evaluation and Research Manager, Intra. Health International, Inc. Erik Reavely, Monitoring and Evaluation Advisor, Intra. Health International, Inc. Elizabeth Bunch, Manager, Knowledge Sharing, Intra. Health International, Inc.

The Capacity Project - Acknowledgements • USAID, Office of Population and Reproductive Health • The Capacity Project - Acknowledgements • USAID, Office of Population and Reproductive Health • USAID, Office of HIV/AIDS • 7 Partners: Intra. Health International, IMA World Health, • Informants (Senior Technical Advisors, USAID colleagues) Jhpiego, Liverpool Associates in Tropical Health (LATH), Management Sciences for Health (MSH), Program for Appropriate Technology in Health (PATH), Training Resources Group, Inc. (TRG)

Outline • What did Capacity Project do? • How were results tracked? • How Outline • What did Capacity Project do? • How were results tracked? • How did technical advisors define success? • How can we refine our indicators?

Examples of technical work • Plan • • • Southern Sudan: assessment, stakeholder group Examples of technical work • Plan • • • Southern Sudan: assessment, stakeholder group development, policy, costed plan HRH information systems in 10 countries Develop • • Rwanda: HIV, FP in PSE, IST Support • G-CAP (Guatemala, Belize, Costa Rica, Nicaragua, El Salvador, Panama) performance support to decentralize HIV services

Outline • What did Capacity Project do? • How were results tracked? • How Outline • What did Capacity Project do? • How were results tracked? • How did technical advisors define success? • How can we refine our indicators?

Capacity Project Results Framework Plan Develop Support Capacity Project Results Framework Plan Develop Support

Monitoring Method • Project PMP and targets established with USAID approval (12 core-, 40 Monitoring Method • Project PMP and targets established with USAID approval (12 core-, 40 sub-indicators) • Country-specific PMP developed Year one • Core indicators appropriate for country strategy • Baseline “country profile” established • Quarterly discussions with country “pointperson” • Annual indicator assessment and reporting

Most indicators track output • Number of HRH systems strengthened • • Pre-service education Most indicators track output • Number of HRH systems strengthened • • Pre-service education systems, in-service training systems, human resources for health information systems, retention systems, supervision systems… HRH strategic plan, policies in place

Some indicators track outcomes • Improved use of data systems • Improved health worker Some indicators track outcomes • Improved use of data systems • Improved health worker retention • Improved health care access (special studies only)

Capacity Project Indicator Areas: HRH System Strengthening Area Examples Have the necessary information, tools, Capacity Project Indicator Areas: HRH System Strengthening Area Examples Have the necessary information, tools, policies, “building blocks” to improve capacity HR information resources, job descriptions, approved workforce plan, workforce assessments Improved infrastructure clinical practice sites with updated providers, necessary equipment Improved systems in place HRIS, training, HRM, employee recognition, retention Improved individual skills HR management skills, business acumen Improved collaboration between groups stakeholders working together to address HCD issues, key global partnering mechanisms

Outline • What did Capacity Project do? • How were results tracked? • How Outline • What did Capacity Project do? • How were results tracked? • How did technical advisors define success? • How can we refine our indicators?

Findings: New HRH Outcome Indicators From • Realigning the workforce To • Needs-based deployment Findings: New HRH Outcome Indicators From • Realigning the workforce To • Needs-based deployment • HRH plans in place • HRH plans implemented • HRM systems strengthened • Reducing vacancy rates • PSE strengthened • PSE tutor/student balance • Better HRM & Retention sys • HRM sys and retention links • n/a (small scale evals) • Quality of services • n/a (small scale evals) • Links to service delivery

Findings: New HRH Process and Output Indicators, “Critical Success Factors” From • na To Findings: New HRH Process and Output Indicators, “Critical Success Factors” From • na To • Key people are now in key places making key decisions • Changes in system efficiency • Systems, products are used, integrated into existing, users innovate • Quality of relationships • Program ownership

Implications • Need more sophisticated immediate and intermediate outcomes reflecting more sophisticated TA and Implications • Need more sophisticated immediate and intermediate outcomes reflecting more sophisticated TA and country context • Include wide range of process, or “critical success factors” indicators • Link HRH system changes with service delivery outcomes • Bring together HRH measurement experts for “next generation indicator development”