ASSESSING IMPACT OF HEALTH AID HOW CAN PUBLIC

ASSESSING IMPACT OF HEALTH AID: HOW CAN PUBLIC HEALTH SCIENCES CONTRIBUTE TO MUTUAL ACCOUNTABILITY? MOVING FROM EVALUATING THE IMPACT OF PROGRAMMES AND INTERVENTIONS TO HEALTH SYSTEMS AND CAPACITY STRENGTHENING PHYLLIDA TRAVIS, 6 FEBRUARY 2012

IHP+ experience with 1. Issues and challenges in communicating results of health aid 2. Some examples of linking aid effectiveness to 'results' 3. Of strengthening mutual accountability for results 4. Approaches to making health aid for system strengthening more effective / efficient

Who is IHP+? 2007 2012 Developing countries: 8 30 Bilateral donors: 8 13 Int'l agencies and foundations*: 11 12 TOTAL 27 55 *African Development Bank, Bill and Melinda Gates Foundation, European Commission, GAVI Alliance, Global Fund, International Labour Organization, UNAIDS, UNICEF, UNDP, WHO, UNFPA, World Bank

1. Issues, challenges in communicating results Want results that are memorable, persuasive, credible, useful, but Politicians / public v public health experts Country policymakers/managers v global perspective Stories v numbers Progress v impact Impact of aid per se v impact of way give aid Joint v individual results Plausible v causal links 'Quick wins' v sustained gains Improved messaging now v in 5 years, w new methods

Communicating benefits, results IHP+ country teams meeting 2010: Clear progress on putting aid effectiveness principles into practice Need to better document results / outcomes, find ways of conveying better. Need mix of good evidence, good stories, moderate claims In 2011 IHP+Results annual report on signatories' progress - visual; what not why OECD TTHATS report for Busan - sound; not sexy Accumulation of narrative country examples - what, why, how; distillation needed Commission on Information and Accountability - new catalyst

May 2011 IHP+ Results report Individual scorecards

And comparisons across partners

2. Some country specific examples linking aid effectiveness and results Ethiopia, Nepal, Mali (all of which have positive trends in service delivery and health outcomes) plus DRC, Benin Asking 3 questions To what extent have aid effectiveness principles been put into practice? Has this contributed to better results? Has health aid actually become more effective? Have health systems been strengthened? Have health services improved? What was critical in achieving these results? Where were the barriers and how have they been overcome?

Aid effectiveness and results …different perspectives Can trace plausible links from 'putting Paris into practice' via stronger systems to improved services and then …more tenuously…. health outcomes All note 'two speed implementation' of Paris agenda Is also an argument that goes Outcomes are important Proving link between AE and results (outcomes) is more of a global than country concern Risks excluding other important, common sense benefits to countries of working to 'one plan, one budget, one report' Need frame the debate to reflect wider range of benefits/results

Links between effective aid and results: examples Nepal: scale up of free maternal health care from a few districts to nationwide over 2 -3 years was possible because government and donors acted collectively – no one could have done it alone. Institutional deliveries risen from 18% 2006 to 28% 2011. Ethiopia: Health extension worker programme went nation-wide over 5 years, faster than many thought possible, because FMOH provided a clear vision; credible plan and indicated where support was needed from DPs DRC: A new MOH single donor coordination unit led to threefold reduction in management costs for donor funds, from 28% - 9%. Liberated funds for other uses. Message: better coordination of resources can deliver greater value for money.

DRC: Reform of health financing – some results MOH wanted to see The reform benefits from the support, both technical and financial, of all partners even if its utilisation is gradual

AE principles in practice? Other types of results one MOH is interested in Indicators Target for 2009/10 (Compact) Achievement 2009/10 % DPs providing confirmation 95% of long term commitments 63% % DPs which don't ask Government for a separate plan 95% 58% % DPs' activities and budgets reflected in the government's plan 100% % of Funds provided through Government preferred 60% modalities (no baseline available) (baseline 23%) 76% (baseline 44%) 19% (baseline 24%)

3. Strengthening mutual accountability MA important route towards greater aid effectiveness, but slow progress Mutual accountability efforts in health Countries Global / regional level MOH and EDPs: Compacts; some MOHs incorporating indicators of DP aid behaviour into JAR agendas eg Mozambique, Ethiopia, Nepal… MOH and parliaments: politicians getting involved eg Nigeria MOH and CSOs, research institutes – mixed; efforts to increase; Uganda 3 rd round of monitoring by IHP+Results – 19 countries in 2012 (10 last time); 15 DPs Upcoming meeting of southern health CSOs – will discuss scorecards as one tool for MA Note: mutual v joint accountability / responsibility crops up a lot in-country

4. Making health aid more effective in health system and capacity strengthening: Nepal Existing instruments 5 year health sector programme (NHSP 2), results framework; partnership agreements; joint annual review (JAR) 2012 JAR: systems problems prominent E. g. only 15% of facilities had no drug stock outs (target 70%) Getting TA better aligned with NHSP-2 priority problems a major theme New actions to harmonize/align TA with priority problems TA matrix against NHSP 2 objectives identifies gaps / overlaps Joint technical co-operation agreement agreed To match TA to priorities in Annual Work Plan and Budget Ie not just about systems knowledge/ research but practical instruments

Factors enabling aid to effectively contribute to health systems and capacity strengthening Critical enabling factors seem to be Government ownership Leadership from MOH and at least some DPs Strategic plans and review processes established and taken seriously Joint financing arrangements that are adhered to (including more predictable funding), complemented by more joined up TA? Some home-grown form of mutual accountability Common barriers DP own organizational incentives/adaptability Willingness to synchronize work Trust

spares

Moving from impact evaluation to health systems and capacity strengthening An aid effectiveness perspective…joint responsibility Context Old chestnuts post Busan; One does not follow the other – move in parallel what politicians and the public listen to – and when Impact has to be more than health Benefits of 'more effective aid' often on more proximal aspects of system performance Existing approaches – 2 country examples New developments Increased emphasis Utility of compacts – what they contain

Common monitoring framework of health progress and performance Inputs & processes Financing Indicator domains Governance Infrastructure; ICT Health workforce Supply chain Outputs Intervention access & services readiness Intervention quality, safety Outcomes Impact Coverage of interventions Improved health outcomes & equity Prevalence risk behaviours & factors Social and financial risk protection Responsiveness Efficiency Information Data collection Administrative sources Financial tracking system; NHA Databases and records: HR, infrastructure, medicines etc. Policy data Facility assessments Service readiness Population-based surveys Coverage, health status, equity, risk protection, responsiveness Clinical reporting systems quality, coverage, health status Civil registration Analysis & synthesis Communication & use Data quality assessment; Synthesis of progress and performance; Evaluation Regular country health sector review processes; Global reporting

International funding remains unpredictable

Growing numbers of players in health: Chad ONG nationales et internationales Nations Unies MSFs DED CCSI MDM Croix Rouge UNAD OMS ONUSIDA COOPI UNFPA CSAI ACF CICR BASE e r rd O PAM UNICEF Libye UNHCR Egypte USAID Chine Cuba Allemagne Suisse France ECHO UE Membres de l'UE PNUD OCHA FAO Initiatives mondiales EMEET de a M lte OR H T M EN BM BAD t. sti In. fin. int GAVI Fonds mondial contre le Sida, la tuberculose et le paludisme

issues Focus on (quick) results does not need to compromise HSS / CB efforts, but needs to be managed Results need to be expressed in different ways Traditional approaches to HS and capacity strengthening mixed record AE p's apply to this as

Links back to impact evaluations / public health research One of the 9 standard JAR background documents is a research report, summarising major health related studies in Nepal in last year. In 2011 research report covered findings from Major periodic surveys eg DHS, NLSS, service tracking survey Financing studies commissioned as part of strategy development Studies on effectiveness of new policies eg reducing barriers to maternal health services Studies on aid effectiveness and results Improving background documents is an iterative process Some research topics identified in JAR eg on procurement

Snapshot of Mali Progress with putting AE principles into practice Coherent planning framework (PRODESS+, MTEF); stronger MOH ownership; more alignment with strategies; less with systems; greater management for results; some greater donor harmonization, MA Results 1: more effective aid Mixed: more predictable pledges; disbursement worse; some transaction costs seen as investment (but many still negative); TA more focused on systemic priorities Result 2: health system strengthening

Mali: Conclusion and perspectives • Important progress in implementing aid effectiveness principles practice and behavior changes highly probable contribution to HSS and sector results • BUT “two-speed implementation” of the principles of the Paris Declaration and IHP+: – – – Only 13/50 donors signed the Compact and 3 do SBS Sharp decrease in executed ext. financing since 2008 Still many earmarked funding (thematic. /geograph. ) Specific procedures and PIUs Aid not predictable Individual missions / audits in addition to joint ones Expectations must be realistic 27 October 2011 Mali Case Study 25