- Количество слайдов: 15
Evaluating the system-wide effects of HIV scale -up: methodological gaps, challenges and recommendations David Hotchkiss Health Systems 20/20/Tulane University
Introduction § Wide agreement that further evidence is needed on how GHIs have influenced health systems § Research in this area has been characterized as serendipitous rather than systematic § Black box between inputs and outcomes, and intended and unintended effects, still needs illumination
Overview § What type of evidence is needed? § What are the obstacles to evaluation? § What are the available M&E approaches? § Where do we go from here?
What type of evidence is needed? § Routine monitoring 8 Monitoring of global goals and GHIs 8 Country-level health system and program monitoring focused on inputs, processes and outputs § Special purpose evaluations of the impact of health systems strengthening initiatives § All these aspects are overlapping and require an integrated approach at the country and global levels
What are the obstacles to evaluation? § Evaluations require collective action 8 Systems research traditionally has not received adequate financial support 8 High quality studies “do get done, but not in the numbers or with the quality that are justified by the global benefits” (CGD 2006) § Arrangements for prospective HSS assessments typically not built in from the beginning 8 Too little investment in baseline survey data 8 No treatment and control groups to assess counterfactual
What are the obstacles to evaluation? (2) § Routine tracking data should provide much of the data required for evaluation, but unfortunately, RHIS data are often incomplete and of poor quality § Attribution is challenging from a technical perspective 8 Systems strengthening a long-term, complex process 8 GHIs are also extremely complex and dynamic 8 Difficulty in attributing impact to any one GHI § Other factors, including concern about possible unfavorable results and limited capacity
What approaches and methods are available for monitoring and evaluation?
Example of a conceptual framework: IHP+ common M&E framework
What approaches and methods are available for monitoring? § National Health Accounts and Sub-Accounts, a tool for tracking sources and uses of funds 8 Comparison of NHA data over time can yield useful insights on system-wide effects § Health Systems Assessment Approach: A How-To Manual (Health Systems 20/20) 8 Provides indicators and sources of data by WHO’s health system functions
What approaches and methods are available for evaluation? § Retrospective 8 Global cross-country comparative analysis 8 Country-level mixed methods analysis (quantitative and qualitative, including case studies) § Prospective 8 Experimental designs assessing impact of interventions; economic evaluations of costs and benefits § Consider intervention-specific approaches 8 For some types of interventions, such as financing, service delivery, rigorous methods are available 8 For others, such as RHIS, more work is needed
Where do we go from here? § Great opportunity exists to invest in systems research 8 GFATM and GAVI increasing systems funding 8 G 8 placing more emphasis on systems strengthening 8 IHP+ has developed a common evaluation approach § Develop “a new field of science for health outcomes and systems research among a constituency that has been fragmented in the past” (MPS 2008)
What does this involve? § Advocate rigorous M&E as a key component of the health systems strengthening agenda § In establishing HSS objectives of GHIs, establish systems to monitor and evaluate process 8 Align health systems M&E with implementation, country planning cycles and mechanisms § Continue to develop frameworks, metrics and methods 8 Invest in RHIS to strengthen data quality and use 8 Develop intervention-specific frameworks and methods 8 Continue to develop mixed-methods approaches 8 Look for opportunities for prospective evaluations
Selected references Atun RA, Bennett S, Duran A. When do Vertical (Stand-Alone) Programmes Have a Place in Health Systems? Policy Brief, WHO European Ministerial Conference on Health Systems, 25 -27 June, 2008, Tallinn, Estonia. Banteyerga, H, Kidanu, A, Stillman, K. (2006). The Systemwide Effects of the Global Fund in Ethiopia: Final Study Report. Bethesda, MD: PHRplus. Abt Associates Inc. Global HIV/AIDS Initiative Network (GHIN) (2006). A Generic Guide to Research Practice: Following discussion at Lilongwe workshop of GHIN African teams. International Health Partnership (2008) Monitoring Performance and Evaluating Progress in the Scale Up for Better Health: A Proposed Common Framework. M&E Working Group. Kruck ME, Freedman LP (2008) Assessing Health System Performance in Developing Countries: A Review of the Literature. Health Policy 85. Rockefeller Foundation (2008) Leveraging HIV Scale-up to Strengthen Health Systems. Report of high-level meeting in Bellagio, Italy, 2 -5 September WHO (2007) Everybody’s Business: Strengthening Health Systems to Improve Health Outcomes. WHO, 2007.
Selected references (2) WHO (2008) Maximizing Positive Synergies Between Health Systems and Global Health Initiatives. Report on the 3 rd expert consultation, WHO, Geneva, 2 -3 October. WHO Maximizing Positive Synergies Collaborative Group (2009) An Assessment of Interactions Between Global Health Initiatives and Country Health Systems. The Lancet (373) June 20.