SYSTEM OF EVALUATION AND MANAGEMENT CONTROL RESULTS-BASED BUDGETING

SYSTEM OF EVALUATION AND MANAGEMENT CONTROL RESULTS-BASED BUDGETING THE CHILEAN EXPERIENCE October, 2008 Heidi Berner H Head of Management Control Division Budget Office, Ministry of Finance, Chile 1

EVALUATION AND MANAGEMENT CONTROL SYSTEM OBJECTIVE: Providing performance information and introducing practices to improve the quality of public expenditure • Improving resource allocation • Improving the use of resources • Improving transparency 1 2

EVALUATION OF PROGRAMS AND INSTITUTIONS • 1997: Evaluation of Public Programs (EPG) • 2001: Impact Evaluation of Programs • 2002: Agency Evaluation (Comprehensive Spending Review) 2001: STRATEGIC DEFINITIONS 1993: PERFORMANCE INDICATORS 1997: COMPREHENSIVE MANAGEMENT REPORT (BGI) 2001: PRESENTATION OF PROGRAMS TO BUDGET INSTITUTIONAL WAGE INCENTIVE MECHANISMS • 1998: Management Improvement Program: - Basic Framework Program (2001) - Advance Framework Program (2003) 1 • 2003: Incentive to Physicians 3 3

EVALUATION OF PROGRAMS AND INSTITUTIONS 2006: Increasing coverage - 155 programs or agencies will be evaluated between 2006 -2010. 2009: Creation of New Line: Evaluation of New Programs PERFORMANCE INDICATORS STRATEGIC DEFINITIONS COMPREHENSIVE MANAGEMENT REPORT (BGI) PRESENTATION OF PROGRAMS TO BUDGET 2007: Technical Assistance 2008: Reviewing Program Presentation 2008: PUBLIC MANAGEMENT MODERNIZATION FUND INSTITUTIONAL WAGE INCENTIVE MECHANISMS 2009: Management Improvement Program (PMG) – Framework Program of Quality 1 2007: Institutional Efficiency Goals 4 4

PERFORMANCE AND BUDGETARY INFORMATION • Not Direct relation (not mechanical) Use together with other categories of information - Political Priorities - Financial Restrictions • Presentation to the Congress along with the Budget Law 1 5

PERFORMANCE INDICATORS • Aimed at telling how a government organization is performing over time • Agencies compete with their past record • Ongoing, periodical information • Measure performance in different: – Dimensions (effectiveness, efficiency, economy, service quality) – Delivery levels (process, output, outcome) • Need reliable data collection methods • Support from strategic plan, management information system (PMG) • Started 1993 as a pilot, since 2001 is part of the budget preparation • Incrementalist approach, starting on voluntary basis • Budget-related (public agencies presents their indicators and its goals during budget preparation) • Disclosure policy: Congress and general public 1 6

PERFORMANCE INDICATORS Coverage and Quality of Performance Indicators 1 7

EVALUATION OF PROGRAMS AND INSTITUTIONS • • • • Assess programs against their stated aims and expected results on their beneficiaries Requirements: relevance, independence, timeliness, transparency, public, reliable, efficient. Agreement with Congress and Financial Management Law and its Regulation (2003) Started 1997 based on logical framework methodology (EPG) Budget-related (the results of the evaluation are taking into account in the budget preparation) Programs selected with Congress Performed by independent evaluators through panels of experts or universities and consulting firms. Counterpart in ministries/agencies in charge of programs Impact evaluations (2001) Agency evaluations (2002) Delivered in 6 to 12 months Reported to Budget, Congress and the public Commitments to incorporate recommendations from the evaluation Follow up of commitments compliance 1 8

EVALUATION OF PROGRAMS AND INSTITUTIONS Number of Programs and Agencies Evaluated ( ): On progress. 1 9

EVALUATION OF PROGRAMS AND INSTITUTIONS Evaluation of Programs (EPG - Impact) (2000 - 2008) EFFECTS % PROG. 1. - MINOR ADJUSTMENTS 23% 2. - MODIFICATIONS IN THE DESIGN OF THE COMPONENTS AND PROCESSES 37% 3. - MAJOR REFORMULATIONS 27% 4. - INSTITUTIONAL RELLOCATION 6% 5. - PROGRAM CANCELLED OR COMPLETELY REFORMULATED 7% TOTAL PROGRAMS (174) 1 100% 10

EVALUATION OF PROGRAMS AND INSTITUTIONS New Line of Evaluation: Evaluation of New Programs (EPN). To assure that Chile remains in the leading ranks of countries with systematic evaluation processes by updating evaluation procedures and processes to world frontier levels. i) Designing of the evaluation at the beginning of each new program ii) Establishing control groups for the evaluation, based on randomized trials whenever is possible iii) Establishing an international advisory committee to periodically review and assess the process of evaluation iv) Implementing this new line gradually 11 1 11

PRESENTATION OF PROGRAMS TO BUDGET • Started 2000 with preparation of 2001 Budget • Standard format to submit public programs for funding (2005 – 2009) • Ministries submit new programs and existing programs with importance budget increases • Submission based on logical framework matrix (aims, goals, expected results, components, indicators, target population) • Finance makes proposal on basis of quality, consistency with government priorities and strategies, President makes final decision • Provides better base line for future impact evaluation • 2008: Budget Direction gives technical assistance (new Department in the Management Control Division) • 2009: Review of all programs presented to budget 1 12

COMPREHENSIVE MANAGEMENT REPORTS • Started 1997, from Executive-Congress agreement • Conceived as accountability mechanism • Report performance against stated mandates, goals, commitments, resources • Supports compliance with other management control mechanisms (performance indicators, program evaluation, management improvement programs) and financial management regulations • Standard report format defined by Budget Office • Distributed to Congress, publicly available • Basis for gathering information for the evaluation stage of the budget cycle 1 13

MANAGEMENT IMPROVEMENT PROGRAM (PMG) • The achievement of management objectives are associated to a monetary incentive to all employees in the public institution (Law 19. 553, 1998). • The percentages of incentives are: % of Incentive for Institutional Performance 2007 - 2010 % of Achievement of Management Objectives 2007* 2008 2009 2010 5, 7% 6, 3% 7, 0% 7, 6% 90% - 100% 2, 85% 3, 15% 3, 8% 75% - 89% 0% 0% < 75% * Before 2007 the % of incentives were 5% and 2, 5%. The modification was done by Law 20. 212, august 2007. 1 14

MANAGEMENT IMPROVEMENT PROGRAM (PMG) ü ü The PMG has four period of development clearly defined. In each of these periods the accomplishment of the management objectives committed was associated to: Performance indicators. Period 1998/2000. Management Systems established in a Basic Framework Program (2001). Management Systems established in a Advance Framework Program including the requirements of ISO 9001: 2000 (2005) Framework Program of Quality – certifying the processes involved in the provision of good and services by ISO 9001: 2000 standard 1 15

MANAGEMENT IMPROVEMENT PROGRAM (PMG) Framework Program of Quality Advance Framework Program Basic Framework Program Public Management Areas and Systems (2001) Management Quality System implemented and processes of provision of good and services certified by ISO 9001: 2000 standard (2009) Public Management Areas and Systems defined in the Basic Framework Program certified by ISO 9001: 2000 standard (2005) 16 1 16

MANAGEMENT IMPROVEMENT PROGRAM (PMG) n. Basic Framework Program 5 areas and 11 management systems with 4 or 6 development stages. The evaluation of the accomplishment of each stage is done by the Ministries of Finance, Internal Affairs and Presidency with the help of an Expert Network (public institutions in charge) n Advance Framework Program The advanced framework program incorporates the objectives of the Basic Framework Program and the requirements of ISO 9001: 2000 standard, in order to certificate each PMG system. Four stages: preparation to certification, certification and maintenance of certification for two years. 1 17

MANAGEMENT IMPROVEMENT PROGRAM (PMG) EXAMPLE Areas Human Resources Customer Service Management Planning and Control Financial Management Gender focus I II Objectives of Management Development Stages III IV V VI VIII IX X o medium 10% o high 15% o high 11% o high 12% o low 5% o medium 10% o low 5% Systems Training Performance Evaluation Hygiene, Security and Improved work environment Integral Customer Service system e- goverment Internal audits Territorial management Planning/ Management Control Government Procurement Financial Accounting Gender focus Priority Weight high medium 11% 9% 1 18

PUBLIC MANAGEMENT MODERNIZATION FUND • Created by the Budget Law 2008. • Budding fund finances the elaboration of proposals to solve public management problems in some agency or program. • The proposals can be presented by public agencies or by universities, consulting firms or private consultants. • In January 2008 was the first bidding process. 66 proposals were submitted and 17 were selected to be developed during the first semester of 2008. • 7 from public agencies and 10 from universities, consulting firms and private consultants. • The budget 2009 considers resources to implement 5 of these 17 proposals. 1 19 19

END OF PRESENTATION www. dipres. cl / Sistema Evaluación y Control de Gestión 1