Ex post Evaluation of ERDF 2000 -2006 Adam

Ex post Evaluation of ERDF 2000 -2006 Adam Abdulwahab Evaluation Unit, DG Regional Policy Budapest, 6 th May 2010 1

Work Packages • • • Coordination Data feasibility Macro-economic modelling Structural change & globalisation Transport & Environment Enterprise & Innovation Gender & Demographic change Transport Modelling Rural Development Efficiency (Unit costs of major projects) Management & Implementation 2

Process • • Terms of Reference Open call for tenders for each contract Kick off meeting Inception Report and Interim Reports Draft Final Report and Final Report All deliverables are published on Internet Follow through (new task) – Proposals to decision makers – Information (knowledge? ) dissemination – Interactivity 3

Methodological Considerations • Very limited resources (budget & personnel) • Two goals: – Accountability – Learning • Various methods to get general overview • Case Studies for specific issues • Experimenting with new methods (counterfactual impact evaluation, modelling) 4

Lessons for Evaluators • Preparation of evaluation should start very early • Detailed Terms of Reference is a must • Early warning of any difficulty – There is always difficulty and (usually data gathering) – Beware of contractors who pretend everything is running smoothly • Kick off meeting and Inception Report are very useful • Contractors struggle to develop and substantiate – Narrative – Judgement – Recommendations 5

Results - Macroeconomic modelling • Two models: HERMIN, QUEST • Models measure what they are built to measure • Impact on GDP level by 2015: – EU 4: 18. 3% / 21. 1% – EU 10: 6. 9% / 10. 2% – donors: -4. 0% (QUEST only) 6

Results – Structural Change • “Vulnerability to globalisation” exists • Different responses and different use of ERDF support – Innovation and internationalisation – Employment creation and safeguarding – Balancing territorial development • Success factors: – Clear strategic policy response and early awareness – Interventions aligned with a broader regional strategy – Commitment and vision of regional planners, rather than specific regional specialisation or pattern of growth, determines policy responses and its effectiveness – There is need for differentiated strategies – ERDF had positive effect on policy learning about tackling structural change and globalisation 7

Results - Transport • Coordination with Cohesion Fund was not problem • 100, 000 km of roads - focus is still predominantly on roads (both “old” and “new” MS) – are they easy? • 4, 000 km railroad - progress is slow in the rail sector • ERDF interventions in airports/ports are extremely diverse but raise questions whether public investment is justified • Link between needs and strategies not always clear: filling missing links, inter-modality, cross-border planning are cited but implemented 8

Results – Environment & Climate Change • Significant contribution to environment – water supply: 3913 projects, additional population served: 14. 1 million – wastewater: 6211 projects, additional population served: 20. 5 million • Several factors limited effectiveness – – Investments were often oversized Financial sustainability was not always ensured Affordability vs. 'polluter / user pays principle' Delayed introduction of the polluter / user pays principles, relying more heavily on European funds in main beneficiary MS • Need to comply with European directives was main driver – Impact of environmental infrastructure on economic development is unclear – An environmental measure was justified even if its only effect was environmental – Capacity problems in implementation favoured selecting projects according to maturity to be implemented, wider socio-economic impacts were less important 9

Results – Demographic Change • Demographic and gender issues are often mentioned but rarely taken into account (awareness or design problems) • Few specific selection criteria and monitoring devices but rare involvement of stakeholders • Little quantitative evidence or specific indicators • Qualitative evidence shows positive effects • Specific measures aimed at specific groups work better than general principles 10

Results – Rural Development • No generally accepted definitions • ERDF supported regions lagging behind (less attention to rural/urban characteristics) • Big variety of solutions, delivery systems – Structural weaknesses are more important – Urban / rural distinction is less important • Division between ERDF and EAGGF/Guidance was clear at local level but cannot be generalised 11

Results – Unit Costs • Unit Costs: only meaningful for infrastructure • Very serious data problems – No publicly available database (relevant to EU) – No generally agreed definition of “unit cost” – Data is difficult or impossible to produce retrospectively • Evaluation produced “sector snapshots” • Poor performance in line with international experience – Optimism bias – Risk mitigation and sharing 12

Results – Implementation Systems • EU 10 on a learning curve: – – successful draw down available resources compliance with regulatory framework deficiencies in administrative and institutional capacities over-complexity and rigidity at the expense of strategic orientation • EU 15: compliance vs. performance – Main drivers: regulatory compliance and maintaining pace of spending – Insufficient focus on performance • Positive spillover effects on domestic management practice, strongest where – driven by committed 'policy entrepreneurs' – status and weight of Cohesion policy relative to domestic policies significant. • Lack of effective “policy management” and leadership 13

Thank you for your attention For further information, see: http: //ec. europa. eu/regional_policy/sources/docgener/evaluation_en. htm 14