Issues Particularities and potential Pitfalls 6 CP Spring

Issues, Particularities and potential Pitfalls 6 CP Spring Workshop 2007, Dublin JOANNEUM RESEARCHInstitute for Technology and Regional Policy INNOVATION aus TRADITION The Regional Dimensions of Evaluation Christian Hartmann Wolfgang Polt wolfgang. polt@joanneum. at JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at ISO 9001 zertifiziert christian. hartmann@joanneum. at 1

Overview Ú Recent trends in Evaluation Ú The Regional Dimension Ú The „typical“ regional Evaluation Project ISO 9001 zertifiziert Ú Particularities and potential Pitfalls JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 2

Recent Trends in Evaluation Ú Increased emphasis on impact assessment Ø ex-ante ( priority setting, links to foresight) Ø ex-post ( economic effects, behavioural additionality) Ø Widening the scope: IA of basic research Ú ‚Concept evaluations‘ (ex-ante, interim) Ø Checking the rationale and early implementation Ú ‚Systemic evaluations‘ OECD (CSTP Oct 2006, Seoul) asked countries to „improve the capacity to carry out ‚Systemic evaluations‘“ Ø Addressing Innovations systems as a whole (Trend. CHART; ERAwatch, EIS, OECD STI Scoreboard, . . ) Ø Addressing portfolios RTDI policy instruments (e. g. direct vs indirect support for private R&D) Ú Asking for the impossible. . ? JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at ISO 9001 zertifiziert Ø Addressing the ‚Policy Mixes‘ of countries 3

ISO 9001 zertifiziert Evaluation in the policy cycle JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 4

Evaluation needs a systemic (mulit-actor, multilevel) perspective international ISO 9001 zertifiziert regional JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 5

ISO 9001 zertifiziert The Regional Dimension JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 6

Why does the regional dimension of evaluation matter? Ú Regional Innovation Systems Ø Globalisation leads to a growing importance of localized innovation capabilities, that are hard to copy / to transfer Ø The growing importance of regional innovation systems in the economic literature (Asheim et al. 2003, Cooke 2003, Malmberg and Maskell 2002) leads a stronger focus on the regional level in the policy domain Ø The concept of regional innovation systems offers effective (low cost) roads for policy delivery Ø Devolution of (innovation) policy competencies in some EU countries (e. g. France, England) Ø Growing interest in RTDI policy at regional level in federal provinces or autonomous regions in EU countries Ø Growing importance of regional RTDI policy at EU-Level (i. e. RIS/RITTS, Regions of knowledge) JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at ISO 9001 zertifiziert Ú Regionalisation of RTDI Policy 7

Strategic Policy Intelligence in the Regional Dimension Ú Different Levels of Development among European Regions Ø While some European regions already have a long existing practice (like Styria, Basque Country, etc. )… Ø …in particular regions in the NMS are just at the beginning to develop respective structures and processes. Ú Different Degrees of Freedom for SPI in European Regions Ø Size of the region and corresponding capacities for SPI Ò i. e. Catalunya vs. Burgenland Ò i. e. Friuli Venezia Giulia (IT) vs. South-Transdanubia (HUN) JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at ISO 9001 zertifiziert Ø Degree of regional autonomy for RTDI policy 8

ISO 9001 zertifiziert The „typical“ regional Evaluation Project JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 9

Is there such a thing as a “typical” regional evaluation project? How typical is “typical” - regional evaluation projects do actually cover a wide range of issues and policy areas Ú Regional Evaluation is often Evaluation of Regional Policy (i. e. Structural Funds) Ú Evaluation in the regional dimension does often mean: Evaluation of small and medium sized programmes with regional funding Ø Regional Cluster Initiatives Ø Regional/Municipal Economic Programmes Ø Regional Innovation Infrastructures (i. e. Technology / Science Parks) Ø Key projects in the framework of Community Initiatives JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at ISO 9001 zertifiziert Ú Evaluation of regional strategic projects 10

Are “typical” regional evaluation projects following the general trends? “Typical” regional evaluation projects do reflect general trends in evaluation - but with varying degrees of intensity Ú Concept evaluations do already play a big role on the regional level Ø Structural Funds Ú Increasing emphasis of impact assessments on regional level Ø Regional RTDI Programmes Ø Strategic Projects ISO 9001 zertifiziert Ú Systemic evaluations are at the moment not (yet) of relevance JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 11

The „typical“ regional evaluation project and Strategic Policy Intelligence Ú No (complete) policy cycle for RTDI policy in most European regions Ø Reluctance to confront the past Ú Evaluations are often motivated by the necessity to letigimate the past Ø Evaluations as tools in regional political games Ú There are “natural constraints” for SPI corresponding to the Size and political status of the Region Ø Lack of policy capacities ISO 9001 zertifiziert Ø Lack of degrees of freedom for SPI JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 12

ISO 9001 zertifiziert Potential Pitfalls JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at 13

Potential Pitfalls of evaluation at the regional level Roughly the same that face evaluations on all levels, but more accentuated: ÚLack of capacity to carry out evaluations – and to absorb and implement the results of evaluations ÚDifficulties of ‚role delineation‘: the evaluator as policy maker? Too close to policy ÚPotential to influence the evaluation results because of closeness of policy makers to stakeholders impact assessment) JOANNEUM RESEARCH Forschungsgesellschaft mb. H, Steyrergasse 17, A-8010 Graz, Austria, e-mail: pr@joanneum. at, web: http: // www. joanneum. at ISO 9001 zertifiziert ÚExpectations too high (especially with respect to 14