Indicators to characterize public funding systems Benedetto Lepori

Indicators to characterize public funding systems Benedetto Lepori, 28 th November 2008

The issue o o o Is it possible to produce indicators to characterize/compare public funding systems including n The role of different funding agencies n The allocation methods n The streams of money n The beneficiaries Beyond R&D statistics which is essentially focused on performers n With limited ability to map funders – performers flows n With a too simple description of funders (by “sector”) n Not taking into account the existence of intermediaries Obvious relevance for public policies of these indicators n o See for example the OECD work on steering and funding of public research where no data on allocation could be collected Two PRIME project on the field n n Exploratory project on public project funding Analysis of CEEC funding systems and of their change

An overall view of public funding o Four relevant system layers n n o Two main allocation streams n n o State Funding agencies Performers’ organisations Research groups/individuals) core funding project funding Key issues n n Interactions accross layers and funding modes Increasing role of performers in driving funding

Policy layer National state Funding agencies (national) Agencies layer Core funding Organisations layer Research groups layer Project funding Higher Education Institutions and PROs Internal allocation International agencies (EU, ESA, etc. ) Research units

Project funding analysis o o Try to compare countries concerning the role and composition on project funding Identify in each country the instruments which we can identify as project funding n As well as the main funding agencies n A basic definition + a lot of comparative work to treat in the same way the same instruments Collect the data from different sources n Ministries reports, agencies reports, etc. n Per year and main beneficiary group n Data collection and cleaning procedures Produce aggregations and international comparisons n As well as analyses of the evolution in last 30 years

Criteria o o We consider project funding if n Attributed mostly for research purposes n Limited in time and scope (not recurrent) n Attributed by an external agency to the research organization n No reference to competition/modes of allocation Issues / problems n Use for research activities (since we are not looking to performers) – difficult to assess for example for contracts n Long-term competitive schemes (centres of excellence) n Internal competitive schemes to research organizations (or vertically integrated organizations like academy of sciences)

Problem cases o European Space Agency contracts n n n o Mostly to industry for development of rockets, satellites, etc. Some funding for scientific programs It is nearly impossible to ascertain how much should be considered as R&D We adopted a broad delimitation n Including probably more that would be considered as R&D funding

French CNRS o A very large organization with a double role n n o A partially competitive allocation mechanism n n o Funding its own laboratories (10%) Funding joint laboratories with universities (thus partially outside the CNRS perimeter) Labelling by CNRS gives access to its resources and there is a rather high turnover Mostly in form of personnel, but no mandatory allocation It is something intermediary between general and project funds n n Accounting for nearly 10% of public research funding in France, thus impossible to put in a footnote Our picture of the French system largely depends on the choice we do concerning these funds Considering it as project gives a view of France much nearer to other Western European countries See Theves et al. paper

Discussion o Overall the distinction works and is usable but n n o One needs much care in the comparisons especially for time series We should try to develop a finer typology of funding instruments in the future Intensive discussion was needed to solve dubious cases n n n Ensuring some comparability In-depth knowledge of national systems was essential for this exercise …this is not the end of the story…

Categories o Comparative analysis clearly needs common categories n o Beyond individual list of instruments/agencies at national level We devise three classifications n n n By type of agency (international, ministry, intermediaries) By instrument type By beneficiaries

Instrument types o o Intuitively it is clear that there are different instruments concerning their orientation / type of research they fund n But designing a clear classification is very difficult since these concepts are largely multidimensional n We end up with a simple distinction between “academic” / thematic / innovation-oriented instruments However, this is just a rough approximation of policy intentions, but not necessarily of the research done n Different features need to be combined to better understand the significance of instruments for the research system n Granularity is a problem since the level of aggregation is rather high

Data sources and data collection issues o There is no unique source of these data, but they had to be complied from: n n n o State accounts (France) Research ministry reports Agency reports Some existing databases Surveys (for example the Swiss R&D survey) Direct inquiry to ministries Most data are available but with limitations/problems in many cases n In most cases it was possible to come back to the ‘ 70

Data problems o Funding to companies through loans n o Project decisions instead of allocations n o One needs some kind of averaging European Space Agency n o Estimate of cash value Use national contribution as proxy (national return rule) Contracts from ministries n Coverage is problematic outside the formal programs …even if the methdology is simple there is a lot of dirty work to get usable data…

Some selected results o o PF as the second stream of research funding in the six considered countries n ¼ to 1/3 of funding volume n Strong increase in the role in the last years A composite model of funding instruments n Some academic, some thematic, some innovation n Try to accomodate the different goals instead of chosing a single rationale Differences between countries remain quite large n In the organisation of funding agencies and the type of funding n Strong dependence on history of organisational structures One needs to consider carefully this context when designing European funding policies

Type of PF instruments

PF as % of BIP

From academic to thematic and back… % Thematic instruments % Academic (bottom-up) instruments

From steering to markets o Today’s highly differentiated project funding system resembles increasing to a market n n Where loosely coordinated agencies buy research services from performers With strong strategic behavior of applicants o o Depending on their strenghts, needs, etc. Building stable market structures n n Needed to support long-term research, but also predictable allocation of funding Crucially based on shared representations on others actors behaviour o o n Of funding agencies Of potential competitors Some evidence that core markets are relatively small-scale o o Strong segmentation by domain/type of research/topic Few key players with high success rates, many marginal players

CEEC project o o o To analyse the changes in the organization of public funding in CEEC n Three countries as a test (Czech Republic, Estonia, Poland) n Look also for availability of data (including time series) Approach n Identify the main funding streams and to characterize them n Draw structural diagrams n Understand the role of funding agencies and allocation criteria See Karel’s presentation for more details on specific results

Poland MSHE Institutional funding 24, 19% 70, 66% 30, 81% 17, 37% Higher education 23, 81% 22, 48% AS Government units 7% HEI Institutes 13, 20% 0, 55% 0, 49% Institutes 3, 35% 0, 22% 0, 17% Institutes 5, 2% 0, 26% 0, 68% 2, 45% project funding Pri vat e 1, 46% EU FP 1, 57% EU SF MSHE fund FPS 1, 52% 0, 60% 0. 23%

Structural models o A step towards identifying the main structural features of national funding models n n n o Characterize the models in terms of some key features n n n o Taking into account the interactions between modes and layers (institutional complementarities) Different models related to different national contexts Which can be stable in an evolutionary perspective Ability of the State of steering the system Levels of delegation Performance vs. innovation Three basic models seems to emerge: n n n Project-funding based Centralised Mixed models

The project-funding based model o Most of research funding coming from highly differentiated set of non coordnated agencies n o State « buys » research services from groups n n o With limited attempt of central coordination Mission agencies funding also basic research (Do. D) Reliance on competition rather than on coordination to get the required results n n o High share of project funding and low of core funding stratified university system with concentration in a few players International excellence as the main criterion Limitations of this model when the system is too small n Cumulative effects suppress diversity of the system (the Estonian case)

The centralised model o o o A single large research organisation funded by the state (outside the university sector) n The old CNRS model before joint laboratories n The old system of Academy of Sciences in communist countries n The today’s Polish system The PRO has more functions than just funding n Decision on priority areas n Creation and restructuring of laboratories n Mostly coopting scientists in the PRO board A strongly centralised system n Define priorities and concentrate effort n Central planning, but with all its rigidities in face of new scientific development n Less adapted to new science dynamics more based on complementarities than on heavvy investments (new « search regimes » ; Bonaccorsi)

The mixed model o Some balance between core and project funding n o A differentiated performers sector n o Large university sector alongside a number of non-university research institutes Trying to achieve some balance between n n o Say 60% to 40% Targeting the best groups and spreading out resources (also because of the link with education and regional development) Differentiating the performers sectors to answer to different missions vs. creating competition between performers Typical of most Continental European countries including Switzerland n n n Best performing in small rich countries rather that on large ones Is the efficiency loss and the lack of central coordination acceptable? The Czech republic as a typical case

Conclusions (1) o It is basically feasible to characterize funding systems using existing data at national level n n o Mostly detailed budgetary data Complemented with data from agencies and R&D statistics A very good knowledge of national systems is required n n n To identify and classify funding streams To identify the really relevant features National experts are required for these tasks

Conclusions (2) o This works provides very interesting insights on the structure of funding systems n n Quantitative indicators are essential to characterize them and to distinguish variants The next avenue is to go towards data on micro-structures of funding systems

References (see www. enid-europe. org) o o o Lepori B. , van den Besselaar P. , Dinges M. , van der Meulen B. , Potì B. , Reale E. , Slipersaeter S. , Theves J. , (2007), Comparing the Evolution of National Research Policies: what Patterns of Change? , Science and Public Policy 34 (6), 372 -388. Lepori B. , Dinges M. , Potì B. , Reale E. , Slipersaeter S. , Theves J. , van d. (2007). Indicators for Comparative Analysis of Public Project Funding. Research Evaluation, 16 (4), 243 -255. Lepori B. , Masso J. , Jablecka J. , Sima K. , Ukrainski K. (2008). Research funding system in Central and Eastern European countries: a comparative analysis. Paper presented at the ENID-PRIME Indicators Conference, Olso, May 2008.