New challenges for innovation policies in an uneven European innovation panorama: lessons from evaluation experiences

[Proceedings of the European Academy of Sciences & Arts 2025; 4:52] [page 51] Proceedings of the European Academy of Sciences & Arts 2025; volume 4:52 Introduction Investments in Research, Development and Innovation (R&I)1 have been crucial to societies for different reasons. Investments in science and innovation have been supported with the purpose of getting advantages in military races, health wars, economic competitiveness, and in the achievement of other societal benefits, such as, human training or environment protection. From an economic perspective, the creation of wealth and improved competitiveness have been important reasons to support R&I policies (Mowery and Rosenberg, 1989). From this perspective it is important to point that the foundations on which wealth creation and international competition are based have undergone substanNew challenges for innovation policies in an uneven European innovation panorama: lessons from evaluation experiences José Molero-Zayas,1 Antonio García-Sánchez,2 Ana Fernández-Zubieta3 1Department of Applied Economy, Complutense University of Madrid; 2Department of Economics and Economic History, University of Seville; 3Department of Applied Sociology, Complutense University of Madrid, Spain ABSTRACT The article addresses the recent changes in science and innovation policy studies from an European perspective. Based on the monitoring and evaluation exercises of four different innovation policy exercises carried out in Spain over the last years, we reflect on the challenges of European innovation policy. The monitoring and evaluation exercises cover different administrative levels (national, regional and European) and methodological approaches (impact evaluation and more qualitative-oriented) including: an impact-evaluation exercise of the Research and Development (R&D) public funding programme for firms granted by the main innovation agency in Spain (The Centre for the Development for Industrial Technology CDTI); two survey-based research on R&D from the workers and from the managerial perspective; and a monitoring exercise on the Smart Specialisation Exercises (S3) of three regions (Extremadura, Valencian Community and Catalonia) and the national S3 programme. We find that it is important to address the strengths and limitations of programmes and policies framed in previous innovation policies (innovation for growth and national systems of innovations), while improving coordination between innovation policies with other policy areas, layers (strategic, managerial and performance) and levels (national and regional). In addition, we signal the importance of broadening the understanding innovation to move towards a more transformative-oriented policy paradigm. We indicate that a broad understanding of innovation is especially important for less developed countries and regions to catch-up. Correspondence: José Molero-Zayas. E-mail: [email protected]) Key words: R&D and innovation policy; science and innovation policy studies; evaluation; development; Europe. Contributions: JMZ, AGC, AFZ, conceptualization, methodology, validation, formal analysis, manuscript original drafting; AGC, AFZ, software, data curation; JMZ, AFZ, investigation, resources, supervision, project administration, funding acquisition; AFZ, visualization, manuscript review and editing. All authors have read and agreed to the published version of the manuscript. Funding: this research was partly funded by the evaluation contract “Impact evaluation of the R&D programme grants of CDTI (20122020)” authorized by the European Commission Decision (2015) 4147 final, 22 June 2015, evaluation done in collaboration with Novadays; the contract service (art. 83) code 4157629 with CC.OO “The impact evaluation of innovation public aid to firms on employment and working conditions” (2019-2020); and the collaboration agreement between the UCM and the APD for the valorisation of innovation (2022-2023). Ethics approval: not applicable. Data Availability Statement: Data and code of the studies summarised here are available upon request. Access to impact research analysis data is limited due to Spanish National Statistical Office (INE) regulation to guarantee the anonymity of the database and results. Institutional ethical review information: not applicable. All participants were fully informed of the study and consent was obtained. Received: 7 January 2025. Accepted: 28 February 2025. Publisher’s note: all claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher. ©Copyright: the Author(s), 2025 Licensee PAGEPress, Italy Proceedings of the European Academy of Sciences & Arts 2025; 4:52 doi: 10.4081/peasa.2025.52 This article is distributed under the terms of the Creative Commons Attribution-NonCommercial International License (CC BY-NC 4.0) which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited. Topics: ORIGINAL RESEARCH Category: SOCIAL SCIENCES LAW & ECONOMICS/INTERDISCIPLINARY 1 In the text we use R&I to denote Research, Development and Innovation activities and R&D to activities limited to Research and Development. tial changes over the last decades. Due to those changes, today the so-called “created factors” play a prominent role which, unlike the «given factors» (e.g., endowed labour and capital), are the result of economic and social development. Among these created factors, those related to technology and innovation are particularly important, especially in a world subject to significant tensions and changes. In economic terms it is usually pointed that promoting technological innovation does not depend only on the market as a mechanism that emits certain signals for companies. Markets fail in R&I because there are certain characteristics of innovation, such as, the creation of externalities, the uncertainty of the results or the public good nature of knowledge that makes the market not to work optimally (Nelson, 1959; Arrow, 1962). For these reasons, it is generally accepted that public authorities must intervene with policies that compensate for what would be an investment in innovation below the socially desirable optimum if let alone to private entities. In addition, others have been argued the need of policies that favour technological innovation considering the systemic nature of innovation and the need to promote relations between innovative companies and a set of institutions that are part of the innovation system (Freeman, 1988; Lundvall, 1992). More recently, the rationale for R&I policies has broadened to also include the transformational aspects of society (i.e., transformation towards sustainability), going beyond the adjustments on poor performing firms in their R&I activities or on R&I systems (Schot and Steinmueller, 2018; Kattel and Mazzucato, 2018). Despite the different rationales to support R&I policies, there is a broad agreement on the need to know and evaluate the results of R&I policies. Public intervention is fully supported when the evaluations show positive results. Under the “market failure” rationale for public R&I investments to private companies, it is usually stated that it is not about replacing «market failures» with «public inefficiencies». However, different rationales define what is defined as a positive result of public intervention, changing evaluation patterns. Due to the changes in R&I policies we aim at reflecting on the impact evaluation of public funding to private R&I and on the need of integration different policy frameworks. There is an increasing demand for rethinking of R&I public policies. For example, to promote technological development towards increased technological autonomy, to strength the industrial base of the countries, or to promote transitions towards sustainability. However, it is important to reflect on the strengths and weaknesses of previous policy designs to better integrate these in the new policy frameworks. In addition, new policy designs need to consider that Europe has to redefine its international position due to the great international competition of Asia and America. This article aims to propose a debate for the European Academy regarding the European situation in the creation and dissemination of innovations in the forthcoming international division of labour. By its nature, it must be an interdisciplinary debate that can illuminate critical aspects of decision-making from states and European institutions. Change and innovation policies There is a growing body of literature in innovation studies (Fagerberg and Verspagen, 2009; Martin, 2012) that supports the need to adopt more ambitious and sustainable innovation policies to tackle major environmental and social challenges such as climate change, aging, inequality, etc. The COVID-19 crisis and active role of science and technology in the development of the vaccines have highlighted the effectiveness of the research and innovation systems in working towards a common goal (missionbasis). On the other hand, the COVID-19 crisis has also shown the weaknesses of national innovation systems in their dependence on global production and distribution chains. Both issues (effectiveness of R&I systems and supply chain security) appear to have strengthen the demands for more ambitious R&I policies that have been articulated over the last decade. This demand for science and innovation to be a driver of change is also encouraged by events, such as, the increasingly visible effects of climate change, the geopolitical and economic instabilities (e.g., the war in Ukraine and inflation), or the growing political polarization. In this context, policies shaping science and innovation towards the achievement of more environmentally friendly, equal and inclusive objectives are welcomed. According to the OECD (2021:11) recent challenges have reinforced the urgency to develop better frameworks for collective action towards common and well-defined objectives. This new trend of R&I policies towards more ambitious and sustainable policies responds to different labels, such as, missionoriented policies (European Commission, 2018; OECD, 2021); new industrial policies (Kattel and Mazzucato, 2018), or transformative innovation policies (Schot and Steinmueller, 2018; Fagerberg, 2018). This trend of R&I policy represents a shift towards what Schot and Steinmueller (2018) calls the third framework in innovation policy. The first framework, known as the “linear model”, considers that R&I policies should focus on providing funding for basic and applied science under the assumption that fostering scientific knowledge will eventually distil into innovations and economic growth. In this framework, public R&D investments are necessary to address the so-called “market failures”. The second model of R&I policies, instead of focusing on providing funds to science and technology development, focuses on channelling support for the improvement of the R&I systems. In this second framework, the main objective has moved from knowledge creation to knowledge transfer under the rationale that it is necessary to improve the links between university-business agents to favour knowledge transfer and the commercialization of inventions. In this framework, policy efforts seek to address “system failures”. The third framework of R&I policies argues that social and environmental challenges require new policy impetus as previous policy efforts that focused on providing funding for R&I and on the improvement of the R&I systems are limited. Instead, the third framework requires R&I policies to promote transformative change by addressing social and environmental challenges, such as, the European challenges or the Sustainable Development Goals (SDG). In the third framework, R&I should help to transform society. This third framework goes beyond addressing SDG-related challenges by actively connecting with societal concerns to address social failures. This third approach recognises that R&I policies should not only drive economic growth or improve R&I systems, but also address the gap between societal needs and R&I results. By focusing on social needs, this framework aims to make R&I more relevant and impactful for citizens, which implies the direct participation of citizens in carrying out the scientific activity and in science and technology policy. It should be noted that these R&I policy frameworks coexist, so it is not only necessary to address the differences between the various approaches but also how to integrate them within an existing policy landscape. These different R&I policy frameworks have prevailed at different times, have particular geographic focus, prioritize certain actors, emphasize some characteristics of knowledge and specific [page 52] [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] Article policy instruments, etc. (Table 1). Therefore, in the transformational approach, there is a demand to address major challenges and to go beyond the classic geographical, sectoral or technological limits that characterized previous R&I policies. It is required the involvement of different stakeholders emphasizing the importance of including civil society and going beyond the “Triple Helix” approach that required the involvement of government, science and industry. It is emphasized the directionality of R&I activity towards certain challenges, beyond the mere creation of knowledge and its transfer. The emergent and co-produced nature of knowledge is highlighted, which is no longer assumed to be merely transferable or sticky and centred in place. The socio-technical systems (Geels, 2004) gain prevalence. Similarly, certain policy instruments are prioritized. Innovation and Europe The approach of a renewed perspective of policies to promote innovation needs to be applied considering different contexts. In this article, the context is Europe in its relation to technological change and innovation. This reflection considers two crucial aspects: the comparative performance of Europe in relation to other competing areas or countries; and the asymmetric internal distribution of technological and innovative activities. European overall position In relation to Europe’s position, it is important to take distance from what for many years has been considered the «European paradox». This paradox states that Europe has a strong scientific base and, therefore, its strength lies in knowledge creation but fails in transferring this knowledge into the market. However, the framing of innovation in Europe within the “European paradox” has several limitations (Dosi, Llerena and Labini, 2006). Firstly, it is not exactly true. The scientific leadership of Europe is usually stated considering the number of scientific publications, but it not so much when considering other indicators, such as, the quality of these publications (i.e., European Commission, 2021; Dosi, Llerena and Labini, 2006). Secondly, there is not a direct relationship between scientific productivity and innovation. Finally, this paradox reflects a limited conceptualisation of innovation. A conceptualisation of innovation that is linked to the first policy R&I framework known as “the linear model”. As it was indicated in the previous section, this model portrays innovation as a product of scientific progress (science push). It must be recognised that this conceptualisation of innovation helps to understand some type of innovations, the ones linked to knowledge intensive processes (i.e., knowledge intensive innovations). However, it fails in explaining other sources of innovation (e.g., users as sources of innovation). Therefore, the framing of innovation in Europe within the “European paradox” has somehow limited Europe’s ability to identify its innovation challenges. In order to re-consider the European position on innovation it is important to underline two questions: the relationship of innovation with industrial development and the international position of Europe in relation to scientific-technological aspects. The consideration of industry is essential because this sector remains the largest producer and consumer of innovations and, as Teece and colleagues (1997) pointed out, industrial productive capacity is an essential complementary factor to properly appropriate the fruits of innovation. In this regard, there is a general decline of Europe in productivity during the twenty-first century (Bauer et al., 2020). The trend of recent years has generally been a process of deindustrialization of Europe in a context where especially Asian countries, with China heading, have been increasing their weight in international industry to the detriment of Europe and the United States. However, it must be stressed that European deindustrialisation has been an uneven phenomenon, affecting the countries of the South more importantly and, to a lesser extent, [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] [page 53] Article Table 1. Summary of the three frameworks of R&I policy. Framework 1: Framework 2: Framework 3: Innovation for growth National systems of innovation Transformative change Underlay model of innovation Linear Interactive and system bound Systemic and experimental Time of dominance 1960s-1980s 1980s to today Emerging Geographical focus National National, regional, sectoral system Multi-scalar: grand challenges across of innovation and beyond geographical, sectoral, technological boundaries Focal actors Government, scientists, and industry Triple-helix interaction Dynamic identification of relevant (especially large firms) stakeholders (government, science, industry, civil society, end users and non-users, others Justification for policy intervention Fixing market failures Fixing structural system failures Fixing transformational system failures Main strategy Knowledge creation Knowledge transfer Solving social and environmental challenges Nature of critical knowledge Transferable Sticky and place-based Emergent and co-produced Focal areas Technology Competitiveness Socio-technical system Typical policy activities • R&I stimulation • Building links • Support to experimentation with • Intellectual property regime • DUI (Doing Using and Interacting) niches • STEM education and learning • Support to R&D directionality communication • Entrepreneurship support • Social, inclusive, frugal and pro-poor innovation Source: Marinelli et al., 2021 and SPRU. countries such as Germany (Liboreiro et al., 2021). The consequences of Europe decreasing its technological capabilities and industrial basis has received increasing attention at European and national level. The “Draghi report” builds upon the need of the EU to increase funding to gain competitiveness (European Commission, 2024a). Its consequences in competitiveness, wealth creation, quality of jobs and security are increasing in a changing geopolitical context. Considering the comparative performance of Europe in the production of science and technology at international level, Europe holds a relevant position, but it also presents important weaknesses. For an example, Europe is one of the main contributors of highly cited scientific publications, but its position has been stagnant over the las decade with China overtaking the EU several years ago (Figure 1)2.The knowledge production landscape is changing with new actors (i.e., China and India) challenging the relevant position of the US and, specially, Europe in several research fields (Leydesdorff et al., 2014) and in emerging areas, such as, Artificial Intelligence (National Science Board, 2023). Figure 2 shows the EU’s position in digital and green technologies vs US and China’s position. The EU shows strengths in green technologies but important weaknesses in digital technologies. Internal asymmetries and inequalities Europe is facing new challenges that innovation policies need to address considering the differences across Europe. The European Innovation Scoreboard (EIS) offers an indication of the disparities in innovation performance across Europe and its evolution. EIS includes 32 indicators that are grouped into four main categories and 12 dimensions of innovative activity. These indicators are considered to build a Synthetic Index that summarizes the global position of each country. By relating the relative average performance of the index for each country to the European Union average in 2024 (European Commission, 2024b), Member States are divided into four groups (Figures 3 and 4). The first group comprise the Innovation Leaders and includes Denmark, Sweden, Finland, and the Netherlands with an innovation performance well above the EU average (above 125%of the EU [page 54] [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] Article Figure 1. Output in top 1% highly cited publications, % EU27, UK vs US and China. “The point in time at which China overtakes the EU is based upon whether the United Kingdom is counted as in or out of the EU block (European Commission, 2021). Source: JRC (EC, 2021) based on INCITES (Web of Science) data. Figure 2. Digital and green technologies position, EU vs US and China. Source: European Commission, 2024a. Patent data. Y-axis complexity (0 less complex and 100 more complex); x-axis relatedness (easiness in building comparative advantage depending on closely related to other technology strengths of the country). Size of the bubbles represents specialisation (revealed comparative advantage). 2 There are discrepancies on the year in which China overtakes the EU in highly cited publications according to the methodologies. For example, National Science Board (2023) using top 1% S&E journal articles from Scopus places this change in 2020. In any case despite the methodologies, the trends are similar. average in 2024 and by order of performance). The group of Strong Innovators includes Belgium, Austria, Ireland, Luxembourg, Germany, Cyprus, Estonia and France with a performance above the EU average (from 100% to 125%). Moderate Innovators are Slovenia, Spain, Czechia, Italy, Malta, Lithuania, Portugal, Greece, and Hungary with innovation performances below the EU average (70% to 100%). Finally, Croatia, Poland, Slovakia, Latvia, Bulgaria, and Romania are Emerging Innovators with performances well below the EU average (below 70%). The geographical distribution of these groups (Figure 4) portrays three main regional areas: the North-Centre area with countries with high innovation performance; the Southern area with mainly moderate innovators; and finally, the Easter area with countries with emerging innovators. EIS 2024 shows that the innovation divide persists in Europe, despite a slight decrease in innovation performance between 2017 and 2024. There is a modest convergence in innovation performance between Strong and Moderate innovators, but divergence within the Innovation Leaders and the Emerging innovators (European Commission, 2024b). In general, it can be said that strong leaders or innovative countries perform better in most indicators, showing balanced innovation systems where interactions between innovation actors create “positive spillovers” (innovative actions of one actor generally benefitting others and the system). On the other hand, countries with lower innovation performance (Emerging Innovations and most of Moderate Innovators), generally present greater imbalances between dimensions and categories of innovation. In this case, the interactions between the parts of the system are less fre- [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] [page 55] Article Figure 3. Innovation performance of the EU Member States, relative to the EU. Source: European Commission (2024b), performance scores are relative to that of the EU in 2017. Colored bars show the country’s performance in 2024. Vertical lines show performance in 2023. Figure 4. Innovation performance of the EU Member States. Source: European Commission (2024b). quent, reducing the opportunities to generate «positive spillovers». Innovative countries usually have usually strong R&I systems and stable institutional settings. In short, the implementation of measures to address the economic and social challenges that Europe face and to gain competitiveness vis-à-vis the major competitors of America and Asia needs to be carried out without forgetting the internal European asymmetries and providing flexible policy measures capable of being adapted to pursue intra-European convergence. Innovation case studies and lessons After reviewing the different innovation policy frameworks and two of the main innovation challenges of the European innovation system, this section aims at integrating the challenges and lessons of different R&I policies frameworks considering the Spanish case as an example. It includes a set of monitoring and evaluation exercises of different innovation policies and their impact in areas, such as, firm performance, workers’ and managers’ perspectives and links with regional policies. This benchmarking exercise helps us to highlight the strengths and weaknesses identified in the different exercises considering the different policy frameworks. In addition, we signal the importance of the coordination of innovation policies with other emerging policy areas, such as, new industrial policies and environmental ones. The benchmarking exercise carried out in this section includes a set of four different policy and evaluation exercises (Table 2): i) a more traditionally oriented impact evaluation exercises focused on the impact of the main programme aid granted by the Spanish innovation agency (Centre for the Development of Industrial Technology - CDTI); ii) a survey-based research on innovation from the workers perspective carried out for the main trade union in Spain (Workers’ CommissionsCC.OO. project); iii) surveybased research on innovation from the strategic management perspective (Association for the Progress of Management -APD); and iv) a set of monitoring exercises on a placed-based innovation policy (S3). These projects have different focuses, combine results from projects with different approaches, scopes, data sources, analytical units, and methodologies. They also have different disciplinary approaches and objectives (Table 2). However, the portrayal of the main results and policy implications allow us to see policy innovation efforts from a broad and multidisciplinary perspective considering its implication for different stakeholders. Traditional impact evaluation exercises of R&I policies focus on the economic rationality and efficiency using additionality analysis (input, output, and behaviour), understood as the additional stimulus on innovation greater than would have occurred in the absence of public support (Roper and Hewitt-Dundas, 2016). They do so consider the potential non-positive effects or the «crowding-out» effect on R&D expenditure (a substitution of private fund by public funds), and the potential bias in private agents’ decisions, generating market distortions as results of public programs. An overview of literature review about studies on the impact of public policies shows mixed results, largely due to the heterogeneity of companies, the geographical scope of the policies, the time periods and the type of policy analysed (and project selection rules), as well as in terms of data sources and methodology applied. This leads to different outcomes in terms of the additional private effort induced by policy actions. In any case, it could be pointed out that evaluation exercises tend to focus on input additionality and on output additionality with less attention to behavioural additionality. The results of these exercises show differences depending on the type of expenditure deployed by policies, the size of the company (Busom, 2000), the sector of activity (Huergo et al., 2016), the analysis model used (Lach, 2002) and showing different within country variation (OECD, 2006; Cunningham et al., 2016; Dimos and Pugh, 2016; Fiorentin et al., 2018). In general terms, reviews on the impact of public aid to private R&I show heterogeneous results between sectors, countries and even within countries. The heterogeneity of results supports the need of broadening the scope of impact evaluation practices. New concerns and rationales of R&I policies have emerged. For example, growing inequality in R&D intensity between European countries, and the difficulties of certain countries, such as Spain, to follow in the wake of the most developed countries in R&D. In this context, the demands for innovation policies that move from the need to address market failures to create new markets (Mazzucato, 2015) or that aspire to transformative change (Schot and Steinmueller, 2018) are welcomed. These changes and new frameworks demand new evaluation practices (Molas-Gallart et al., 2021) and ways to integrate them with previous impact analyses. The need of integration of different approaches supports us in combining different monitoring and evaluation exercises. The impact evaluation exercises show the effectiveness of public R&D aid programs in terms of additionality (input, output and behavioural), in line with a large part of the international literature (Table 2). Nevertheless, positive results are much more evident in terms of inputs (dedicated R&D resources) than in the outputs (technological results obtained). Considering the R&D output, public support does not always favour significantly better performance of beneficiary firms compared to non-beneficiary firms. Output results vary across sector, being positive results sometimes small in size. This could indicate the need to explore alternative scenarios that could lead to higher marginal effects (e.g., considering highly innovative initiatives or avoiding a “picking-the-winners” bias). The significant difference across sectors advises new policies to consider the diversity of sectors. While the impact evaluation of programs shows additionality of R&D inputs (R&D investments) and some not consistent across sectors additionality of R&D outputs, the impact of aid on the economic performance (e.g. sales, productivity) of beneficiary firms is hardly proven. Due to the low impact on economic firm performance, non-economic objectives focused on the characteristic of innovations (e.g. sustainable innovations) or the quality of jobs could be considered to tailor R&I policies. Impact evaluation exercises framed in R&I framework one (innovation for growth) do not usually question “innovation”. From this perspective, any innovation is good and will eventually lead to increased economic performance. As this is not always the case, new evaluation approaches could consider the characteristics of innovation pursuing radical, inclusive or sustainable innovations. Impact evaluation exercises also show important positive results on behavioural additionality, with beneficiary firms changing their behaviour towards increased collaboration with other agents of the innovative system. This is an important result pursued under the R&I framework two (national systems of innovation) that sometimes is achieved even without public R&I calls explicitly encouraging it. However, as impact evaluation exercises articulated within the first framework of innovation policies did not question “innovation”, impact evaluation exercises articulated within the second framework of innovation policies do not usually question “collaboration”. In this case, it is assumed that any collaboration will improve the innovation system. Therefore, new evaluation approaches need to consider the character and impact of increased collaboration. [page 56] [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] Article [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] [page 57] Article DCBA@?>=<;:?9A 8?>=<;:?9A7A>65A4A3B66:2>10:6A @:?A/?:;1.A>65A->10:6>,A+*)1<=)A :@AB66:2>10:6(A 8?>=<;:?9A7A>65A4A 8?>=<;:?9A7A>65A4A 8?>=<;:?9A7'A4A>65A &>?10>,,*A%A 31?>6)@:?=>102<A$.>6/<(A #?/>60">10:6>,'A <$:6:=0$A>65A):$0>,A 06)<?10:6A -:A:?/>60">10:6>,A>65A):$0>,A 06)<?10:6'A! 1A$:6)05<?)A $:,,>!:?>10:6A -:A:?/>60)>10:6>,A>65A ):$0>,A06)<?10:6'A! 1A $:6)05<?)A >,01*A:@A <=&,:*=<61A -:A:?/>60)>10:6>,A>65A ):$0>,A06)<?10:6'A! 1A 06$, 5<)A066:2>10:6A $ ,1 ?>,A>)&<$1)A <)'A&?:=:1<)A )1>9<.:,5<?)A 062:,2<=<61A 8>$0,01>1<)A:?A!,:$9)A 5<2<,:&=<61A -:A5<2<,:&=<61A$:6)05<?>10:6A >&>?1A@?:=A<$:6:=0$A/?:;1.A -:A5<2<,:&=<61A $:6)05<?>10:6'A! 1A >55?<))<)A:!A06< >,01*A -:A5<2<,:&=<61A $:6)05<?>10:6)A :6)05<?)A?</0:6>,A 5<2<,:&=<61'A! 1A 0=&,<=<61>10:6A$: ,5A!<A 0=&?:2<5A +$:&<A>65A10=<A)&>=A +&>06ABA& !,0$A>05)A&?:/?>=A 4744A>65A+&>06A447A +&>60).A;:?9<?)A@?:=A 065 )1?0>,A)<$1:?)A4 44A +&>60).A50?<$1:?)A>65A =>6>/<?)A3=<=!<?)A ('A444A D</0:6>,A:,0$0<)A3DB+%(A 4744A +&>06A>1>,:60>'A 1?<=>5 ?>'A>65A >,<6$0>A 2>, >10:6A@:$ )A B65<&1.A0=&>$1A<2>, >10:6A>65A $:6:=0$)A:@AB66:2>10:6A :?9<?)A&<?$<&10:6A:@A DCBA&:,0$0<)A>65A !<6<@01)A >6>/<?)A&<?$<&10:6A:@A DCBA&:,0$0<)A>65A !<6<@01)A :601:?06/A&:,0$0<)A>65A &?:/?>=)'A/:2<?6>6$<A >1>A): ?$<A B'ABA) ?2<*)'A065<&1.A 061<?20<;)A >1>A:6A+ ?2<*A:6A  )06<))A+1?>1</0<)A 3+('A>5.:$A) ?2<*'A <&<?1)A?: 65A1>!,<A A) ?2<*A1:AA =<=!<?)A :,0$*A&?:/?>=)'A ?</0:6>,A: 1$:=<)'A06 5<&1.A061<?20<;)A;01.A )1>9<.:,5<?)A 6>,*10$>,A 601A 80?=)A3BA/?>61<5A>65A6:1A /?>61<5(A 80?=)A3 >6101>102<(A>65A 0650205 >,)A3 >,01>102<(A B650205 >,)A.>206/A 50?<$102<A:?A<<$ 102<A ?<)&:6)0!0,010<)A D</0:6)'A&:,0$0<)'A &?:/?>=)'A)1?>1</0<)A <1.:5:,:/*A  ,102>?0>1<A>6>,*)0)A;01.A  >,01>102<A1?0>6/ ,>10:6A5>1>A &>6<,'A50@@0650@@'A >,01>102<A >6>,*)0)A  ,102>?0>1<A>6>,*)0)'A  >,01>102<A>6>,*)0)'A 1?0>6/ ,>10:6A +1>10)10$>,A>65A = ,102>?0>1<A>6>,*)0)A  >,01>102<A>6>,*)0)A !>)<5A:6A065<&1.A 061<?20<;)A;01.A =<=!<?)A:@A/:2<?6>6$<A )*)1<=A>65A)1>9<.:,5<?)A 3& !,0$A>65A&?02>1<(A  ,1050)$0&,06>?*A B66:2>10:6A)1 50<)A B66:2>10:6A+1 50<)'A >!: ?A<$:6:=0$)A B66:2>10:6A+1 50<)'A #?/>60">10:6>,A)1 50<)A B66:2>10:6A+1 50<)'A D</0:6>,A+1 50<)'A :2<?6>6$<A >?/<1A& !,0$A  ?:&<>6A:==0))0:6A)1>@@A>65A :1.<?A&:,0$*=>9<?)A36>10:6>,A /:2<?6=<61'ABA)1>@@('A >$>5<=0$)A>65A$020,A):$0<1*A +1>9<.:,5<?)'A>$>5<=0$)'A &:,0$*=>9<?A>65A$020,A ):$0<1*A +1>9<.:,5<?)'A >$>5<=0$)'A &:,0$*=>9<?)A>65A$020,A ):$0<1*A :,0$*=>9<?)'A&:,0$*A 5<)0/6<?)'A$020,A):$0<1*'A >$>5<=0$)A #!<$102<)A 2>, >10:6A:@A&:,0$*A0=&>$1A>65A &:,0$*A96:;,<5/<A <11<?A 65<?)1>6506/A >!: 1ADCBA<@@<$1)A:6A <=&,:*=<61A <11<?A 65<?)1>6506/A >!: 1A50?<$1:?)A>65A =>6>/<?)A&<?$<&10:6A >!: 1A066:2>10:6A :601:?06/'A !<6$.=>?906/'A,<>?606/A >06A?<) ,1)A #2<?>,,A&:)0102<A0=&>$1A?<) ,1)A A55010:6>,01*A:@A?<): ?$<)A 06& 1A>55010:6>,01*A 3&?:&<6)01*A>65A061<6)01*A06A DCA<&<6501 ?<)'A>65ADCA :!)A$?<>10:6(A A+:=<A: 1& 1A>55010:6>,01*A 3&?:5 $1A066:2>10:6'A=>06,*A 06A1?>5010:6>,A)<$1:?)'A! 1A ;01.A0=&:?1>61A)<$1:?>,A 50@@<?<6$<)(A A+1?:6/A!<.>20: ?>,A >55010:6>,01*A306$?<>)<5A &?:&<6)01*A1:A$::&<?>1<A;01.A ?<)<>?$.A$<61?<)'A06$?<>)<5A 061<?6>10:6>,0)>10:6A>65A 502<?)0@0$>10:6A:@A &>?16<?).0&)(A <>9A<$:6:=0$A?<) ,1)A;01.A )0/60@0$>61A50@@<?<6$<)A>$?:))A )<$1:?)A :)0102<A<@@<$1)A06A:!1>0606/A >,1<?6>102<A@06>6$06/A): ?$<)A !)<6$<A:@A=>?9<1A50)1:?10:6)A 3$?:;506/: 1(A >$9A:@A@:?=>,A=<$.>60)=)A06A 1.<A50))<=06>10:6A:@A96:;,<5/<A :)0102<A<@@<$1)A06A  >6101>102<A 3<=&,:*=<61(A>65A  >,01>102<A3>2<?>/<A 1?>0606/A,<2<,A>65A )>,>?0<)(A1<?=)A B=&:?1>61A<@@<$1)A:6A <=&,:*=<61A 3 >6101>102<A>65A  >,01>102<(A>)):$0>1<5A1:A ?>&05A1<$.6:,:/0$>,A $.>6/<A>65A065 )1?0>,A 1?>6)@:?=>10:6A 8:)1<?06/A@>$1:?)A A?>6)&>?<6$*A A:?9<?)A &>?10$0&>10:6A AB6A>55010:6A1:A)0"<'A $::&<?>102<A061<6)01*A >65A@:?<0/6A :;6<?).0&A A B66:2>102<A05<>)A$:=<A =:)1,*A@?:=A06)05<'A;01.A $:,,>!:?>10:6A06A 066:2>10:6A5<2<,:&=<61A ;01.A A602<?)010<)A AD<)<>?$.A$<61?<)A A#1.<?A&?:205<?)A06A 1.<A2>, <A$.>06(A B=&:?1>61A)0"<A>65A )<$1:?A<@@<$1)A <))A@?< <61A $:==<?$0>,A066:2>10:6A >06A!>??0<?)A0)A$ ,1 ?>,A ?<)0)1>6$<A1:A$.>6/<'A @:,,:;<5A!*A 6$<?1>061*A >65A,>$9A:@A)90,,<5A ;:?9<?)A A #&10=0)10$A20)0:6A>!: 1A 066:2>10:6A<@@<$1)A:6A <=&,:*=<61A B=&:?1>61A6>10:6>,A>65A ?</0:6>,A50@@<?<6$<)A06A &:,0$*A0=&,<=<61>10:6A :)0102<A&<?$<&10:6A:6A )1?>1</0$A&:,0$*A5<)0/6A <<?$0)<)A>65A )1>9<.:,5<?)A 062:,2<=<61A 8:)1<?06/A@>$1:?)A A:,0$*A,<>5<?).0&A A8:?=>,A 06)101 10:6>,0)>10:6A:@A $::?506>10:6A =<$.>60)=A>$?:))A /:2<?6>6$<A,>*<?)'A ,<2<,)A>65A@ 6$10:6)A =>11<?)A 3.*!?05>10:6A:@A 1<$.60$>,A!:50<)('A ;.<6A<6$:=&>))06/A !?:>5A1<$.6: <$:6:=0$A>65A):$0>,A 5:=>06)A A A A A Table 2. Summary of the policy and other evaluation exercises. Focus Innovation and firms’ Innovation and the Innovation and the Innovation and (Project) performance workers perspective strategic management regional perspective (CDTI) (CC.OO.) perspective (RIS3) (APD) To be continued on next page Other factors influence the innovation performance of firms but are not usually considered in an impact evaluation approach. For example, the institutional and cultural factors (how innovation is perceived and how innovation practices take place). Surveybased exercises on the workers and managers views on innovation ratify the need of broadened the scope of R&I policies to align R&I programs with, for example, the quantity and quality of employment created (Table 2). These studies show the need to foster linkages between innovation performance and innovation culture and work development and the importance of pursuing innovation towards sustainability and structural transformations. Managers’ views on innovation clearly show the importance of non-R&D related factors on innovation (e.g., design and product engineering). To consider other sources of innovation nonrelated to high technology development is especially important for small, less innovative firms or firms located in less knowledge intensive sectors. These survey-based research exercises show the importance of education and the promotion of innovative cultures. Therefore, unequal imaginaries of innovation (techno-optimistic vs pessimistic across employment levels) need to be considered when designing new R&I policies. The fourth policy exercise carried out is a monitoring exercises on different Smart Specialisation Strategies (S3) in Spain. S3 is a placed-based innovation policy approach that aims at encouraging regional innovation through the identification of strategic areas for intervention based on an analysis of research and economic strengths through an entrepreneurial discovery process engaging wide stakeholder involvement (Foray and Goenaga, 2013; Marinelli and Periañez Forte, 2017; Fernández-Zubieta, 2023). The monitoring exercise on several Spanish S3 confirms that R&I policy spaces is complex due to the existence of different layers (strategic, management and performance), levels (international, national, regional and local) and policy mixes (different programmes and policy domains) (European Commission, 2019) (Table 2). This complexity makes policy leadership especially important, together with coordination mechanisms. It is found that hybrid bodies need to be reinforced to be able to navigate and coordinate across policy layers, levels while being responsible for the management and implementation of policies and programs. The involvement of stakeholders in S3 is welcomed and could be encouraged. S3 has some elements of the transformative R&I policy framework (i.e., experimental character and stakeholders’ in- [page 58] [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] Article 7 654321730/432.-35,+7 *)(76'&43273,-%$#%,-35,73+7 "'+-3!3% 7%7.3 7/$5$.00%7 %4/+7-5730/$5#%7+50%7 3,,5#.-3#%7.2-3#3-3%+75!7+'//5$-% 7 !3$0+72.,3,7-%3$7&%.#35'$7 3,7-%$0+75!72544.&5$.-35,734%7 .#53 3,7-%7,%.-3#%730/.2-+75!7 /'&43273,-%$#%,-35,7*3%7 2$5 3,75'-7%!!%2-+(7 *(70/.2-75,7/$5 '2-73,,5#.-35,7 3+7%.7., 72.,%+7.2$5++7 +%2-5$+7 ++'0/-35,+7 7,173, 75!73,,5#.-35,73+7 55 7*3,,5#.-35,73+7,5-7 '%+-35,% (7. 32.47 3,24'+3#%75$7+'+-.3,.&4%7 3,,5#.-35,+7.$%7,5-7 3$%2-417 /'$+'% 7++'0/-35,7/543217 !$.0%5$77*+%%7/.$.$./7 ,,5#.-35,72.+%7+-' 3%+7., 7 4%++5,+(7 7,2$%.+% 72544.&5$.-35,73+7 55 7++'0/-35,7/543217 !$.0%5$77*+%%7/.$.$./7 ,,5#.-35,72.+%7+-' 3%+7., 7 4%++5,+(7 .3,74%++5,+7 7%7,%% 75!70%.+'$3,7-%7 30/.2-75,7-%7'.43-175!7"5&+7 ., 7-%70.$%-75$3%,-.-35,75!7 7%!!5$-+7 757/.17.--%,-35,7-575-%$7 5&+-.24%+7-573,,5#.-35,7 &%15, 7!3,.,23.474303-+7 7,,5#.-35,+7., 7 2544.&5$.-35,7%!!5$-+725'4 7&%7 -.$%-% 7-5.$ +73,24'+35,7 ., 7+'+-.3,.&343-17 %4%#.,2%7-57.43,7 !', 3,77/$5$.0+7 /'$/5+%+73-7 %0/4510%,-73, 32.-5$+7 *'.,-3-.-3#%7., 7 '.43-.-3#%(7 7 %% 7-55+-%$743,.%+7 &%-%%,7 7,,5#.-35,7., 7 3,,5#.-3#%72'4-'$%7 75$7 %#%45/0%,-7 +'+-.3,.&343-17 7-$'2-'$.47., 7 -$.,+!5$0.-3#%7 /54323%+7 ,730/5$-.,-7/.$-75!7 -%2,54532.472.,%7 ., 73,,5#.-35,7250%+7 !$507,5,7 .2-3#3-3%+7 7 654323%+70'+-725,+3 %$7 3!!%$%,2%+7.++523.-% 7-57 +3%7., 7+%2-5$.47 2.$.2-%$3+-32+7 7 6$505-35,75!73,,5#.-3#%7 2'4-'$%73+7%++%,-3.47 ,%'.4730.3,.$3%+75!7 3,,5#.-35,7*-%2,5 5/-303+-327#+7 /%++303+-327.2$5++7 %0/4510%,-74%#%4+(7 ,%% 7-57&%725,+3 %$% 7 75#%$,.,2%73+7 3,-%$-3,% 73,7.7250/4%7 /543217+/.2%7-73+7 &5', % 73,7-%$0+75!7 76543217037 * 3!!%$%,-7 /$5$.00%+7., 7 /543217 50.3,+(7 7.1%$+7*+-$.-%327 0.,.%0%,-7., 7 /%$!5$0.,2%(7 7%#%4+7 *3,-%$,.-35,.47 ,.-35,.47$%35,.47 ., 7452.4(7 7 %% 7-57%,25'$.%7 765432174%. %$+3/7 7.-35,.47., 7 $%35,.47255$ 3,.-35,7 %7!5$0.43+.-35,75!7 &5 3%+7., 7255$ 3,.-35,7 0%2.,3+07%3,!5$2%7 1&$3 7&5 3%+7*+%%7 /.$.$./7,,5#.-35,7 2.+%7+-' 3%+7., 7 4%++5,+(7 7/5432173,,5#.-35,7 ., 7+-.%54 %$+7 3,#54#%0%,-73+7 %4250% 7., 725'4 7&%7 %,25'$.% 7 10/.-,+*)(+'&)/.%()$.#)"+!+ ..-(//!-*&-'+.+(+/-.+/!.),-()....#.+/!.-(//!-*&-'+.+  *  . Table 2. Continued from previous page. Focus Innovation and firms’ Innovation and the Innovation and the Innovation and (Project) performance workers perspective strategic management regional perspective (CDTI) (CC.OO.) perspective (RIS3) (APD) volvement). However, it has also elements from previous policy frameworks, such as, the focus on firms and economic growth and a kind of traditional approach to stakeholders’ involvement. Therefore, some limitations found in previous exercises apply also here. For example, it is necessary to work on different conceptualizations of innovation. However, this monitoring exercise shows the importance of institutional and governance factors in explaining innovation performance. Conclusions This article has reviewed R&I policies in Europe considering the challenges that Europe face. The increasing international competition and inequality of R&I European systems require more ambitious R&I policies. We have seen that there is an increasing demand for R&I policies to move from previous rationales (innovation for growth or the strengthening of the national systems of innovation) towards transformative changes. However, this third policy framework is still nascent. The review of the evaluation and monitoring exercises indicates that the objectives aimed by transformative policies are practically absent in Spain, with policy frameworks 1 (innovation for growth) and 2 (national and regional systems of innovation) dominating. Therefore, it is clear the need of innovation policy design to be substantially modified towards sustainability and reducing inequality, being more ambitious: innovation policies considering innovation as an instrument of social change. We could only identify some characteristics of the transformative framework in the policy approach of Smart Specialisation (S3) in its attempt to include stakeholders and its placed-based approach to innovation. However, this is a European innovation policy exercise that has been implemented very differently across regions, varying according to leadership, governance structures and coordination mechanisms. We have shown that traditional impact evaluation exercises show the importance of considering different type of additionalities, while considering factors, such as, sector and size. The thoroughness of its methodological approach in analysing causal effects is also a strength of traditionally oriented impact evaluation analysis. The surveybased studies on innovation have shown the importance of considering, the quality of employment and other institutional and cultural factors when analysing R&I. Similarly, non-R&D innovation appeared to be crucial to understand innovation for less innovative actors and regions. Different conceptualizations of innovation and collaboration need to be considered. Therefore, it is important to align traditional innovation purposes with other social needs as employment level and quality, sustainability and the promotion of innovative culture, transparency, tolerance and eliminate hampering elements (uncertainty, resistance to change or excessive risk aversion, for instance). For this to be carried out successfully, two basic conditions must be met: on the one hand a new design of the concrete actions, much more transversal and oriented to the resolution of critical problems. In line with the above, this implies that policies «for» innovation are more open to the inclusion of other incentives and objectives to be achieved. On the other hand, it requires a practice of evaluation of these policies that include both the traditional concrete impact measurements and other forms of ex ante evaluation and open monitoring exercises that allow us to see the pros and cons of the proposed actions and the capacities of the administration to carry them out. In any evaluation, it is also necessary to check whether public aid has negative effects. These negative effects include concerns about whether aid distorts the market. However, these also need to be open to considering other beneficiaries and losers, making R&I policies more reflexive. These insights on innovation research follow the lessons of Freeman and Soete (2009) that pointed that “the center of the debate is not the impact of technology transfer on economic development, but rather the organizational, economic and social insertion of such technologies in a development environment and how that facilitates or blocks specific development and growth opportunities” (Freeman and Soete, 2009). This is an open and complex approach to the innovation process that it’s still mainly absent in the Spanish policy exercises considered here. An approach that needs to be complemented with different and more recent views to obtain more sustainable, equal and fairer results from innovation an innovation policy. Acknowledgments We thank all participants in the different studies and the anonymous reviewers for their constructive comments. This work was partially supported by: the evaluation contract «Impact evaluation of the R&D programme grants of CDTI (20122020)» authorized by the European Commission Decision (2015) 4147 final, 22 June 2015, evaluation done in collaboration with Novadays; the contract service (art. 83) code 4157629 with CC.OO “The impact evaluation of innovation public aid to firms on employment and working conditions” (2019-2020); and the collaboration agreement between the UCM and the APD for the valorisation of innovation (2022-2023). The authors would especially like to thank the personnel of the Spanish National Institute (INE) and CDTI for their support and technical cooperation. We also like to thank to Jaime Laviña, Irene Laviña, Saraí López and José María Insenser for their collaboration in some of the case studies. The authors would like to thank the anonymous reviewers for their helpful comments and feedback. References Arrow K. (1962). Economic welfare and the allocation of resources for invention, pp. 609-625. In: Nelson, R. (Ed.), The rate and direction of inventive activity. Princenton , National Bureau of Economic Research and Princeton University Press. Bauer P, Fedotenkov I, Genty A, et al. (2020). Productivity in Europe – Trends and drivers in a service-based econom. EUR 30076 EN. Luxembourg, Publications Office of the European Union. Busom I (2000). An empirical evaluation of the effects of R&D subsidies. Econ Innov New Techno 9:111-148. CESIN-APD (2022). [La innovación en la empresa española]. CESIN-APD. Available from: https://www.apd.es/estudiocesin-apd-innovacion-empresa-espanola/ Cunningham P, Gök A, Laredo P (2016). The impact of direct support to R&D and innovation in firms, pp. 54-107. In: J. Edler, P. Cunningham, A. Gök, P. Shapira (eds.), Handbook of innovation policy impact. Cheltenham, Edward Elgar Publishing. Dimos C, Pugh G (2016). The effectiveness of R&D subsidies: a meta-regression analysis of the evaluation literature. Res Policy 45:797-815 Dosi G, Llerena P, Labini MS (2006). The relationships between science, technologies and their industrial exploitation: An il- [Proceedings of the European Academy of Sciences & Arts 2025; 4:52] [page 59] Article