Evaluating the impact of public financial support on innovation activities of European Union enterprises: Additionality approach

Lewandowska, Małgorzata; Weresa, Marzenna Anna; Rószkiewicz, Małgorzata Article Evaluating the impact of public financial support on innovation activities of European Union enterprises: Additionality approach International Journal of Management and Economics Provided in Cooperation with: SGH Warsaw School of Economics, Warsaw Suggested Citation: Lewandowska, Małgorzata; Weresa, Marzenna Anna; Rószkiewicz, Małgorzata (2022) : Evaluating the impact of public financial support on innovation activities of European Union enterprises: Additionality approach, International Journal of Management and Economics, ISSN 2543-5361, Sciendo, Warsaw, Vol. 58, Iss. 3, pp. 248-266, https://doi.org/10.2478/ijme-2022-0020 This Version is available at: https://hdl.handle.net/10419/309762 Standard-Nutzungsbedingungen: Die Dokumente auf EconStor dürfen zu eigenen wissenschaftlichen Zwecken und zum Privatgebrauch gespeichert und kopiert werden. Sie dürfen die Dokumente nicht für öffentliche oder kommerzielle Zwecke vervielfältigen, öffentlich ausstellen, öffentlich zugänglich machen, vertreiben oder anderweitig nutzen. Sofern die Verfasser die Dokumente unter Open-Content-Lizenzen (insbesondere CC-Lizenzen) zur Verfügung gestellt haben sollten, gelten abweichend von diesen Nutzungsbedingungen die in der dort genannten Lizenz gewährten Nutzungsrechte. Terms of use: Documents in EconStor may be saved and copied for your personal and scholarly purposes. You are not to copy documents for public or commercial purposes, to exhibit the documents publicly, to make them publicly available on the internet, or to distribute or otherwise use the documents in public. If the documents have been made available under an Open Content Licence (especially Creative Commons Licences), you may exercise further usage rights as specified in the indicated licence. https://creativecommons.org/licenses/by-nc-nd/4.0/ International Journal of Management and Economics 2022; 58(3): 248–266 Małgorzata Stefania Lewandowska1,*, Marzenna Anna Weresa2, Małgorzata Rószkiewicz3 Evaluating the impact of public financial support on innovation activities of European Union enterprises: Additionality approach https://doi.org/10.2478/ijme-2022-0020 Received: December 30, 2021; accepted: September 30, 2022 Abstract: The study aims at estimating the effects of support for research and development and innovation from the European Union (EU) budget for boosting innovation in European enterprises, using input, output, and behavioral additionality approach. The study is based on microdata of the Community Innovation Survey 2012–2014, covering a sample of 98,809 enterprises from 14 EU countries. The direct and indirect relationships between the variables were studied using path analysis. For the whole sample, three additionality dimensions were confirmed; however, the result differs across EU member states. Multi-additionality of EU grants was confirmed only for Spain; in eight EU countries, input and behavioral additionality were proved, and in two, only behavioral additionality was found. This leads to the conclusion that the potential of EU support is not fully exploited, in particular in Central and Eastern European countries, and there is room for improvements with regard to policy design and implementation. Keywords: additionality of public support, European Union, innovation, innovation policy, R&D grants JEL Classification: C50; L2; O3 1 Introduction Innovation is an important factor for tackling the major societal challenges such as climate change, scarce natural resources, or aging of society. To address these challenges, a holistic approach to innovation policy is required [Bengtsson and Edquist, 2022]. Therefore, it is important to study additionality effects of different public support instruments to design innovation policy mix tailored to the problems to be solved. In this study, we focus on research and development (R&D) grants offered to European firms from the European Union (EU) budget. The rationale for government intervention in the field of R&D and innovation (R&D&I) is Empirical Paper Open Access. © 2022 Lewandowska et al. published by Sciendo. This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 License. *Corresponding author: Małgorzata Stefania Lewandowska, Collegium of World Economy, SGH Warsaw School of Economics, Warsaw, Poland. E-mail: m[email protected]w.pl; ORCID: https://orcid.org/0000-0002-5778-8009 Marzenna Anna Weresa, Collegium of World Economy, SGH Warsaw School of Economics, Warsaw, Poland. ORCID: https://orcid.org/0000-0003-3112-3460 Małgorzata Rószkiewicz, Collegium of Economic Analysis, SGH Warsaw School of Economics, Warsaw, Poland. ORCID: https://orcid.org/0000-0002-5859-9904 Public financial support on innovation activities of European Union 249 to reduce market failures (e.g., information asymmetries and coordination failures) that impede enterprise innovation activity, minimize the negative effects of technology on society, coordinate interdisciplinary research, or reduce the costs and risks of creating innovations required for the development of economywide infrastructure [Atkinson and Ezell, 2012; Rothwell and Zegveld, 1982, pp. 9–10]. However, public involvement is appropriate only when the market cannot fulfil the country’s strategic goals and intervention can help alleviate or minimize current problems [Edquist, 2011; Edquist and Zabala-Iturriagagoitia, 2020]. Well-designed innovation policy tools should provide some additionality impacts while not replacing, duplicating, or crowding out activities that firms can carry out on their own [Edquist, 2019]. Thus, the idea of additionality assumes that as a result of public funding supporting firms’ R&D&I additional business R&D investment occurs, and new products or business processes will be introduced on the market. However, the results of empirical studies on this topic are mixed. A comprehensive review of 98 empirical studies published between 1960 and 2017 on the effectiveness of public support for R&D&I in the EU, China, and Taiwan confirmed that the impact can be positive or negative, depending on firm size, sectors, the source of the money (international, national, or regional), the type of the support scheme, and how the scheme fits with other elements of the innovation system [Petrin, 2018]. The most recent research on the additionality of R&D has also demonstrated that the impact differs between industries, countries, and additionality types. A study on R&D subsidies for patenting activity of enterprises in Norway validated output additionality, demonstrating that its quantity is strongly dependent on firm characteristics [Bye et al., 2019]. By contrast, Nilsen et al. [2020] showed in their research on Norway that the output additionality of this support supplied to incumbent enterprises performing regular R&D activity was statistically insignificant, which mattered for R&D starters. Grabowski and Staszewska-Bystrova [2020] discovered that the influence of public support on innovation varies between EU member states and sectors. Some studies also showed that the approach employed in modeling the influence of policy measures on innovation matters for the results obtained as no model is flawless [e.g., Annicchiarico et al., 2020; Benedetti et al., 2020]. The examples of fragmented and mixed results on additionality given above indicate that further empirical research is needed to compare the additionality effects of financial public support for innovation at the business level. This work attempts to address this research gap by providing new evidence on all categories of additionality (input, output, and behavioral) in 14 EU countries. There are three dimensions of the impact of the public support to innovation analyzed in the study: (1) change in input for innovation (measured by additional money invested in R&D or in a purchase of a new machinery), (2) increase in the innovation output (measured by the proportion of turnover coming from new and significantly improved products), and (3) behavioral change leading to innovation (measured by enterprises’ cooperation in innovation activity and additional training of their employees that would not appear without public support). These three dimensions are defined in the literature as additionality effects of public support. They are analyzed in this study in order to find out whether and how the R&D grants received by European enterprises from the EU budget impact their innovation activity. The total initial sample of N = 98,809 covered 15 countries: Bulgaria (BG), Cyprus (CY), the Czech Republic (CZ), Germany (DE), Estonia (EE), Greece (EL), Spain (ES), Croatia (HR), Hungary (HU), Lithuania (LT), Latvia (LV), Norway (NO), Portugal (PT), Romania (RO), and Slovakia (SK). This comparative analysis was carried out based on firm-level data derived from the Community Innovation Survey (CIS) conducted for the period of 2012–2014. The article is divided into four sections. Following the Introduction, the concept of additionality of public support to R&D&I is explained on the basis of the relevant literature. Then, distinct additionality aspects are explained, and the results of prior empirical research on this topic are reviewed to generate hypotheses on potential additionality effects of R&D&I support to business innovation activity coming from the EU budget. Following the literature review, there is a description of the firm sample and a summary of the methodology applied in this study. The path analysis is carried out independently for each sample of European firms. Following that, hypotheses are tested, and empirical findings are presented and discussed. The final section of the study draws conclusions, which provide new insight into the evaluation of the success of EU support to R&D&I as well as policy implications. 250 M. S. Lewandowska et al. 2 Literature review and research hypotheses Schot and Steinmueller [2018] distinguished three approaches to science, technology, and innovation policy: the first is based on neoclassical assumptions about the linear relationship between R&D&I, the second is based on the introduction of the innovation system concept in innovation policy design, and the third is based on the need to address social and environmental challenges. Nonetheless, all approaches to innovation policy recognize that it is a component of economic policy aimed at stimulating the development of new products, processes, or services, as well as their implementation and widespread adoption [Edler et al., 2013; Weresa, 2017; Borrás and Edquist, 2019; Geels, 2020]. Innovation is required to increase enterprise efficiency and improve economic growth for the benefit of society as a whole [Crépon et al., 1998; Van Leeuwen and Klomp, 2006]. The system of support for innovative activities includes a number of instruments, such as R&D grants, innovation subsidies, tax credits, public procurement rules, and support measures for intellectual property exploitation. A two-dimensional taxonomy of innovation policy instruments identifies two major categories: those geared for supply-side interventions and those designed for demand-side interventions [Edler et al., 2013, p. 13]. Becker [2015] mapped public innovation support across Europe, highlighting disparities between EU countries with regard to instruments used as well as political levels of support [Becker, 2015]. Therefore, the efficiency of public support for innovation may be investigated while taking into account a variety of policy support instruments as well as the different levels at which the support is provided. This article considers two major features of the influence of public financial support to the innovation activities of EU enterprises: the diversity of impact areas (type of additionality) and the diversity of impact levels (direct and indirect impact). We examine the financial assistance received by European businesses from the EU level, taking into account three major areas of impact (additionalities) identified in the economic literature: input, output, and behavioral additionality. Input additionality occurs when public funds transferred to enterprises in a form of a subsidy or grant result in additional corporate R&D expenditures [Georghiou, 2004]. However, it does not happen automatically. Government R&D funding may also crowd out private R&D spending. This suggests that private R&D spending may be reduced as a result of the public subsidy or grant [Edquist et al., 2004]. Different scholars have studied the issue of input additionality [e.g., David et al., 2000; Clausen et al., 2007; Marzucchi and Montresor, 2012; Christensen et al., 2018; Czarnitzki, and Hussinger, 2018], but the results vary for different countries and also depend on the source and type of support, as well as industry. Hussinger [2003] investigated German corporations and found no evidence of crowding out impacts of governmental R&D subsidies on firms’ R&D expenditures. On the contrary, the input additionality was validated. Czarnitzki and Hussinger [2018] obtained similar results by using data from the Mannheim Innovation Panel and conducting a treatment analysis to assess for crowding out effects of public R&D granted by the German Federal Government to over 500 grantees from 1992 to 2000. The impacts of full and partial crowding out, as well as acceleration, were investigated. The key finding was that supported enterprises expanded their volume of R&D spending by using their own resources and raising additional private capital. Gonzalez et al. [2005] conducted a large-scale study in which they examined a panel of over 2,000 Spanish enterprises and concluded that public subsidies stimulate R&D expenditures, but they also demonstrated that innovation projects could have been continued without public support. Another study on Spanish enterprises by Afcha and Lucena [2021] used data from the Spanish Business Strategy Survey also confirmed input additionality showing that public support induced additional private R&D effort. When it comes to the source of public assistance, Grabowski et al. [2013] conducted an interesting comparative study on manufacturing businesses from Turkey and Poland estimated an econometric model developed by Crépon, Duguet and Mairesse (so called CDM model) for the CIS micro-data for the period 2008–2010. Input additionality was confirmed at the national and EU levels for both countries. Grabowski and Staszewska-Bystrova [2020] discovered industry and country disparities in the impact of R&D&I funding granted to European small and medium sized enterprises (SMEs) from the EU and national budgets Public financial support on innovation activities of European Union 251 using CIS 2014 data in a multivariate, multi-stage econometric model. The input additionality of EU public support for R&D&I was confirmed for a group of “new” EU member states, while output additionality was revealed in the “old” EU countries and Norway. In addition, the additionality effects differed between the manufacturing and service sectors. The authors also demonstrated that simultaneous assistance for R&D&I from both the EU and state budgets resulted in a higher beneficial impact. In turn, Czarnitzki and Lopes-Bento [2014] found that for German enterprises, national and European policy instruments complement each other. Czarnitzki and Delanote [2015] also studied smalland medium-size German enterprises that received grants from the EU. The full crowding out effect was not confirmed for all studied firms, and the input additionality was the strongest in high-tech young enterprises. Alecke et al. [2012] analyzed the impact of public subsidies on firms in East Germany and confirmed both input and output additionality. Some research have found that public financial support to innovative activities has a crowding out impact. It was discovered for instance for big Spanish firms [Serrano-Velarde, 2008]. The crowding out effects of public assistance were also confirmed in French enterprises that received public assistance for their innovation activity between 1993 and 2009 [Marino et al., 2016]. Wallsten [2000] investigated a sample of American businesses and demonstrated the crowding out impacts of publicly financed R&D funds. However, some favorable benefits of the investigated programs were discovered. Additional R&D funding supported the continuation of higher levels of R&D spending, which would not have been possible in the long run. The review of empirical research on input additionality conducted above shows that the results regarding input additionality are mixed. Based on the literature review, we formulate the following input additionality hypothesis: H1: Public financial support from the European Union programs stimulates directly firms’ R&D budget. Another aspect of public support for innovation is output additionality, which means that grants or subsidies have a direct positive impact on the degree of firm’s innovation [OECD, 2006; Edler et al., 2013]. Such direct effects are typically measured by new products, new processes, patents, and publications, but some indirect output effects of publicly supported projects, such as improvements in financial standing and performance caused by the introduction of new products or processes, can also be observed [Georghiou, 2002]. Previous studies on this topic yielded a variety of results, ranging from significant innovation output increases caused by public R&D subsidies to lack of evidence on output additionality or even confirming crowding out effects (decreased innovative output) resulting from public R&D grants. The output additionality was discovered in German firms, but not in innovative young enterprises [Schneider and Veugelers, 2010]. Roper and Hewitt-Dundas [2016] examined the influence of state subsidies on the innovative activity of 3,254 Irish manufacturing facilities from 1991 to 2011, confirming output additionality. Similar results were obtained for Hungarian enterprises [Halpern, 2010]. The positive influence of government subsidies on the number of new innovations introduced has also been confirmed for Spanish firms [Albors-Gorrigos and Barrera, 2011]. Radas et al. [2015] investigated the effects of two innovation policy instruments in Croatia, namely, direct R&D subsidies and tax breaks for SMEs. Based on a sample of 700 Croatian enterprises that received public subsidies or tax breaks between 2005 and 2010, they discovered that both policy instruments boosted sales achieved by the firms studied. Bronzini and Piselli [2016] validated the output additionality of Italian companies’ patenting activity by demonstrating that government subsidies boosted the likelihood of applying for patents in the case of small enterprises. A study of German enterprises’ patenting activity revealed that publicly funded R&D had a beneficial impact on patents. Furthermore, the positive benefit of additional R&D caused by government funding has been demonstrated [Czarnitzki and Hussinger, 2018]. The effectiveness of R&D grants depends on the source of funds. According to the findings of a study conducted on Austrian enterprises between 1998 and 2000, support from national public sources boosted 252 M. S. Lewandowska et al. the share of sales from innovative items, while EU support was shown to be insignificant [Garcia and Mohnen, 2010]. Weresa and Lewandowska [2014] obtained similar results for public support from the EU budget received by enterprises in Poland. Structural equation modeling (SEM) of CIS 2010 data revealed that EU-funded R&D&I initiatives had no direct effects on Polish companies (i.e., output additionality measured as a share of sales of innovative items in total sales did not increased due to the EU grants). Grabowski et al. [2013] studied the effects of public support in Poland and Turkey and also found that the source of grants matters. Output additionality occurred exclusively for the national government and EU support, but not for local government support. In turn, Becker [2015] discovered a negative influence of public innovation assistance on turnover changes, but he also observed some favorable benefits on labor productivity. Freitas et al. [2017] demonstrated a boundary perspective on output additionality. The research looked at businesses in three European countries: France, Italy, and Norway. The authors demonstrated both input and output additionality using data from three waves of the CIS (2002–2004, 2004–2006, and 2006–2008), as well as confirming that the higher the level of technology, the stronger the effects emerged. However, the results varied depending on the country studied. There are significant disparities in the results when it comes to the source of money. Radicic and Pugh [2017], based on a sample of smalland medium-sized companies from 28 European countries, claimed no evidence of output additionality from national programs and crowding out impact from EU programs, which may be mitigated by combining them with national programs. Similarly, neither input nor output effects were discovered for smalland medium-sized businesses from six Western Balkan nations [Orlic et al., 2019]. As the results of empirical research are again inconclusive, we will test the following research hypothesis about output additionality: H2: Public financial support from the EU programs causes a direct improvement of firms’ innovation performance. The third dimension of public support studied in this work is behavioral additionality as programs supporting R&D&I offered at the European level are frequently designed to facilitate the establishment or continued development of collaboration in innovation activity and the formation of pan-European networks [Buisseret et al., 1995]. Behavioral additionality also refers to the effects of policy interventions such as new competencies gained by firm personnel, changes in working procedures, building new networks, and entering new business areas [Neicu et al., 2016, p. 101]. As the variety of behavioral effects may occur as a result of public support to R&D&I, it inspired many scholars to distinguish sub-categories of this phenomenon. Roper and Hewitt-Dundas [2016, pp. 12–17] distinguished three types of behavioral additionality: congenital additionality, which is related to employee education and experience; inter-organizational additionality, which is related to new innovation linkages created by public support schemes; and experiential additionality, which is related to changes in firm processes and routines induced by public support for innovation, and they tested them empirically. The study of these three behavioral effects of public subsidies, conducted over a 20-year period for a large sample of 3,254 Irish manufacturing firms revealed that prior public support for new product development activity enhanced technical skills, confirming congenital additionality. Prior public subsidies were also significant for the positive benefits of collaboration in innovative activity, resulting in inter-organizational additionality. However, no evidence of experiential additionality was found [Roper and Hewitt-Dundas, 2016]. All in all, behavioral additionality is more difficult to trace than input or output additionality as it covers many different elements characterizing firm’s innovation capabilities and is not easy to measure with quantitative indicators. Apart from cooperation and training, the behavioral additionality includes learning externalities (sometimes distinguished as project additionality) or some scale effects, such as undertaking of more ambitious projects [Weresa et al., 2018, p. 20]. Empirical results of research on behavioral additionality are inconclusive. Some scholars—for example, Roper and Hewitt-Dundas [2016] already referred to the previous aspect—proved the significance of public subsidies for innovation for behavioral changes related to education and cooperation, not fully Public financial support on innovation activities of European Union 253 confirming experiential additionality connected with changes of processes and routines. Furthermore, they confirmed “the legacy effects” of public support taking into account subsidies received by firms in the previous period. Many studies concentrated on the outcomes of public R&D&I assistance for changes in companies’ relationships with the research sector, suppliers, clients, and competitors. The idea of collaboration additionality of public support was validated for technologically specialized enterprises in the Austrian transportation sector with relatively moderate R&D intensity [Wanzenböck et al., 2013]. The scope of collaboration in innovation, as well as the scale of innovative projects, underwent behavioral change in these sponsored firms. Weresa and Lewandowska [2014] demonstrated, using data from the CIS 2010 for Poland, that both domestic public assistance and EU-funded projects resulted in behavioral additionality, as evaluated by collaboration with institutional partners. Lewandowska and Kowalski [2015] demonstrated the behavioral additionality of EU financial support for Poland, but only for major firms grouped in clusters and cooperating with partners from the same clusters, not with partners from outside. Another study on Poland based on micro-data from 2010 indicated no behavioral additionality [Szczygielski et al., 2017], leading to the conclusion that EU grants for human capital development were ineffective in encouraging innovation in Poland. In turn, the evidence for behavioral additionality in Asian countries is equivocal. For example, public R&D&I support schemes enabled South Korean biotechnology firms evaluated by Kang and Park [2012] to collaborate more actively in innovation activity with their suppliers and clients. Another study on biotechnology Korean corporations verified behavioral additionality, finding that firms backed by public R&D subsidies engaged in more strategic alliances than non-subsidized firms [Shin et al., 2019]. Behavioral additionality was also investigated for individual R&D programs and again results varied depending on the program type and country studied. A network study was conducted on a new program in the United States supported by the Alfred P. Sloan Foundation, which was aimed at incubating a new scientific subject “Microbiology of the Built Environment.” The results revealed an increased density and networks among researchers, as well as a higher intensity of collaboration within this research community, thus confirming behavioral additionality [Hicks et al., 2019]. Another program-level study obtained opposite results. The ineffectiveness of Turkey’s Priority Areas R&D Grant Program in terms of output, input, and behavioral additionalities was shown, and a stronger prioritization and coordination of national macro-, meso-, and micro-level policies were advised [Gürbüz and Erdil, 2020]. As existing empirical studies show ambiguous results regarding the impact of public innovation support on behavioral change in different countries, the third hypothesis concerning behavioral additionality is formulated: H3: Public financial support from the EU programs intensifies firms’ activity in cooperation with external partners and personnel training. The literature confirms that the public financial support can enhance innovation performance both directly and also in an indirect manner. In order to test this effect, we first suppose the direct connection between knowledge acquisition and innovation performance, as well as cooperation and training and innovation performance and test a mediating role of both knowledge acquisition and cooperation and training. Wubben et al. [2015] presented some intriguing findings regarding the effects of receiving knowledge from outside sources. According to the authors, knowledge exploration techniques have long-term consequences and produce radical discoveries, whereas knowledge exploitation tactics provide incremental innovations. Based on data from the Danish CIS, the authors claim that companies that use licenses develop incremental innovations, whereas radical innovations emerge as a result of enterprise acquisitions and mergers. The role of knowledge-sourcing activities for effectiveness of R&D subsidies was also researched by Afcha and Lucena [2021] for a sample of 1,178 manufacturing firms from Spain. The authors provided evidence that R&D subsidies for external knowledge drive firms innovation. 254 M. S. Lewandowska et al. Van Beers and Zand [2014] investigated 12,811 innovative firms that took part in the Danish version of the CIS, as well as the R&D and Information and Communication Technology Surveys, between 1994 and 2006. They demonstrated, using tobit analysis for a panel of businesses, that diversifying collaboration partners in innovations has a synergistic impact for the introduction and commercialization of innovations. Simultaneously, geographical diversification of collaboration partners promotes adaptation of innovations to market requirements such as technical standards, market restrictions, or buyer preferences. Based on these results, supposing the positive link between the knowledge acquisition as well as the positive influence of cooperation on the innovation performance of enterprises, we post the last hypothesis: H4: Public financial support from the EU programs intensifies firms’ innovation performance indirectly via the knowledge acquisition (H4a), and indirectly, through cooperation and personnel training (H4b). All hypotheses gathered in the theoretical model are presented in Figure 1. While taking into account the three dimensions of the influence of innovation policy mentioned earlier (input, output, and behavioral), we can determine both the strength of impact on each of the three dimensions and the degree of complexity of impact. The effect of public financial support for innovative activities may take the following forms [Marzucchi and Montresor, 2012]: 1. Multidimensional (systemic) support: This is the case in which support takes place in the case of R&D expenditure (i.e., input), outcomes, and the behavior of the supported entity. 2. Bi-dimensional support: It is a situation in which the additional effect occurs in two of three dimensions. 3. Mono-dimensional support: This is the case in which the additional effect can be observed in only one dimension. 4. Partial crowding out effect: This is a situation in which we deal with the effect of crowding in one or two dimensions with a positive effect in the case of others. 5. Total crowding out effect: A situation in which there is a displacement effect for all three dimensions. 3 Research methodology 3.1 Context of the research The quantitative analysis is based on anonymous firm-level micro-data from the CIS, covering years 2012– 20141 from selected EU member states. 1 CIS micro data obtained based on the “Contract on the use of Community Innovation Survey (CIS) micro data for research purposes 148/2016-CIS” signed with European Commission Eurostat, Unit B1 – Quality, methodology and research. Figure 1. Theoretical model of the impact of public financial support on different activities of treated enterprise. Source: Own elaboration. Public financial support on innovation activities of European Union 255 The CIS is a survey on innovation activity of enterprises covering EU member states and candidate countries, Iceland and Norway, based on a common survey questionnaire and methodology, with reference to the Oslo Manual, ed. 2005. The CIS is designed to obtain information on firms’ innovation activities, and it also contains data on the introduction of organizational and marketing innovations. The target population are small, medium, and large enterprises from NACE (Nomenclature statistique des Activités économiques dans la Communauté Européenne) sections A to N. Research for CIS purposes is carried out by companies quoted in the official registers of enterprises kept by state statistical offices or other organizations recognized for this purpose by state authorities in the countries that provide data for CIS purposes. Surveys may be conducted on the total population of businesses, on randomly chosen samples, or using a combination of the two methods [CIS, 2008]. Stratification of the studied set is performed in accordance with the following factors: type of conducted activity (according to NACE Rev.2, indicating the nature of the activity), size of the employment (from 10 to 49 employees; from 50 to 249 and more people), and regional location. National innovation research institutes are free to choose a sample of enterprises to be surveyed. The principle of pursuing the most accurate outcome without distorting the true picture of the issue serves as the foundation for research on trials. Data for this research were obtained based on the individual research proposal submitted to Eurostat. It has to be remembered, however, that although the CIS questionnaire covers all 28 member states and candidate countries, not all of them revealed data for the research purposes on the micro-level, which is why the full coverage of all EU countries cannot be expected. Also, the release of the subsequent CIS waves for research purposes has a strong delay compared to the releases of aggregated CIS data. 3.2 Data collection and sample characteristics The total sample of N = 98,809 covered 15 countries: Bulgaria (BG), Cyprus (CY), the Czech Republic (CZ), Germany (DE), Estonia (EE), Greece (EL), Spain (ES), Croatia (HR), Hungary (HU), Lithuania (LT), Latvia (LV), Norway (NO), Portugal (PT), Romania (RO), and Slovakia (SK). (See Table 1. for details.) The share of EU new member states (mostly CEE countries) amounted to 47.8% of the initial sample. Table 1. Initial sample description Country Sample Sample split BG 14,255 14.4 CY 1,346 1.4 CZ 5,198 5.3 DE 6,282 6.4 EE 1,760 1.8 EL 2,507 2.5 ES 30,333 30.7 HR 3,265 3.3 HU 6,817 6.9 LT 2,421 2.5 LV 1,501 1.5 NO 5,045 5.1 PT 7,083 7.2 RO 8,206 8.3 SK 2,790 2.8 Total 98,809 100.0 BG, Bulgaria; CY, Cyprus; CZ, Czech Republic; DE, Germany; EE, Estonia; EL, Greece; ES, Spain; HR, Croatia; HU, Hungary; LT, Lithuania; LV, Latvia; NO, Norway; PT, Portugal; RO, Romania; SK, Slovakia; CIS, Community Innovation Survey. Source: Own calculations based on micro-data from CIS 2012–2014. In this context, the question arises about the reasons of these differences between the European countries in terms of the scale of the additionality effects. In majority of studied countries, only bi-dimensional additionality related to the impact of R&D public grants on knowledge acquisition and personnel training occurred, while no increase in the innovation output was confirmed. A few following reasons derived from the literature for this can be considered, indicating new avenues for future research on this topic: • TimelagbetweenR&Dgrantsandtheireffects[Dimosetal.,2021]. There is a time lag between the grant absorption and output. The study on U.K. firms found that output additionality occurs at least 3 years after the receipt of R&D grants, while input and behavioral additional effects may be seen much earlier [Dimos et al., 2021]. This temporal effect of R&D grants may be one of the reasons why output additionality occurred only in one studied country. Therefore, provided new data availability, it could be interesting to conduct a similar study after a few years taking into account a time lag between grant receipt and expected output additionality effects. • R&Dprogramcomplementarityandcriticalmassargument[RadicicandPugh,2017]. As the literature suggests in countries where European, national, and regional/local programs complement each other, stronger additionality effects can be expected due to synergies that may be achieved, thanks to combining funds from different sources as well as due to achieving critical mass in fund absorption [Czarnitzki and Lopes-Bento, 2014; Radicic and Pugh, 2017]. In this work, we studied the EU grants, but we did not look at the effects of national or regional grants. In some countries (e.g., Spain), funding from national and regional budgets plays a much more important role than in other countries (e.g. the Czech Republic) [Weresa and Lewandowska, 2014]. Furthermore, there are differences between the analyzed countries in their innovation performance (e.g., Spain and the Czech Republic were classified as moderate innovators, while Latvia, Romania, and Bulgaria as modest innovators; see IUS, 2014, p. 5). Thus, innovation policy design and implementation combined with current innovation performance may have an impact on the real absorption of EU funds. • Firmcharacteristics:Internalfinancialpotential,absorptivecapacity,andexperienceinmanagingEU grants by the recipients [Bye et al., 2019; Nilsen et al., 2020]. Differences in firms characteristics in European countries may explain Spanish firms’ success in delivering multidimensional additionality from public support as they have longer experience than firms from Central and Eastern Europe in applying and utilizing R&D grants from the EU to support their innovation activity. • DesignofR&Dgrantsassupply-sidepolicyinstrumentsfocusedondirectsupportofinnovationinput (i.e., R&D efforts) and supporting innovative output (i.e., introduction of new products and processes on the market) indirectly only [Edler et al., 2013, Becker, 2015]. Our results indicate that the simple policy instruments, such as grants or subsidies, may not be fully effective, at least in a short run in supporting innovation output. R&D grants can be compared to public procurement. Therefore, there is a need for new and more sophisticated policy tools that can be offered to enterprises at the European level. Albors-Garrigos and Barrera [2011] measuring the impact of European subsidies speculated that the lack of their impact results from their influence on pre-commercial research, rather than commercialization. As the EU grants for R&D&I differs across EU member states, it can be concluded that there is still room for improvements with regard to European innovation policy design and implementation in order to make it more effective. It seems that a traditional approach that focused on mitigating market failures and transfer of best practices from other countries is not sufficient anymore as the policy rationales have been extended to cover societal challenges. This conclusion goes in line with a mission-oriented approach to innovation policy [Mazzucato, 2018] as well as with recent findings of Uyarra et al. [2020] and Geels [2020] and seem to be even more important in the turbulent times of the coronavirus disease (COVID-19) pandemic. Public financial support on innovation activities of European Union 263 This study is not without limitations. First, it must be noted that it is positioned within the science, technology, and innovation research approaches, which give main emphasis to promoting R&D and creating access to explicit codified knowledge, and does not discuss the “doing, using, and interacting” approach [Jensen et al., 2007], according to which innovation strategies are mainly based on learning and interacting. Another limitation is related to data availability. Despite the representativeness of the initial sample of firms, the extracted number of innovative firms is relatively small. In addition, the overrepresentation of the entities from Bulgaria may bias the results for the aggregated sample. It should be also noted that especially while examining the impact of public financial support, the CIS data should be used cautiously as they are anonymous, and therefore, it is not possible to conduct a follow-up survey based on more than one period of observations, which would be beneficial especially in the case of output additionality, where the effects may be postponed. We also have to mention that even though the methodical reflection on the consequences of innovation policy in the three dimensions of the additional effect is the strength of our approach, when assessing the impact of innovation policy, it is critical to also consider interactions at various levels (regional, national, and supranational), diverse support tools that interact with other tools, and the diversity of the entities supported [Chaminade and Edquist, 2010, Magro and Wilson, 2013; Guerzoni and Raiteri, 2015]. Researchers also prove that there is a direct interrelation between different types of additionality [Czarnitzki, Licht, 2006; Hegerty et al., 2022], which we have not investigated. Despite these limitations, our study is consistent with more general evidence of the positive role of cooperation on innovation performance of enterprises. A deeper understanding of the effects of innovation cooperation at firm-level and its underlying mechanisms is a prerequisite for a future design of policies fostering the cooperation-friendly environment. The constructed models, which highlight the importance of external behavioral additionality, may help policy authorities to develop a better understanding of how policy contributes to support innovation cooperation especially within transition economies that constitute for the vast majority of the sample. The analysis points to the possible fruitfulness of further research on connections between innovation performance, cooperation, and public financial support as well as on reasons concerning differences between European countries regarding additionality dimensions. This article also provides evidence for the managers of innovative firms about additional effects of public R&D funds, especially in the context of innovation cooperation, which is still limited in many European countries. References Afcha, S., Lucena, A. (2021), The effectiveness of R&D subsidies in fostering firm innovation: The role of knowledge-sourcing activities, Business Research Quarterly, Vol. 24, No. 4, pp. 302–323. Albors-Garrigos, J., Barrera, R.R. (2011), Impact of public funding on a firm’s innovation performance. Analysis of internal and external moderating factors. International Journal of Innovation Management, Vol. 15, No. 6, pp. 1297–1322. Alecke, B., Mitze, T., Reinkowski, J., Untiedt, G. (2012), Does firm size make a difference? Analysing the effectiveness of R&D subsidies in East Germany, German Economic Review, Vol. 13, No. 2, pp. 174–195. Annicchiarico, B., Licandro, O., Mohnen, P., Ortega, E., Veugelers, R. (2020), Moving the frontier of macroeconomic modelling of research and innovation policy, Luxembourg: Publications Office of the European Union, doi: 10.2777/34199. Atkinson, R.D., Ezell, S.J. (2012), Innovation economics. The race for global advantage. Yale University Press, New Haven, London. Becker, L. (2015), Effectiveness of public innovation support in Europe. Does public support foster turnover, employment and labour productivity? CAGE Discussion paper No. 236, Centre for European Governance and European Development Research, Georg-August-University Göttingen. Benedetti Fasil, C., Sanchez-Martinez, M., Ravet, J., Martino, R. (2020), Macroeconomic models for research and innovation policy: the present and the future, directorate-general for research and innovation, European Commission, Luxembourg: Publications Office of the European Union, doi: 10.2777/339051. Bengtsson, L., Edquist, Ch. (2022), Towards a holistic user innovation policy, International Journal of Innovation Studies, Vol. 6, pp. 35–52. Bollen, K.A., Stine, R.A. (1992, November), Bootsrapping goodness-of-fit measures in structural equation models, Sociological Methods Research, Vol. 21, No. 2, pp. 205–229. 264 M. S. Lewandowska et al. Borrás, S., Edquist, Ch. (2019), Holistic innovation policy. Theoretical foundations, policy problems, and instrument choices, Oxford University Press, Oxford, New York. Bronzini, R., Piselli, P. (2016), The impact of R&D subsidies on firm innovation, Research Policy, Vol. 45, No. 2, pp. 442–457. Buisseret, T., Cameron, H., Georghiou, L. (1995), What difference does it make? Additionality in the public support of R&D in large firms, International Journal of Technology Management, Vol. 10, No. 4/5/6, pp. 587–600. Bye, B., Klemetsen, M., Raknerud, A. (2019), The impact of public R&D support on firms’ patenting, Discussion Papers, No. 911, Statistics Norway, Research Department, Oslo. Chaminade, C., Edquist, C. (2010), Rationales for public policy intervention in the innovation process: systems of innovation approach, in: R.E. Smits, S. Kuhlmann, P. Shapira, (Eds), The theory and practice of innovation policy. An international research handbook, Edward Elgar, Cheltenham (UK) and Northampton, MA (USA), pp. 95–114. Christensen, C.M., McDonald, R., Altman, E.J., Palmer, J.E. (2018), Disruptive Innovation: An Intellectual History and Directions for Future Research, Journal of Management Studies, Vol. 55, Issue 7, pp. 1043–1078. Clausen, T., Nås, S.O., Verspagen, B. (2007), Norwegian Innovation and Industrial Structure: Insiders and Outsiders?, Working Papers on Innovation Studies 20070610, Centre for Technology, Innovation and Culture, University of Oslo. Community Innovation Survey (CIS) (2008), The Harmonised Survey Questionnaire, retrieved from http://innovacion.ricyt.org/ files/CIS%202006%202008.pdf [dostęp: 22nd June 2022]. Crépon, B., Duguet, E., Mairesse, J. (1998), Research, innovation and productivity: An econometric analysis at the firm level, Economics of Innovation and New Technology, Vol. 7, No. 2, pp. 115–158. Czarnitzki, D., Delanote, J. (2015), R&D policies for young SMEs: input and output effects, Small Business Economics, Vol. 45, No. 3, pp. 465–485. Czarnitzki, D., Hussinger, K. (2018), Input and output additionality of R&D subsidies, Applied Economics, Vol. 50, No. 12, p. 1341, doi: 10.1080/00036846.2017.1361010. Czarnitzki, D., Licht, G. (2006), Additionality of public R&D grants in a transition economy: the case of Eastern Germany, Economics of Transition, Vol. 14, No. 1, pp. 101–131. Czarnitzki, D., Lopes-Bento, C. (2014), Innovation subsidies: does the funding source matter for innovation intensity and performance? Empirical evidence from Germany, Industry & Innovation, Vol. 21, No. 5, pp. 380–384. David, P.A., Hall, B.H., Toole, A.A. (2000), Is public R&D a complement or substitute for private R&D? A review of the econometric evidence, Research Policy, Vol. 29, pp. 497–529. Dimos, C., Fai, F., Tomlinson, P.R. (2021), The temporal effects of R&D subsidies on R&D, innovation and innovation behaviour: evidence from UK, Academy of Management Proceedings, Vol 2021, No. 1. Edler, J., Cunningham, P., Gök, A., Shapira, P. (2013), Impacts of innovation policy: synthesis and conclusions. Compendium of Evidence on the Effectiveness of Innovation Policy Intervention Project, Manchester Institute of Innovation Research Manchester Business School, University of Manchester, retrieved from http://research.mbs.ac.uk/innovation/. Edquist, Ch. (2019), Towards a holistic innovation policy: Can the Swedish National Innovation Council (NIC) be a role model? Research Policy, Vol. 48, No. 4, pp. 869–879. doi: 10.1016/j.respol.2018.10.008. Edquist, C., Malerba, F., Metcalfe, S., Montobbio, F., Steinmueller, E. (2004), Sectoral systems: implication for European innovation policy, in: F. Malerba (Eds), Sectoral systems of innovation in Europe – Concepts, issues and analyses of six major sectors in Europe, Cambridge University Press, Cambridge, UK, pp. 427–461. Edquist, Ch. (2011), Design of innovation policy through diagnostic analysis: identification of systemic problems (or failures), Industrial and Corporate Change, Vol. 11, pp. 1–29. Edquist, Ch., Zabala-Iturriagagoitia, J.M. (2020), Functional procurement for innovation, welfare and the environment: A mission-oriented approach, Papers in Innovation Studies Paper no. 2020/1, Lund University, CIRCLE – Center for Innovation, Research and Competences in the Learning Economy, retrieved from http://wp.circle.lu.se/upload/CIRCLE/ workingpapers/202001_edquist.pdf Efron, B. (1979), Bootstrap methods: another look at the Jackknife, The Annals of Statistics, Vol. 7, No. 1, pp. 1–26. Freitas, I.B, Castellacci, F., Fontana, R., Malerba, F., Vezzulli, A. (2017), Sectors and the additionality effects of R&D tax credits: a cross-country microeconometric analysis, Research Policy, Vol. 46, No. 1, pp. 57–72. Garcia, A., Mohnen, P. (2010), Impact of government support on R&D and innovation, UNU-MERIT Working paper Series p. 2010-034, United Nations University – Maastricht Economic and social Research and Training Centre on Innovation and Technology, Maastricht. Gaul, M., Machowski, A. (1987), Elementy analizy ścieżek, in: W: J. Brzeziński (red.), Wielozmiennowe modele statystyczne w badaniach psychologicznych. PWE, Warszawa. Geels, F.W. (2020), Transformative innovation and socio-technical transitions to address grand challenges, R&I Working papers Series 2020/02, Publications Office of the European Union, Luxembourg, doi: 10.2777/967325. Georghiou, L. (2002), Impact and additionality of innovation policy, in: P. Boekholt (Eds), Innovation policy and sustainable development: Can innovation incentives make a difference? Brussels: IWT-Studies Innovation Science Technology no 40, pp. 57–67. Georghiou, L. (2004), Evaluation of behavioural additionality, Concept Paper, in: L. Georghiou, B. Clarysse, G. Steurs, V. Bilsen, J. Larosse, (Eds), ‘Making the difference’. The evaluation of ‘behavioural additionality’ of R&D, IWT – Observatory no.48, Brussels. Public financial support on innovation activities of European Union 265 Gonzalez, X., Jaumandreu, J., Pazo, C. (2005), Barriers to innovation and subsidy effectiveness, RAND Journal of Economics, Vol. 36, No. 4, pp. 930–949. Grabowski, W, Staszewska-Bystrova, A. (2020), The role of public support for innovativeness in SMEs across european countries and sectors of economic activity. Sustainability, Vol. 12, No. 10, p. 4143. doi: 10.3390/su12104143. Grabowski, W., Pamukcu, T., Szczygielski, K., Tandogan, S. (2013), Does government support for private innovation matter? Firm-level evidence from Turkey and Poland. CASE Network Studies & Analysis, Warszawa, 458/20. Guerzoni, M., Raiteri, E. (2015), Demand-side vs. supply-side technology policies: Hidden treatment and new empirical evidence on the policy mix, Research Policy, Vol. 44, No. 3, pp. 726–747. Gürbüz, K., Erdil, E. (2020), Prioritization and R&D support mechanisms: Turkish case, Journal of Knowledge Economy, doi: 10.1007/s13132-020-00648-y. Halpern, L. (2010), R&D subsidies and firm performance in Hungary, Micro-Dyn Working Paper, 38/10. Hegerty, S.W., Kowalski, A.M., Lewandowska, M.S. (2022), Complementarity of additionalities resulting from European Union funds: Perspective of the users of research infrastructures, Review of Policy Research, online access, doi: 10.1111/ ropr.12480 Hicks, D.J., Coil, D.A., Stahmer, C.G., Eisen, J.A. (2019), Network analysis to evaluate the impact of research funding on research community consolidation, PLoS ONE, Vol. 14, No. 6, p. e0218273, doi: 10.1371/journal.pone.0218273. Hussinger, K. (2003), Crowding out or stimulus: The effect of public R&D subsidies on firm’s R&D expenditure, Centre for European Economic Research (ZEW), draft version, July. Innovation Activities of Enterprises in 2008–2010. (2012), Central Statistical Office, Statistical Office in Szczecin, Warsaw. IUS. (2014), Innovation Union Scoreboard 2014, European Union, Brussels, https://op.europa.eu/en/publication-detail/-/ publication/d1cb48d3-4861-41fe-a26d-09850d32487b. Jensen, M., Johnson, B., Lorenz, E., Lundvall, B.-Å. (2007), Forms of knowledge and modes of innovation, Research Policy, Vol. 36, pp. 680–693. Kang, K.N., Park, H. (2012), Influence of government R&D support and inter-firm collaborations on innovation in Korean biotechnology SMEs, Technovation, Vol. 32, No. 1, pp. 68–78. Kline, R.B. (2011), Principles and Practice of Structural Equation Modeling, Guilford Press, New York. Konarski, R. (2009), Modele równań strukturalnych. Teoria i praktyka, Wydawnictwo Naukowe PWN. Lewandowska, M.S., Kowalski, A.M. (2015), Współpraca polskich przedsiębiorstw w sferze innowacji a wsparcie z funduszy unijnych, Gospodarka Narodowa, Vol. 4, No. 278, pp. 69–89. Magro, E., Wilson, J.R. (2013), Complex innovation policy systems: towards an evaluation mix, Research Policy, Vol. 42, No. 9, pp. 1647–1656. Marino, M., Lhuillery, S., Parrotta, P., Sala, D. (2016), Additionality or crowding-out? An overall evaluation of public R&D subsidy on private R&D expenditure, Research Policy, Vol. 45, No. 9, pp. 1715–1730. Marzucchi, A., Montresor, S. (2012), The multi-dimensional additionality of innovation policies. A multi-level application to Italy and Spain, preliminary version, retrieved from https://www.sussex.ac.uk/webteam/gateway/file. php?name=montresor2-ita-spa-spru.pdf&site=25 Mazzucato, M. (2018), Mission-oriented innovation policies: challenges and opportunities, Industrial and Corporate Change, Vol. 27, No. 5, pp. 803–815. doi: 10.1093/icc/dty034 Neicu, D., Teirlinck, P., Kelchtermans, S. (2016), Dipping in the policy mix: do R&D subsidies foster behavioral additionality effects of R&D tax credits?, Economics of Innovation and New Technology, Vol. 25, No. 3, pp. 218-239, doi: 10.1080/10438599.2015.1076192 Nilsen, Ø.-A., Raknerud, A., Iancu, D.-C. (2020), Public R&D support and firm performance: A multivariate dose-response analysis, Research Policy, Vol. 49, No. 7, pp. 1–16, doi: 10.1016/j.respol.2020.104067 OECD. (2006), Government R&D funding and company behaviour, Measuring behavioural additionality, Paris, OECD Publishing. Orlic, E., Radicic, D., Balavac, M. (2019), R&D and innovation policy in the Western Balkans: are there additionality effects? Science and Public Policy, Vol. 46, No. 6, pp. 1–19. doi: 10.1093/scipol/scz036. Pang, S., Dou, S., Li, H. (2020), Synergy effect of science and technology policies on innovation: Evidence from China. PLoS ONE, Vol. 15, No. 10, p. e0240515. doi: 10.1371/journal pone.0240515. Petrin, T. (2018), A literature review on the impact and effectiveness of government support for R&D and innovation, Working paper No. 05/2018, http://www.isigrowth.eu/wp-content/uploads/2018/02/working_paper_2018_05.pdf Radas, S., Anića I.-D., Tafro A., Wagner, V. (2015), The effects of public support schemes on small and medium enterprises, Technovation, Vol. 38, pp. 15–30. Radicic, D., Pugh, G. (2017), R&D programmes, policy mix, and the “European Paradox”: evidence from European SMEs, Science and Public Policy, Vol. 44, No.4, pp. 1–16. Roper, S., Hewitt-Dundas, N. (2016), The legacy of public subsidies for innovation: input, output and behavioural additionality effects, ERC Research Paper 21, Warwick: Enterprise Research Centre. Rothwell, R., Zegveld, W. (1982), Industrial innovation and public policy: preparing for the 1980s and the 1990s. Pinter, London. 266 M. S. Lewandowska et al. Schneider, C., Veugelers, R. (2010), On young highly innovative companies: Why they matter and how (not) to policy support them, Industrial and Corporate Change, Vol. 19, No. 4, pp. 969–1007. Schot, J., Steinmueller, W.E. (2018), Three frames for innovation policy: R&D, systems of innovation and transformative change, Research Policy, Vol. 47, No. 9, pp. 1554–1567. Serrano-Velarde, N. (2008), Crowding-out at the top: the heterogeneous impact of R&D subsidies on firm investment, European University Institute, Department of Economics, November 24, 2008. Job Market Paper. Shin, K., Choy, M., Lee, C., Park, G. (2019), Government R&D subsidy and additionality of biotechnology firms: the case of the South Korean Biotechnology Industry, Sustainability, Vol. 11, No. 6, p. 1583. Szczygielski, K., Grabowski, W., Pamukcu, M.T. (2017), Does government support for private innovation matter? Firm-level evidence from two catching-up countries, Research Policy, Vol. 46, No. 1, pp. 219–237. Uyarra, E., Zabala-Iturriagagoitiac, J.M., Flanagana, K., Magrod, E. (2020), Public procurement, innovation and industrial policy: rationales, roles, capabilities and implementation, Research Policy, Vol. 49, No. 1, doi: 10.1016/j. respol.2019.103844. Van Beers, C., Zand, F. (2014), R&D cooperation, partner diversity, and innovation performance: an empirical analysis, Journal of Product Innovation Management, Vol. 31, No. 2, pp. 292–312. Van Leeuwen, G., Klomp, L. (2006), On the contribution of innovation to multi-factor productivity, Economics of Innovation and New Technology, Vol. 15, No. 4/5, pp. 367–390. Wallsten, S.J. (2000), The effects of government-industry R&D programs on private R&D: the case of the small business innovation research program, RAND Journal of Economics, Vol. 31, No. 1, pp. 82–100. Wanzebock, I., Scherngell, T., Fischer, M.M. (2013), How do firm characteristics affect behavioural additionalities of public R&D subsidies? Evidence for the Austrian transport sector, Technovation, Vol. No. 33(2–3), pp. 66–77. Weresa, M.A., Poel, M., Cunningham, P., den Hertog, P. (2018), Mutual learning exercise on evaluation of business R&D grant schemes: behavioural change, mixed-method approaches and big data, European Commission, Directorate-General for Research and Innovation, doi:10.2777/79197. Weresa, M.A. (2017), Research and development policy, innovation policy, and industrial policy: an interface, in: A. Ambroziak (Ed.), The New Industrial Policy of the European Union, Springer, pp. 187–204, Switzerland Weresa, M.A., Lewandowska, M.S. (2014), Innovation system restructuring in Poland in the context of EU membership, in: M.A. Weresa (Ed.), Poland: Competitiveness Report 2014. A Decade in the European Union, Warsaw School of Economics Press, Warsaw. Wright, S. (1934), The method of path coefficients, Annals of Mathematical Statistics, Vol. 5, No. 3, pp. 161–215. doi: 10.1214/ aoms/1177732676. Wright, S. (1921), Correlation and causation, Journal of Agricultural Research, Vol. 20, pp. 557–585. Wubben, E.F.M., Batterink, M., Kolympiris, C., Kemp, R.G.M., Omta, O.S.W.F. (2015), Profiting from external knowledge: the impact of different external knowledge acquisition strategies on innovation performance, International Journal of Technology Management, Vol. 69, No. 2, pp. 139–165.