TY - JOUR
AU1 - Kundu,, Oishee
AU2 - James, Andrew, D
AU3 - Rigby,, John
AB - Abstract Public procurement and innovation is the subject of a growing body of literature. This article systematically reviews the existing research, documenting its evolution and highlighting dominant and overlooked themes. We find a dramatic increase in the number of journal publications on this topic since 2008, the existence of thematic communities within the literature, and a focus on empirical work. We analysed keywords and abstracts to identify the broad boundaries of research on public procurement and innovation as well as particular areas of focus in the literature. We found a variety of terms used to describe the application of public procurement as an innovation policy tool and a variety of thematic interests and theoretical foundations. We argue that this makes it challenging to consolidate the research on the topic. From a policy perspective, the geographical context of research is overwhelmingly concentrated in OECD and EU countries. 1. Background There is a growing body of literature on public procurement and innovation. While there are a number of excellent reviews on this topic (Hommen and Rolfstam 2009; Lember et al. 2015; Chicot and Matt 2018), to our knowledge there has been no systematic review of the literature. Lember et al. (2015) and Chicot and Matt (2018) are specific in their attention to ‘public procurement of innovation’, which is defined as ‘purchasing activities carried out by public agencies that lead to innovation’ (Rolfstam 2013: 12). Some studies on public procurement and innovation use the term ‘public procurement for innovation’ which is described by Edquist et al. (2015) as ‘a demand-side innovation policy instrument in the form of an order, placed by a public organization, for a new or improved product to fulfil its particular needs’ (p. 1). There are other papers in the literature that make reference to pre-commercial procurement (PCP). PCP is a concept introduced in 2006 by the European Commission drawing on the example of the US Small Business Innovation Research (SBIR) programme. There has been a debate about whether PCP is a demand-side policy tool in the same sense as public procurement ‘for’ innovation (see Edquist and Zabala-Iturriagagoitia, 2015).1 In order to address these multiple ideas regarding public procurement and innovation, this article systematically collected literature on the topic. The article analyses this corpus to examine the evolution, foundational concepts, methods, and contexts of the research on public procurement and innovation. To be clear, public procurement is commonly understood as the purchase of goods, services, and works by government in order to perform its functions. The idea that public procurement can be a powerful tool in driving innovation and spurring technological development is based on models of innovation that emphasize the role of demand (Dalpé 1994; Edquist and Hommen 1999; Alic 2008). There is a long history of studies of the demand-side, the role of public procurement in industrial innovation and its influence on the emergence and growth of technology-intensive sectors such as aerospace, computers, semiconductors, and software (Pavitt and Walker 1976; Rothwell 1981; Nelson and Langlois 1983; Ruttan 2006). The last two decades have seen growing interest in placing demand at the core of innovation policy (Boon and Edler 2018). The idea has captured the attention of policymakers for two reasons. Firstly, it reflects the dissatisfaction with supply-side innovation policies amongst policy makers (Edler and Georghiou 2007). Secondly, using public procurement as an innovation policy tool is considered to be an opportunity for increasing the efficiency of public spending in the economy, especially after the public spending cuts following the 2008 financial crisis. Government spending accounts for 40 per cent to 55 per cent of GDP in most OECD countries, making government procurement a potentially powerful demand-side influence (OECD 2011). This article reports the results of a systematic literature review of academic journal papers on public procurement and innovation and public procurement as a demand-side innovation policy tool. A systematic literature review uses rigorous and transparent methods to screen, select, and analyse a body of literature (Boland et al. 2017). The motivation for conducting such an exercise is to map the literature in a manner that identifies key concepts and gaps (Pham et al. 2014).2 The findings offer an opportunity for existing researchers in the field to be reflective and the review also suggests directions for further development of the field. This article makes three contributions to knowledge. The first contribution is a systematic literature review on public procurement and innovation, a methodologically rigorous process of reviewing existing research on a topic. To our collective knowledge, this is the first systematic literature review on public procurement and innovation. We find this surprising given the academic and policy interest in the topic. Secondly, as a literature review, the article also contributes to a more fine-grained understanding of the characteristics of the literature, including such important matters as the evolution of research interest in the topic, and foundational assumptions and methods used over time. Thirdly, the systematic review provides an opportunity to identify key themes in public procurement and innovation research and identify dominant themes as well as under-researched areas. This article is structured as follows. Section 2 describes the methodology used including the search and selection criteria for building the corpus and the coding procedures. Section 3 presents the findings, discussing the evolution of the field, the themes and research questions found in the literature, its theoretical foundations, research foci and contexts. Section 4 summarizes and concludes. 2. Methods Systematic literature reviews are characterized by a careful recording of the process of searching and selecting items in the literature for review. This helps in improving the replicability and reliability of the literature review. This section describes how the corpus was created and investigated, justifying the choices made when beginning this study. 2.1 Building the corpus The search was conducted across four large abstract and citation databases of peer-reviewed literature—Web of Science, Scopus, Business Source Premier, and EconLit. The Boolean expression ((public OR government) AND (procurement) AND (innovation OR technology)) OR (demand AND side AND innovation AND policy) was searched across abstracts. The inclusion/exclusion criteria for building the corpus were as follows: Criterion 1: the item should be a research article published in a peer-reviewed journal and in English The restriction to work published in journals is made with the knowledge that despite some criticisms and drawbacks, the peer-review process is the most commonly used method of research validation, and therefore creating a corpus from peer-reviewed work reveals the accepted ideas on the topic. Admittedly, this misses out some of the research present in books, book chapters, and the grey literature. For instance, a number of significant contributions on the topic have been in books or book chapters, including Edquist et al. (2012) and Rigby (2016). Our decision to focus on peer-reviewed journal papers was made to keep the items in the corpus comparable and to code each record across a variety of variables in line with the method of systematic literature reviews (León and Farris 2011; Cheng et al. 2018). The restriction to studies published in English is due to resource constraints on translation from other languages. Criterion 2: the item should discuss public procurement and its role in innovation This is the relevance criterion. For the purposes of this review, ‘public procurement’ means purchases made by government or contracts signed by government to obtain goods, services, or works. This review aims to draw insights on the relationship between public procurement and innovation, including the role of public procurement as an innovation policy tool. Therefore, articles discussing innovations in the organization and management of public procurement processes were excluded. Similarly, exclusive discussion of standards or the efficiency achieved from procuring new technology was not considered.3 The search was conducted in February 2018, so the publication time was fixed at an upper limit of 2017 for this study. The relevance criterion identified a corpus of 110 papers. Eleven of these articles could not be accessed,4 leaving us with a corpus of ninety-nine journal papers (Table 1). Table 1. Search and selection of corpus. . Web of Science . Scopus . Business Source Premier . EconLit . Hits 1,042 1,516 1,027 199 Fulfilling Criterion 1 706 647 351 106 After removing duplicates 1,110 Removed for not fulfilling Criterion 1a 125 Removed for not fulfilling Criterion 2 985 Selected Corpus 110 Articles could not be accessed 11 Actual Corpus examined 99 . Web of Science . Scopus . Business Source Premier . EconLit . Hits 1,042 1,516 1,027 199 Fulfilling Criterion 1 706 647 351 106 After removing duplicates 1,110 Removed for not fulfilling Criterion 1a 125 Removed for not fulfilling Criterion 2 985 Selected Corpus 110 Articles could not be accessed 11 Actual Corpus examined 99 a Some articles in languages other than English had been automatically included because they had an English summary and were removed. Open in new tab Table 1. Search and selection of corpus. . Web of Science . Scopus . Business Source Premier . EconLit . Hits 1,042 1,516 1,027 199 Fulfilling Criterion 1 706 647 351 106 After removing duplicates 1,110 Removed for not fulfilling Criterion 1a 125 Removed for not fulfilling Criterion 2 985 Selected Corpus 110 Articles could not be accessed 11 Actual Corpus examined 99 . Web of Science . Scopus . Business Source Premier . EconLit . Hits 1,042 1,516 1,027 199 Fulfilling Criterion 1 706 647 351 106 After removing duplicates 1,110 Removed for not fulfilling Criterion 1a 125 Removed for not fulfilling Criterion 2 985 Selected Corpus 110 Articles could not be accessed 11 Actual Corpus examined 99 a Some articles in languages other than English had been automatically included because they had an English summary and were removed. Open in new tab 2.2 Coding We manually coded data from the corpus of publications by reading all ninety-nine journal papers. The coding was guided by a coder manual (see Supplementary data). It was structured like a questionnaire to allow coders to examine the articles in the corpus for various information like themes, contexts, underlying assumptions, and methods. 2.2.1 Inter-coder reliability tests To test the robustness of the coder manual, an inter-coder reliability test was applied. A subset of ten papers was randomly selected from the corpus and coded by two independent reviewers using the coder manual. Inter-coder reliability can be calculated as percentage agreement and using the Cohen’s kappa coefficient (Cohen 1960). In percentage agreement, the percentage indicates the share of papers where two independent coders provided the same codes. In Cohen’s method, the statistic (K) is calculated as K ≡ po-pe1-pe ⁠, where po is the observed agreement between two coders and pe is the probability of chance agreement. Cohen’s kappa is considered a more robust measure to simple percent agreement calculation as it takes the possibility of agreement occurring by chance into account. A standard acceptable rate of percentage agreement is 80 per cent (Jones et al. 2015) and this was achieved for all but one of the variables (Table 2). The value of Cohen’s K ranges from −1 to +1 where positive values reflect agreement and negative values reflect disagreements. Acceptable rates for the Kappa coefficient are debated, especially amongst health researchers (McHugh 2012). Given that the calculation of the coefficient is affected by sampling errors (Lacy and Riffe 1996), it is sufficient to note here that none of the K-values are negative, that is actual agreement was always more than chance agreement. Table 2. Results of inter coder reliability test. Variable . Coder 1 and Coder 2 . Coder 1 and Coder 3 . Percentage agreement . Cohen’s Kappa . Percentage agreement . Cohen’s Kappa . Type of article 80 0.286 90 0.429 Research method 90 0.615 100 1 Assumption/rationale 60 0.545 100 1 Function of government (reason for procurement) 80 0.762 90 0.878 Level of procurement 80 0.74 100 1 Variable . Coder 1 and Coder 2 . Coder 1 and Coder 3 . Percentage agreement . Cohen’s Kappa . Percentage agreement . Cohen’s Kappa . Type of article 80 0.286 90 0.429 Research method 90 0.615 100 1 Assumption/rationale 60 0.545 100 1 Function of government (reason for procurement) 80 0.762 90 0.878 Level of procurement 80 0.74 100 1 Open in new tab Table 2. Results of inter coder reliability test. Variable . Coder 1 and Coder 2 . Coder 1 and Coder 3 . Percentage agreement . Cohen’s Kappa . Percentage agreement . Cohen’s Kappa . Type of article 80 0.286 90 0.429 Research method 90 0.615 100 1 Assumption/rationale 60 0.545 100 1 Function of government (reason for procurement) 80 0.762 90 0.878 Level of procurement 80 0.74 100 1 Variable . Coder 1 and Coder 2 . Coder 1 and Coder 3 . Percentage agreement . Cohen’s Kappa . Percentage agreement . Cohen’s Kappa . Type of article 80 0.286 90 0.429 Research method 90 0.615 100 1 Assumption/rationale 60 0.545 100 1 Function of government (reason for procurement) 80 0.762 90 0.878 Level of procurement 80 0.74 100 1 Open in new tab 3. Findings and discussion 3.1 The corpus and the evolution of the field We begin by describing the corpus—its sources (in terms of publication outlets), the types of contribution made to the literature, and its history. This section provides a preliminary introduction to the literature, particularly highlighting its growth and diversity. 3.1.1 Temporal evolution of the corpus The first paper in the corpus was published in 1976. Written by Pavitt and Walker, it reviews government policies that support industrial innovation, including ‘the encouragement of technically progressive procurement practices’ (p. 76). The role of government demand in industrial innovation and technological change was a theme of a number of subsequent papers. Rothwell (1981) notes procurement as a possible government action to promote innovation in the economy and combat the global economic crisis that had set in after the 1973 oil price shock. Nelson and Langlois (1983) look at six industries and evaluate the success of government innovation policy, including procurement. Hutton and Hartley (1985) note the influence of procurement policy on R&D in the medical equipment industry in the UK and find that procurement by the National Health Service had positively influenced the level of research pursued by British medical equipment manufacturers. Mowery and Langlois (1996) study the role of the US government, including procurement by the Department of Defense in the development of the software industry. The corpus grew incrementally between 1976 and 2007 (Fig. 1). A noticeable shift in growth can be detected from 2005 and the growth in number of papers is almost exponential from 2008 (Fig. 2). The structural break was detected by using the CUSUM method to detect changes in the number of publications every year (Page 1954). To address the issue of how best to demonstrate the changes that have occurred over the period in which the publications have appeared on this subject, and in particular the significant increase in interest in the topic indicated by a rise in the number of papers published at around 2005–10, we have calculated the Cusum chart with publication counts transformed to lognormal. While the Shapiro Wilk statistic for the resulting distribution of transformed publication is significant, indicating non-normality, the Q-Q plot and Z-scores for skewness and kurtosis are within acceptable limits with z-values of 1.44 for skewness and 0.72 for kurtosis.5 From 2008, there is a marked increase in the number of journal articles published, peaking in 2015, when Innovation -The European Journal of Social Science published a special issue on the topic. Other indicators of growth and interest in the topic include the number of authors involved in the topic, the number of keywords, and publication outlets over time (Table 3). Since 2008, 121 researchers have published 70 papers across 44 journals on the topic of public procurement and innovation, which is striking when compared to early years. The potential reasons for this increase are explored in the following paragraphs and a more detailed analysis of the diversity of keywords and journals is provided in the subsequent sections. Figure 1. Open in new tabDownload slide Research output over time (n = 99). Figure 1. Open in new tabDownload slide Research output over time (n = 99). Figure 2. Open in new tabDownload slide Cusum plot to detect structural break in corpus. Figure 2. Open in new tabDownload slide Cusum plot to detect structural break in corpus. Table 3. Some indicators of academic interest over time. Time . Authors (first-time contributors) . Keywordsa . WoS Keywordsb . Journals . Articles . 1976–87 7 0 0 5 6 1988–97 20 (18) 14 10 10 11 1998–2007 23 (23) 40 17 11 12 2008–17 121 (115) 189 146 44 70 Total 163 236 159 59 99 Time . Authors (first-time contributors) . Keywordsa . WoS Keywordsb . Journals . Articles . 1976–87 7 0 0 5 6 1988–97 20 (18) 14 10 10 11 1998–2007 23 (23) 40 17 11 12 2008–17 121 (115) 189 146 44 70 Total 163 236 159 59 99 a Author-provided keywords exist only for 66 of the 99 papers in the corpus. b Web of Science generated KeywordsPlus could be obtained for 63 of the 99 papers in the corpus. 12 of these 63 papers contained WoS KeywordsPlus but not author-provided keywords. Open in new tab Table 3. Some indicators of academic interest over time. Time . Authors (first-time contributors) . Keywordsa . WoS Keywordsb . Journals . Articles . 1976–87 7 0 0 5 6 1988–97 20 (18) 14 10 10 11 1998–2007 23 (23) 40 17 11 12 2008–17 121 (115) 189 146 44 70 Total 163 236 159 59 99 Time . Authors (first-time contributors) . Keywordsa . WoS Keywordsb . Journals . Articles . 1976–87 7 0 0 5 6 1988–97 20 (18) 14 10 10 11 1998–2007 23 (23) 40 17 11 12 2008–17 121 (115) 189 146 44 70 Total 163 236 159 59 99 a Author-provided keywords exist only for 66 of the 99 papers in the corpus. b Web of Science generated KeywordsPlus could be obtained for 63 of the 99 papers in the corpus. 12 of these 63 papers contained WoS KeywordsPlus but not author-provided keywords. Open in new tab With respect to the authors engaging in the topic, an interesting finding is that a large majority of authors in every decade are ‘first-time contributors’ in the sense that they have not appeared in the corpus before. In this respect, the 1998–2007 period seems remarkable as all the twenty-three contributing authors had not made any previous contribution to the study of public procurement and innovation but began contributing in this period. Perhaps this fresh interest in the topic can be explained by wider trends in policy and research on public procurement that began to emerge in the mid-1990s. The significant growth of interest in the general topic area from 2008 is likely to have resulted in part from the engagement by innovation scholars in the development of EU and Member States policy on procurement during the first decade of the new millennium. The EU and Member State activity began to consider how procurement could play a role in developing the economy from the European Council of 2000 in Lisbon (European Council 2000). At the same time that the EU was developing policy, the UK was launching its own variation on the US SBIR, although without immediate success (Connell 2004; Bound and Puttick 2010). Policy action at EU level was relatively quick and the attempt to embed mechanisms to promote innovation through procurement law was achieved in the new procurement directives (2004/17/EE and 2004/18/EE). This was accompanied by high-level policy statements that supported the legal framework and its objectives (Kok 2004; Aho et al. 2006). Additionally, the European Commission’s procedure for pre-commercial procurement (European Commission 2007) came in 2007 which attempted to set out a procedure to promote innovation of new goods and services that bypassed the directives (Rigby 2016) and which, to some extent, reflected the operation of US SBIR (Apostol 2017). Those working in the academic environment such as Edler, Edquist, and Georghiou were simultaneously involved in various international/supranational initiatives at EU and OECD level and in more abstract and theoretical discussion of the policy in academic circles. The 2007 Research Policy paper by Edler and Georghiou is the most cited text in this corpus of research articles focussing on public procurement and innovation, and both individuals participated in the EU-level dialogues (Professor Luke Georghiou was the rapporteur of the Independent Expert Group which produced the 2006 ‘Creating an Innovative Europe Report’). Forty-six of seventy research articles in the corpus published since 2008 cite this paper. A further key figure in the debate was Professor Charles Edquist whose coining of the acronym ‘PPI’ for ‘public procurement for innovation’ in 2009 was subsequently made reference to by a number of authors (seventeen of sixty-two articles in the corpus refer to it since 2010). The use of this term and the related term ‘public procurement of innovation’ introduce an important difference of emphasis which we discuss in our section on Thematic analysis (below). The policy interest in procurement for the purposes of achieving innovation was the result of a wider revival of interest in the role of the state in the economy (Rolfstam 2015) following the signing of the WTO Agreement on Government Procurement (GPA) which came into force in 1996. This new legal order required a wholesale re-examination of how governmental priorities for industrial development and national specialization could be pursued. Differences began to appear, notably, between the European Union and the USA. In the European Union, Martin (1996) has argued that the new policy framework created placed more limitations on the use of public procurement to pursue goals of innovation, industrial policy, and growth than that created in the USA where procurement as an innovation policy tool allowed the state to be entrepreneurial. Mazzucato (2013) stimulated further interest in this issue amongst academics and policy makers, 3.1.2 Where is the corpus found? The papers were published in fifty-nine different journals. However, Fig. 3 indicates that the corpus is particularly concentrated in a few journals with seven journals accounting for 40 per cent of the papers in the corpus. Figure 3. Open in new tabDownload slide Distribution of corpus across 59 journals. Figure 3. Open in new tabDownload slide Distribution of corpus across 59 journals. Table 4 lists journals that have published at least three papers on public procurement and innovation. Innovation—The European Journal of Social Science Research leads the list not least because it published a special issue on the topic in 2015 edited by Max Rolfstam from Aalborg University. The second largest number of papers were published in Research Policy. Research Policy has seen an active discussion on the topic of public procurement and innovation over a long period going back to publications by Pavitt and Walker (1976), Hutton and Hartley (1985), and Mowery and Langlois (1996). Research Policy continues to publish highly-cited articles related to the topic, not least among which is the Edler and Georghiou (2007) paper that stimulated renewed discussion on public procurement as a demand-side innovation policy. Table 4. Journals with at least three articles in the corpus. . . Journal Impact Factora . Journal Title . Count . (2017)a . Innovation 11 1.018 Research Policy 9 4.661 Journal of Public Procurement 4 1.12b Science and Public Policy 4 1.368 Technological Forecasting & Social Change 4 3.131 Technology in Society 4 1.60b Technovation 3 4.802 . . Journal Impact Factora . Journal Title . Count . (2017)a . Innovation 11 1.018 Research Policy 9 4.661 Journal of Public Procurement 4 1.12b Science and Public Policy 4 1.368 Technological Forecasting & Social Change 4 3.131 Technology in Society 4 1.60b Technovation 3 4.802 a Source: Web of Science InCites Journal Citation Reports 2017. b Source: Scopus CiteScore 2017. Open in new tab Table 4. Journals with at least three articles in the corpus. . . Journal Impact Factora . Journal Title . Count . (2017)a . Innovation 11 1.018 Research Policy 9 4.661 Journal of Public Procurement 4 1.12b Science and Public Policy 4 1.368 Technological Forecasting & Social Change 4 3.131 Technology in Society 4 1.60b Technovation 3 4.802 . . Journal Impact Factora . Journal Title . Count . (2017)a . Innovation 11 1.018 Research Policy 9 4.661 Journal of Public Procurement 4 1.12b Science and Public Policy 4 1.368 Technological Forecasting & Social Change 4 3.131 Technology in Society 4 1.60b Technovation 3 4.802 a Source: Web of Science InCites Journal Citation Reports 2017. b Source: Scopus CiteScore 2017. Open in new tab Amongst the journals that have published at least three papers, there are top-ranked journals (by impact factor) including not only Research Policy but Technovation, Technological Forecasting & Social Change and Science & Public Policy.6 In August 2018, Science & Public Policy published a ‘special section’ on the role of demand-oriented policies for innovation. While the corpus does not include contributions from 2018, the SPP papers which are included in the corpus offer important and interesting perspectives. Rolfstam (2009) explores the role of institutions in affecting the possibilities for public procurement of innovation and Li and Georghiou (2016) argue about the differences between developed and developing countries with respect to public procurement and innovation. It is worth noting that the journals with at least three articles are all in the field of innovation management and policy. The one exception is the Journal of Public Procurement which is a specialist journal that aims to further the understanding of public procurement. We find it interesting that the public procurement community has published so few papers on the consequences of public procurement for innovation. Most of the authors of papers in the Journal of Public Procurement are innovation management and policy scholars (e.g. Hommen and Rolfstam 2009; Yeow and Edler 2012). To examine the highly-cited articles within the corpus, citation counts were noted from Web of Science (n = 76), Scopus (n = 84), and Google Scholar (n = 99). Since Google Scholar metrics are available for the entire corpus, the distribution of corpus by citation (Fig. 4) is based on those figures. Figure 4. Open in new tabDownload slide Distribution of articles by citation (n = 99) as on 18 April 2018. Figure 4. Open in new tabDownload slide Distribution of articles by citation (n = 99) as on 18 April 2018. The most highly cited paper on the topic of public procurement and innovation is the 2007 Research Policy paper by Edler and Georghiou titled ‘Public Procurement and Innovation- resurrecting the demand side’. The article emphasizes the importance of the demand side as a potential source of innovation and argues for the place of public procurement as a demand-side measure in the taxonomy of innovation policy tools. The article goes on to discuss the rationales and justifications of public procurement policies that spur innovation as well as the challenges that such policies may face. This article proposes public procurement of innovation as a possible important future public policy. However, in framing its recommendations, it does not include an extensive evidence base. The second most cited paper is Edquist and Hommen’s 1999 paper in Technology in Society. Starting from the systems of innovation perspective, their paper discusses innovation as an iterative, interactive learning process and points out that procurement is not just a matter of price signals and quantities but also a learning process through which the government conveys its requirements. Again, the approach is to layout a framework and scope for policy action. The authors note their aim is to identify ‘some elements of a general policy perspective’ (Edquist and Hommen 1999: 63). The third most highly cited paper (and the most highly cited by a non-European author) is Lichtenberg’s 1988 paper in the American Economic Review. Lichtenberg emphasizes the importance of government procurement by design and technical competitions as a means of encouraging private investment in research and notes the importance of ‘government revealing its demand for certain types of technological innovations’ (Lichtenberg 1988: 550). This relatively early investigation of the topic employs a model which tests firm-level data on R&D expenditure and share of competitive and non-competitive contracts for 169 firms between 1979 and 1984 in the USA at a time of a ‘major defence build-up’ in the USA. The lessons of this research are supportive of the general case that government procurement through competitive procurement at scale provides incentives for higher R&D expenditure by firms and has potentially broader innovation effects. 3.1.3 Types of journal papers We also examined the types of journal papers found in the corpus. It is difficult to find a universal classification of journal articles, especially in the social sciences. Nissen (1996) developed a classification system to review engineering literature according to the contribution of the paper and we adapted that typology.7 We coded the papers into one of four possible categories—(1) empirical, when the primary contribution of the article is information on a case or policy evaluation (using either or both qualitative and quantitative methods); (2) methodological, when the article provides different approaches to analyse data and addresses the research topic in methodologically novel ways; (3) investigative, when the article probes theory and practice to develop guidelines and frameworks for further research, and (4) theory-building, when the primary purpose of the article is to propose testable models and hypotheses.8 Articles making an empirical contribution to the literature were most common (fifty-five of ninety-nine). These mainly highlighted policy practice and provided case studies of the use of public procurement as a demand-side innovation policy. Some of these articles chose to focus on a particular industry in a particular country—for example, satellite telecommunications in Italy (Landoni 2017), biorefineries in Sweden (Hellsmark and Söderholm 2016), while others chose a number of industries or economic sectors to evaluate national (and sometimes regional or local) policies on public procurement and innovation—like in the UK (Gee and Uyarra 2013), China (Li and Georghiou 2016), or the USA (Alic 2015). While a majority of these empirical contributions conclude their work with positive remarks and are hopeful about the use of public procurement as an innovation policy instrument, some are cautious—Li and Georghiou (2016) note that ‘the appropriateness and effectiveness of such instruments are closely dependent upon the state of both the innovation and the procurement systems in which they are set’ (p. 349), and a few diverge from the view that public procurement can be an effective innovation policy tool (Walsh 1993; Davis and Brady 2015; Meehan et al. 2017). Second in order of frequency are investigative articles (twenty-four of ninety-nine) which review trends in literature and policy practice to suggest frameworks for further research on the topic. These include seminal texts like Pavitt and Walker (1976) and Edler and Georghiou (2007), as well as Rolfstam (2009) which talks about institutions, Kattel and Lember (2010) which explores the feasibility of using procurement as a demand-side innovation policy in a developing country context, and Edquist and Zabala-Iturriagagoitia (2012) which examines public procurement for innovation as a ‘mission-oriented’ innovation policy. Investigative articles often explore different ways that would help make public procurement more effective as an innovation policy tool and seem to agree that public procurement is most effective when the government (or public) is itself a user of the product, service, or system. The various contributions on improving procurement policy with respect to innovation include coordinating demand and supply-side policies for innovation to reduce the risk of contradiction or non-coherence (Rothwell 1981), the use of innovation-friendly procurement at all levels of government (Uyarra and Flanagan 2010), and the use of project management strategies (Yeow and Edler 2012). There are thirteen papers in the corpus that make a primarily methodological contribution. For example, Aschhoff and Sofka (2009) use Tobit modelling to compare public procurement with other innovation policy instruments to understand the unique contributions of public procurement, and Markovic-Hribernik and Detelj (2016) measure the impact of public procurement towards innovation by running regressions on an EU-wide panel data. Nine of the thirteen papers find positive evidence about the use of public procurement as an innovation policy tool. However, not all methodological contributions relate to the use of quantitative methods. For example, Li (2017) proposes the use of two dimensions of policy coherence (horizontal and vertical) to assess the process and practice of public procurement of innovation. There are seven articles whose primary contribution is theory-building by proposing testable, falsifiable models that link public procurement and innovation. For example, Lichtenberg (1988) proposes that firm R&D levels respond positively and significantly to government procurement competitions, Dalpé et al. (1992) create a model to test the importance of government’s role as the ‘first user of innovation’ in promoting technological development in various industries, and Goel (2001) develops a principal-agent model to test how R&D efforts by the agent (supplier) are affected by the varying degrees of control imposed by the principal (government contractor). Although few in number, by modelling the conditions under which public procurement can be expected to succeed as an innovation policy, these papers provide considerable support to the research on public procurement and innovation 3.2 Thematic analysis We now turn to discuss our findings related to themes in the corpus. These help to understand the research directions and areas of interest for the research community. Keywords provide a simple indication of themes, vocabulary and its diversity. Semantic analysis of texts allows the identification of thematic groups and their proximity and level of interactions. 3.2.1 Keywords Keywords help in searching for articles and help connect researchers who share similar vocabulary. Sixty-six of the ninety-nine journal articles listed keywords and therefore comments cannot be made about the entire corpus.9 However, even these sixty-six journal articles yield 236 distinct keywords of which only ten appear at least three times which indicates a diverse vocabulary. ‘Public procurement’ is the keyword in thirty-three papers, leading the list as the most frequently used keyword, followed by ‘innovation’ with twenty-seven and ‘innovation policy’ with thirteen instances of use. Another popular keyword is ‘demand’ (ten instances). There are variations in the use of ‘public procurement’ and ‘innovation’ which combine the two terms—three instances of ‘public procurement of innovation’, two of ‘public procurement for innovation’, and one of ‘public end-user driven technological innovation’. The most frequent words reflect the broad boundaries of the corpus, but it is the less frequent keywords (but occurring more than once) which provide insights into the interests of the corpus. There are five instances where ‘industrial development’ (or a similar term) is used to convey an interest in the use of public procurement and innovation for promoting industrial growth (Shyu et al. 2001; Srinivas 2006; Malerba et al. 2008; Myoken 2010; Matelly and Lima 2016). The term ‘interactive learning’ is used by three papers to draw attention to a systemic, iterative model of innovation, where demand and supply elements need to interact to create something new (e.g. Edquist and Hommen 1999; Edquist and Zabala-Iturriagagoitia 2012; Phillips et al. 2007). There are different keywords to suggest interest in topics like health—‘health care’ (Walker and Preuss 2008; Meehan et al. 2017) and ‘medical technology’ (Torbica and Cappellaro 2010; Sorenson and Kanavos 2011), and environmental protection—‘renewable fuels’ (Alic 2015; Aldenius and Khan 2017), ‘environmental innovation’ (Ghisetti 2017), and ‘sustainability’ (Cerin 2006; Walker and Preuss 2008; Gee and Uyarra 2013; Cohen and Amoros 2014). One recent paper in the corpus uses the term ‘green public procurement’ (Aldenius and Khan 2017). A systematic literature review on green public procurement (GPP) was recently published (Cheng et al. 2018) and GPP appears to be a special branch of public procurement and innovation which focuses on public procurement for environmental protection. 3.2.2 Analysis of abstracts Shared keywords and terms imply a sense of research community. This can be probed further by conducting semantic analysis of texts and producing a co-occurrence network based on the terms extracted. In this study, we used the abstracts from the ninety-nine papers to identify co-occurrences of terms. To generate our co-occurrence network, we conducted a textual analysis. We uploaded the corpus of abstracts to Cortext, an open-source lexical analysis software platform. With the condition of at least two occurrences in abstracts at the document level, terms extraction produced a list of 221 terms. The list of terms was scanned and irrelevant terms were manually removed. A list of 135 terms remained. As with our analysis of keywords, ‘public procurement’ and ‘innovation policy’ occur most frequently in the abstracts, but what is more interesting is the variety of ways in which the link between public procurement and innovation is expressed, the most popular being ‘public procurement for innovation’, and ‘public procurement of innovation’ (twelve documents each). Edler and Georghiou (2007) provide the first instance of the use of ‘public procurement for innovation’ and Rolfstam (2009) is the first instance of the use of ‘public procurement of innovation’. Rolfstam (2015) later suggests that an important difference between the two terms, which is generally observed in their use, is that public procurement for innovation refers to the procurement of ‘goods and services that do not yet exist’ (an expression used by Edquist and Hommen (1999)) and tends to denote ‘technology procurement’ while public procurement of innovation is broader in its meaning, denoting public procurement that leads to innovations of any kind (like innovation in the procurement process or through the combination of existing technologies). It is also interesting to note which terms occur together. To form a network of the terms extracted from the abstracts, the terms were mapped by using the Louvain method (Blondel et al. 2008). In the network, nodes represent the terms and the edges represent documents that connect these terms. Clusters of terms are formed based on optimal modularity, that is the density of edges inside the clusters (compared to outside) is optimized to detect clusters or communities in the network. Several closely-knit communities can be found by analysing the network (Fig. 5). ‘Public procurement of innovation’ and ‘public procurement for innovation’ occur in different (although not disconnected) communities. The term ‘public procurement of innovation’ co-occurs with terms like ‘policy initiatives’ and ‘institutional analysis’, while ‘public procurement for innovation’ co-occurs more prominently with terms like ‘European Union’. Both ‘public procurement of innovation’ and ‘public procurement for innovation’ belong to clusters with terms like ‘EU member states’ and ‘policymakers in the EU’, suggesting an association between European Commission-led policy initiatives and papers on this subject. Figure 5. Open in new tabDownload slide Homogenous network of terms extracted from abstracts. Figure 5. Open in new tabDownload slide Homogenous network of terms extracted from abstracts. A further important cluster revealed by the analysis is the one on the bottom-left of the figure. Linked by ‘government policy’, both the clusters have terms that can be associated with industries and industrial innovation (‘american industries’, ‘domestic firms’, ‘computer industry’, ‘small firms’, ‘enterprises and SMEs’), which reflects the original interest of the topic in studying public procurement as a government policy tool for promoting innovation in industries. Another cluster in the figure contains terms like ‘value chains’, ‘innovation systems’, ‘market demand’, ‘policy tools’, and ‘government agency’. Together, they would suggest a potential interest in, or application of, the systemic perspective in studying innovation theorized by Lundvall (1985) and (Nelson (1993)). Another cluster shows co-occurrence and close proximity of terms like ‘regional government’ and ‘local community’, exemplifying meso-level studies in the corpus (Rothwell 1984; Lember et al. 2011; Dale-Clough 2015). 3.3 Public procurement and innovation: justifications and contexts By examining the articles for the justifications provided and their contexts, it is possible to characterize research on public procurement and innovation along several dimensions. Identifying the justifications in a research corpus helps to appreciate the philosophies that inform the development of the field. Knowledge about the research contexts helps to identify dominant interests as well as overlooked areas which can inform further research efforts. 3.3.1 Justifications What are the justifications used in the literature to link public procurement and innovation? Edler and Georghiou (2007) identified three justifications for the use of public procurement as an innovation policy tool and called them ‘rationales’: the importance of lead user and demand, market and system failures, and the applicability of public procurement towards improving public policy and services. Chicot and Matt (2018) also describe policy rationales of public procurement and innovation research using the concept of failures which justify the policy. They identify three types of failures: demand-side failures relate to the provision of public services and adoption and diffusion of technology; supply-side failures relate to the incentives and capacity of producers to engage with research and development; and user-supplier interaction failures relate to uncertainties related to information asymmetry and lack of interactive learning spaces. In our corpus, we encountered a variety of justifications linking public procurement and innovation that could be classified into three categories, informed by the existing frameworks. The first is the ‘buyer and user’ rationale, where the government engages as a user (sometimes the ‘first user’) of a product or service. The ‘buyer and user’ rationale focuses on the importance of demand (expressed through procurement) for the development of technology, but also includes the importance of interaction with and perspectives from the user. The second type of rationale could be classified as the ‘market failure’ rationale. These justifications usually stressed on the ability of public procurement to generate incentives for producers and suppliers to engage in innovation and technology, which the market on its own was not able to generate. The third type of rationale was the ‘public services’ rationale, where the application of public procurement as an innovation policy is justified for improving the provision of public services. The papers were coded according to ‘buyer and user’, ‘market failure’, and ‘public services’ rationale. Where more than one rationale was discussed we coded multiple rationales. All ninety-nine papers were coded and the degree of overlap and relative sizes of the three rationales in the corpus is indicated in a Venn diagram in Fig. 6. A total of forty-two articles make reference to the buyer and user rationale, and the corresponding numbers for market failure and public services rationales are fifty and forty respectively. Figure 6. Open in new tabDownload slide Venn diagram on the use of justifications in the corpus (n = 99). Figure 6. Open in new tabDownload slide Venn diagram on the use of justifications in the corpus (n = 99). Seventy-two of the ninety-nine articles used just one of the rationales. The ‘buyer and user’ rationale is discussed exclusively in twenty-two papers. ‘Public services’ is examined exclusively in twenty-four papers. Only six papers consider all three rationales. The mutual exclusivity, i.e. the reference to only one of the rationales in seventy-two of the ninety-nine papers, would suggest that the literature has developed by using different justifications and theoretical bases. 3.3.2 Contexts A large number of papers had a clearly defined research focus in terms of what was the purpose of procurement, by which level of government, and in which country. A systematic review helps to reveal the dominant and the overlooked contexts in research. 3.3.2.1 Reason for procurement The UN classifies government functions into ten categories. This was adapted to identify the reasons for public procurement covered in the corpus in order to get a sense of the purpose of public procurement. Seventy articles made reference to at least one of the ten functions. Procurement of technology for environmental protection and health (medical equipment, hospital services) dominate the list (Table 5). The popularity of environmental protection as a reason for public procurement is related in part to the development of green public procurement (GPP) as a topic related to public procurement and innovation research. In this literature, the intention to mobilize procurement towards purchase of environmentally-friendly goods and services is sometimes discussed alongside promoting innovation and technological development. Manufacturing industries and ICT industries are also often discussed as case studies when public procurement is motivated towards improving the manufacturing sector and/or developing a more technology-driven manufacturing industry. Technology procurement for defence is discussed in fifteen papers, many of which discuss the role of military demand in technology development (Nelson and Langlois 1983; Mowery and Langlois 1996; Alic 2008). Table 5. Frequency of government functions fulfilled by technology procurement. Function of Government . Number of articles . Environmental protection 18 Health 18 Constructiona 16 Communicationb 15 Defence 15 Public safety and social protection 11 Transport 9 Housing and community development 6 Agriculture, forestry, fishing, fuel, energy, and mining 5 Education 5 Function of Government . Number of articles . Environmental protection 18 Health 18 Constructiona 16 Communicationb 15 Defence 15 Public safety and social protection 11 Transport 9 Housing and community development 6 Agriculture, forestry, fishing, fuel, energy, and mining 5 Education 5 a These refer to public works—construction of buildings, roads, etc. b This refers to government’s role in establishing infrastructure and communication networks. Open in new tab Table 5. Frequency of government functions fulfilled by technology procurement. Function of Government . Number of articles . Environmental protection 18 Health 18 Constructiona 16 Communicationb 15 Defence 15 Public safety and social protection 11 Transport 9 Housing and community development 6 Agriculture, forestry, fishing, fuel, energy, and mining 5 Education 5 Function of Government . Number of articles . Environmental protection 18 Health 18 Constructiona 16 Communicationb 15 Defence 15 Public safety and social protection 11 Transport 9 Housing and community development 6 Agriculture, forestry, fishing, fuel, energy, and mining 5 Education 5 a These refer to public works—construction of buildings, roads, etc. b This refers to government’s role in establishing infrastructure and communication networks. Open in new tab 3.3.2.2 Level of government Public procurement can be conducted at different levels of government: national, regional, or local. Almost all papers in the corpus defined the level of government at one of these levels, except for two which considered procurement at a multi-national level and therefore could not be classified in any of the three categories. Seventy-three papers look at national government procurement policy and practice, forty-three at the local level (municipality, city council), and thirty-two at the regional level. A partial explanation for the dominance of the focus on national level government could be that many of the reasons for pursuing procurement occur for functions of government that are performed by or require coordination at the national-level, like environmental protection, public health, and defence (Table 5). Although reference to regional government procurement and policy can be found in early years (e.g. Rothwell 1984), Table 6 shows that public procurement and innovation research has become more diverse with respect to the level of government in the last ten years with several works that seek to draw attention to local-level policy making (e.g. Pickernell et al. 2011; Dale-Clough 2015; Zelenbabic 2015; Uyarra et al. 2017). The share of corpus exclusively considering procurement by national governments has reduced over time. Table 6. Level of government studied by articles in the corpus (n = 97). Years . 1976–87 . 1988–97 . 1998–2007 . 2008–17 . Total . National (exclusively) 4 6 (↑50) 7 (↑16.67) 28 (↑300) 45 Share of corpus (in %age) 66.6 54.5 58.3 40 45.5 Regional (exclusively) 0 1 0 (↓100) 5 6 Share of corpus (in %age) 0 9.1 0 7.1 6.1 Local (exclusively) 0 0 0 13 13 Share of corpus (in %age) 0 0 0 18.6 13.1 National and regional 0 1 0 (↓100) 2 3 Share of corpus (in %age) 0 9.1 0 2.9 3 National and local 0 1 0 (↓100) 6 7 Share of corpus (in %age) 0 9.1 0 8.6 7 Regional and local 2 0 (↓100) 0 3 5 Share of corpus (in %age) 33.3 0 0 4.3 5.1 All three levels 0 2 4 (↑100) 12 (↑200) 18 Share of corpus (in %age) 0 18.2 33.3 17.1 18.2 Years . 1976–87 . 1988–97 . 1998–2007 . 2008–17 . Total . National (exclusively) 4 6 (↑50) 7 (↑16.67) 28 (↑300) 45 Share of corpus (in %age) 66.6 54.5 58.3 40 45.5 Regional (exclusively) 0 1 0 (↓100) 5 6 Share of corpus (in %age) 0 9.1 0 7.1 6.1 Local (exclusively) 0 0 0 13 13 Share of corpus (in %age) 0 0 0 18.6 13.1 National and regional 0 1 0 (↓100) 2 3 Share of corpus (in %age) 0 9.1 0 2.9 3 National and local 0 1 0 (↓100) 6 7 Share of corpus (in %age) 0 9.1 0 8.6 7 Regional and local 2 0 (↓100) 0 3 5 Share of corpus (in %age) 33.3 0 0 4.3 5.1 All three levels 0 2 4 (↑100) 12 (↑200) 18 Share of corpus (in %age) 0 18.2 33.3 17.1 18.2 Note: The figures in brackets indicate percentage growth or reduction from previous record. ↑ indicates and increase and ↓ indicates a reduction. Open in new tab Table 6. Level of government studied by articles in the corpus (n = 97). Years . 1976–87 . 1988–97 . 1998–2007 . 2008–17 . Total . National (exclusively) 4 6 (↑50) 7 (↑16.67) 28 (↑300) 45 Share of corpus (in %age) 66.6 54.5 58.3 40 45.5 Regional (exclusively) 0 1 0 (↓100) 5 6 Share of corpus (in %age) 0 9.1 0 7.1 6.1 Local (exclusively) 0 0 0 13 13 Share of corpus (in %age) 0 0 0 18.6 13.1 National and regional 0 1 0 (↓100) 2 3 Share of corpus (in %age) 0 9.1 0 2.9 3 National and local 0 1 0 (↓100) 6 7 Share of corpus (in %age) 0 9.1 0 8.6 7 Regional and local 2 0 (↓100) 0 3 5 Share of corpus (in %age) 33.3 0 0 4.3 5.1 All three levels 0 2 4 (↑100) 12 (↑200) 18 Share of corpus (in %age) 0 18.2 33.3 17.1 18.2 Years . 1976–87 . 1988–97 . 1998–2007 . 2008–17 . Total . National (exclusively) 4 6 (↑50) 7 (↑16.67) 28 (↑300) 45 Share of corpus (in %age) 66.6 54.5 58.3 40 45.5 Regional (exclusively) 0 1 0 (↓100) 5 6 Share of corpus (in %age) 0 9.1 0 7.1 6.1 Local (exclusively) 0 0 0 13 13 Share of corpus (in %age) 0 0 0 18.6 13.1 National and regional 0 1 0 (↓100) 2 3 Share of corpus (in %age) 0 9.1 0 2.9 3 National and local 0 1 0 (↓100) 6 7 Share of corpus (in %age) 0 9.1 0 8.6 7 Regional and local 2 0 (↓100) 0 3 5 Share of corpus (in %age) 33.3 0 0 4.3 5.1 All three levels 0 2 4 (↑100) 12 (↑200) 18 Share of corpus (in %age) 0 18.2 33.3 17.1 18.2 Note: The figures in brackets indicate percentage growth or reduction from previous record. ↑ indicates and increase and ↓ indicates a reduction. Open in new tab 3.3.2.3 Country An important variable in terms of research context, especially in policy research, is the country. We coded the papers for the countries that were the focus of the articles: eighty-five of the ninety-nine articles situated their study in one or more countries, and twenty-eight countries can be identified. We classified the European Union as a sovereign entity when EU policy is discussed. The frequency distribution is provided in Table 7. The UK is the most commonly studied country in the corpus, closely followed by the USA. Several European countries follow. China is the most frequently studied non-western country and has been receiving a lot of attention for its use of public procurement as an innovation policy in recent years. Table 7. Frequency of countries studied in the corpus. Country . Number of articles . United Kingdom 22 United States 20 European Union 11 Sweden 8 Finland; France; Germany 7 China 6 Italy; Japan; Netherlands; Spain 5 Norway; Switzerland 4 Denmark 3 Canada; Estonia; Greece; Latvia; Poland 2 Australia; Brazil; India; Ireland; South Korea; Singapore; Taiwan; Turkey 1 Country . Number of articles . United Kingdom 22 United States 20 European Union 11 Sweden 8 Finland; France; Germany 7 China 6 Italy; Japan; Netherlands; Spain 5 Norway; Switzerland 4 Denmark 3 Canada; Estonia; Greece; Latvia; Poland 2 Australia; Brazil; India; Ireland; South Korea; Singapore; Taiwan; Turkey 1 Open in new tab Table 7. Frequency of countries studied in the corpus. Country . Number of articles . United Kingdom 22 United States 20 European Union 11 Sweden 8 Finland; France; Germany 7 China 6 Italy; Japan; Netherlands; Spain 5 Norway; Switzerland 4 Denmark 3 Canada; Estonia; Greece; Latvia; Poland 2 Australia; Brazil; India; Ireland; South Korea; Singapore; Taiwan; Turkey 1 Country . Number of articles . United Kingdom 22 United States 20 European Union 11 Sweden 8 Finland; France; Germany 7 China 6 Italy; Japan; Netherlands; Spain 5 Norway; Switzerland 4 Denmark 3 Canada; Estonia; Greece; Latvia; Poland 2 Australia; Brazil; India; Ireland; South Korea; Singapore; Taiwan; Turkey 1 Open in new tab Remarkable in their low numbers and absence in Table 6 are non-OECD, non-EU countries. One explanation for this neglect can be found in Detelj et al. (2016). In their review of public procurement and innovation research in different countries, they note that the proliferation of studies in the USA, the UK, and Europe may indicate better awareness of policy possibilities, whereas public procurement and innovation studies in countries like Croatia and Serbia focus on improving the efficiency of public procurement and reducing corruption. The literature so far comes from a very western, liberal-democracy specific point-of-view and there may be potential in investigating the opportunities and challenges of using public procurement as a demand-side innovation policy tool in non-OECD, non-EU countries. China would seem an obvious focus for research. 4. Conclusion Academic interest in public procurement and innovation has grown significantly. The role of government as a buyer and user of goods and services is emerging as an important focus in discussions about the role of government in promoting innovation. In any body of research, a systematic literature review is a useful first step in consolidating knowledge on the topic. By searching and annotating journal articles that make reference to public procurement and innovation, this review helps to characterize the literature, comment on its evolution, and identify key themes, underlying assumptions and coverage of contexts. A variety of terms are used in the literature—‘public procurement of innovation’ and ‘public procurement for innovation’ being the most popular. Studies on public procurement for environmental protection and sustainable development have recently started referring to themselves as ‘green public procurement’. In order to keep all these variants of public procurement and innovation studies together under one umbrella, it is important to strengthen the underlying theories that serve public procurement and innovation so that the field is easier to recognize across diverse themes. The finding that papers recognize very different justifications linking public procurement and innovation emphasises the need to have more consensus about the foundations of the topic. In addition to a need for rigorous theoretical work, there is also a justification for paying attention to overlooked empirical contexts and for testing the applicability of public procurement and innovation concepts in different scenarios. While there is diversity in contexts in which the empirical research is situated—the reasons for procurement, the level of government which is conducting the procurement, and the country being considered—the contexts are rather unequally represented in the corpus. The empirical work mostly relates to the UK, USA, and western European countries. Greater diversity of empirical contexts may lead to a more universal understanding of the topic. Therefore, given the present state of knowledge on this topic, we conclude that policymakers should be cautious in applying public procurement as an innovation policy tool because the policy has only been applied in a few countries and a few contexts. The use of public procurement as a demand-side innovation policy tool has been conceptualized and tested with a few case studies and examples for the most part in several of the studies. Only five papers curate and measure evidence and impact of the policy. Although there is a consensus over the importance of the role of demand in promoting innovation and technological development, the academic literature rarely addresses questions on impact. The lack of studies on impact has limited the scope of this systematic review to be a meta-analysis on public procurement as an innovation policy. We propose two directions for future research. The first is more effort focused on constructing theoretical frameworks. There is currently very little theory-building work in the corpus and, given the need to develop a better understanding of demand generally and the role of demand in innovation policy in particular, we regard it as important that more work is carried out in this area. A second direction is empirical. Future research should seek greater variety of contexts for empirical testing of public procurement and innovation policy concepts. In particular, more research on public procurement and innovation should be conducted in non-OECD, non-EU countries. Funding This work was funded through the Alliance Manchester Business School Doctoral Scholarship (2017), The University of Manchester. Conflict of interest statement Oishee Kundu and Andrew D. James have no conflict of interest to declare. John Rigby worked on a project titled “Innovation and Public Procurement: review of issues at stake” (EU No. ENTR/03/24 Fraunhofer ISI) and led the following studies: “A Review of the Small Business Research Initiative” for Innovate UK (published in 2017 in collaboration with Prof Stephen Roper, ERC, and OMB Research), and “A Feasibility study on future EU support to public procurement of innovative solutions: Obtaining Evidence for a Full Scheme” (Contract Notice 2010/S 103-155769). Supplementary data Supplementary data is available at Science and Public Policy online. Footnotes 1 We thank one of our anonymous reviewers for prompting us to consider this aspect. 2 Systematic literature reviews are sometimes pursued with the objective of presenting an assessment of an intervention and are called ‘meta-analysis’ or ‘meta-evaluation’. Because such systematic reviews of literature were first done in medical research with the purpose of assessing the effectiveness of clinical interventions, there is often an expectation of systematic reviews to do the same (i.e. report on the effectiveness of intervention). A ‘scoping review’ is therefore the technically correct term for a systematic study of literature that does not conduct a meta-evaluation. However, in the interest of visibility, we have followed the conventional name ‘systematic literature review’ in this article. 3 The search strategy did not actively look for the use of policy terms like ‘PPI’ (public procurement of/for innovation’ or ‘PCP’ which is short for pre-commercial procurement. This is because the review seeks to include any research that describes public procurement as a possible demand-side innovation policy instrument rather than any specific conceptualization of public procurement as an innovation policy tool. At the same time, the use of terms like PPI or PCP did not lead to automatic exclusion of the articles. 4 Six of the results did not lead to a paper on further search, and five could not be accessed with the institutional subscription available to the authors. 5 The transformation has been undertaken because of the difficulties of using non-normal data for calculating CUSUM (Ryan and Faddy 2001). From Fig. 1 it is evident that the distribution is not normal. 6 The quality of journals (measured in terms of journal impact factor) in which the corpus is found might be of interest to some readers who may ask whether the increasing importance of the topic translates to publication in ‘higher impact’ journals. We used Scopus CiteScore 2018 and found no significant difference between the average impact factor of journals over time. We have not included a more detailed analysis of this point but would like to thank one of our anonymous reviewers for making this suggestion. 7 Nissen (1996) classifies publications into five categories—trade press, redesign cases, expert reengineering methodologies, academic investigations, and theory-building works. We did not classify any articles as trade press and adapted the categories to better suit the types of contributions we noticed in the corpus. 8 In case an article proposes a testable model/hypothesis and uses empirical data to test it, it would continue to be classified as ‘theory-building’ based on its primary contribution. 9 Since the number of papers with author-provided keywords exceeds the number of papers with WoS KeywordsPlus and since it may not be wise to merge author-provided and automated keywords, only author-provided keywords have been used for analysis. References Aho E. , Cornu J. , Georghiou L. et al. ( 2006 ) Creating an Innovative Europe. Report of the Independent Expert Group on R&D and Innovation (Report of the Independent Expert Group on R&D and Innovation appointed following the Hampton Court Summit). <https://www.eesc.europa.eu/en/documents/facing-challenge-lisbon-strategy-growth-and-employment> accessed 26 Apr 2019 Aldenius M. , Khan J. ( 2017 ) ‘ Strategic Use of Green Public Procurement in the Bus Sector: Challenges and Opportunities ’, Journal of Cleaner Production , 164 : 250 – 7 . doi: 10.1016/j.jclepro.2017.06.196 Google Scholar OpenURL Placeholder Text WorldCat Crossref Alic J. A. ( 2008 ) ‘ A Weakness in Diffusion: US Technology and Science Policy after World War II ’, Technology in Society , 30 / 1 : 17 – 29 . doi: 10.1016/j.techsoc.2007.10.005 Google Scholar OpenURL Placeholder Text WorldCat Crossref Alic J. A. ( 2015 ) ‘ Biofuel Battles: Politics, Policy, and the Pentagon ’, Energy Research & Social Science , 10 : 10 –1 8 . doi: 10.1016/j.erss.2015.06.006 Google Scholar OpenURL Placeholder Text WorldCat Crossref Apostol R. ( 2017 ) Trials and Tribulations in the Implementation of Pre-Commercial Procurement in Europe . T.M.C. Asser Press . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Aschhoff B. , Sofka W. ( 2009 ) ‘ Innovation on Demand-Can Public Procurement Drive Market Success of Innovations? ’, Research Policy , 38 / 8 : 1235 – 47 . doi: 10.1016/j.respol.2009.06.011 Google Scholar OpenURL Placeholder Text WorldCat Crossref Blondel V. D. , Guillaume J.-L. , Lambiotte R. et al. ( 2008 ) ‘ Fast Unfolding of Communities in Large Networks ’, Journal of Statistical Mechanics: Theory and Experiment , 2008 / 10 : P10008 . doi: 10.1088/1742-5468/2008/10/P10008 Google Scholar OpenURL Placeholder Text WorldCat Crossref Boland A. , Cherry G. , Dickson R. ( 2017 ) Doing a Systematic Review: A Student’s Guide . SAGE : London. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Boon W. , Edler J. ( 2018 ) ‘ Demand, Challenges, and Innovation. Making Sense of New Trends in Innovation Policy ’, Science and Public Policy , 45 / 4 : 435 – 47 . doi: 10.1093/scipol/scy014 Google Scholar OpenURL Placeholder Text WorldCat Crossref Bound K. , Puttick R. ( 2010 ) Buying Power? Is the Small Business Research Initiative for Procuring R&D Driving Innovation in the UK . NESTA . <&lt; https://www.nesta.org.uk/report/buying-power/> accessed 26 Apr 2019 Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Cerin P. ( 2006 ) ‘ Bringing Economic Opportunity into Line with Environmental Influence: A Discussion on the Coase Theorem and the Porter and van der Linde Hypothesis ’, Ecological Economics , 56 / 2 : 209 – 25 . doi: 10.1016/j.ecolecon.2005.01.016 Google Scholar OpenURL Placeholder Text WorldCat Crossref Cheng W. , Appolloni A. , D’Amato A. et al. ( 2018 ) ‘ Green Public Procurement, Missing Concepts and Future Trends – A Critical Review ’, Journal of Cleaner Production , 176 : 770 – 84 . doi: 10.1016/j.jclepro.2017.12.027 Google Scholar OpenURL Placeholder Text WorldCat Crossref Chicot J. , Matt M. ( 2018 ) ‘ Public Procurement of Innovation: A Review of Rationales, Designs, and Contributions to Grand Challenges ’, Science and Public Policy , 45 / 4 : 480 – 92 . doi: 10.1093/scipol/scy012 Google Scholar OpenURL Placeholder Text WorldCat Crossref Cohen B. , Amoros J. E. ( 2014 ) ‘ Municipal Demand-side Policy Tools and the Strategic Management of Technology Life Cycles ’, Technovation , 34 / 12 : 797 – 806 . doi: 10.1016/j.technovation.2014.07.001 Google Scholar OpenURL Placeholder Text WorldCat Crossref Cohen J. ( 1960 ) ‘ A Coefficient of Agreement for Nominal Scales ’, Educational and Psychological Measurement , 20 / 1 : 37 – 46 . doi: 10.1177/001316446002000104 Google Scholar OpenURL Placeholder Text WorldCat Crossref Connell D. ( 2004 ) Exploiting the UK’s Science and Technology Base: How to Fill the Gaping Hole in UK Government Policy . TTP Ventures : Cambridge. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Dale-Clough L. ( 2015 ) ‘ Public Procurement of Innovation and Local Authority Procurement: Procurement Modes and Framework Conditions in three European Cities ’, Innovation: The European Journal of Social Science Research , 28 / 3 : 220 – 42 . doi: 10.1080/13511610.2015.1012709 Google Scholar OpenURL Placeholder Text WorldCat Crossref Dalpé R. ( 1994 ) ‘ Effects of Government Procurement on Industrial Innovation ’, Technology in Society , 16 / 1 : 65 – 83 . Google Scholar OpenURL Placeholder Text WorldCat Dalpé R. , DeBresson C. , Xiaoping H. ( 1992 ) ‘ The Public Sector as First User of Innovations ’, Research Policy , 21 / 3 : 251 – 63 . doi: 10.1016/0048-7333(92)90019-Z Google Scholar OpenURL Placeholder Text WorldCat Crossref Davis P. , Brady O. ( 2015 ) ‘ Are Government Intentions for the Inclusion of Innovation and Small and Medium Enterprises Participation in Public Procurement being Delivered or Ignored? An Irish Case Study ’, Innovation: The European Journal of Social Science Research , 28 / 3 : 324 – 43 . doi: 10.1080/13511610.2014.985192 Google Scholar OpenURL Placeholder Text WorldCat Crossref Detelj K. , Jagrič T. , Markovič-Hribernik T. ( 2016 ) ‘ Exploration of the Effectiveness of Public Procurement for Innovation: Panel Analysis of EU Countries’ Data ’, Lex Localis , 14 / 1 : 93 – 114 . doi: 10.4335/14.1.93-114(2016) Google Scholar OpenURL Placeholder Text WorldCat Crossref Edler J. , Georghiou L. ( 2007 ) ‘ Public Procurement and Innovation-Resurrecting the Demand Side ’, Research Policy , 36 / 7 : 949 – 63 . doi: 10.1016/j.respol.2007.03.003 Google Scholar OpenURL Placeholder Text WorldCat Crossref Edquist C. , Hommen L. ( 1999 ) ‘ Systems of Innovation: Theory and Policy for the Demand Side ’, Technology in Society , 21 / 1 : 63 – 79 . doi: 10.1016/S0160-791X(98)00037-2 Google Scholar OpenURL Placeholder Text WorldCat Crossref Edquist C. , Hommen L. , Tsipouri L. ( 2012 ) Public Technology Procurement and Innovation . Springer Science & Business Media : boston. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Edquist C. , Vonortas N. S. , Zabala-Iturriagagoitia J. M. ( 2015 ) ‘Introduction’. In: Edquist C. , Vonortas N. S. , Zabala-Iturriagagoitia J. M. , Edler J. (eds) Public Procurement for Innovation . Cheltenham : Edward Elgar . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Edquist C. , Zabala-Iturriagagoitia J. M. ( 2012 ) ‘ Public Procurement for Innovation as Mission-oriented Innovation Policy ’, Research Policy , 41 / 10 : 1757 – 69 . doi: 10.1016/j.respol.2012.04.022 Google Scholar OpenURL Placeholder Text WorldCat Crossref Edquist C. , Zabala-Iturriagagoitia J. M. ( 2015 ) ‘ Pre-commercial Procurement: A Demand or Supply Policy Instrument in Relation to Innovation ’, R&D Management , 45 / 2 : 147 – 60 . doi: 10.1111/radm.12057 Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Crossref European Commission ( 2007 ). ‘Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions - Pre-commercial Procurement: driving innovation to ensure sustainable high quality public services in Europe {SEC(2007) 1668}’. European Council ( 2000 ). ‘Lisbon European Council 23 and 24 March 2000: Conclusions of the Presidency’. Gee S. , Uyarra E. ( 2013 ) ‘ A Role for Public Procurement in System Innovation: The Transformation of the Greater Manchester (UK) Waste System ’, Technology Analysis & Strategic Management , 25 / 10 : 1175 – 88 . doi: 10.1080/09537325.2013.843660 Google Scholar OpenURL Placeholder Text WorldCat Crossref Ghisetti C. ( 2017 ) ‘ Demand-pull and Environmental Innovations: Estimating the Effects of Innovative Public Procurement ’, Technological Forecasting and Social Change , doi: 10.1016/j.techfore.2017.07.020 Google Scholar OpenURL Placeholder Text WorldCat Goel R. K. ( 2001 ) ‘ R&D Effects of Incomplete Procurement Contracts ’, Applied Economics Letters , Google Scholar OpenURL Placeholder Text WorldCat Hellsmark H. , Söderholm P. ( 2016 ) ‘ Innovation Policies for Advanced Biorefinery Development: Key Considerations and Lessons from Sweden ’, Biofuels, Bioproducts and Biorefining , 11 / 1 : 28 – 40 . doi: 10.1002/bbb.1732 Google Scholar OpenURL Placeholder Text WorldCat Crossref Hommen L. , Rolfstam M. ( 2009 ) ‘ Public Procurement and Innovation: Towards a Taxonomy ’, Journal of Public Procurement , 9 / 1 : 17 – 56 . Google Scholar OpenURL Placeholder Text WorldCat Hutton J. , Hartley K. ( 1985 ) ‘ The Influence of Health Service Procurement Policy on Research and Development in the UK Medical Capital Equipment Industry ’, Research Policy , 14 / 4 : 205 – 11 . doi: 10.1016/S0048-7333(85)80012-3 Google Scholar OpenURL Placeholder Text WorldCat Crossref Jones M. D. , Peterson H. L. , Pierce J. J. et al. ( 2015 ) ‘ A River Runs Through It: A Multiple Streams Meta‐Review ’, Policy Studies Journal , 44 / 1 : 13 – 36 . doi: 10.1111/psj Google Scholar OpenURL Placeholder Text WorldCat Crossref Kattel R. , Lember V. ( 2010 ) ‘ Public Procurement as an Industrial Policy Tool: An Option for Developing Countries? ’, Journal of Public Procurement , 10 / 3 : 368 – 404 . Google Scholar OpenURL Placeholder Text WorldCat Kok W. ( 2004 ) Facing the Challenge—The Lisbon Strategy for Growth and Employment (Report from the High Level Group chaired by Wim Kok). <https://www.eesc.europa.eu/en/documents/facing-challenge-lisbon-strategy-growth-and-employment> accessed 26 Apr 2019 Lacy S. , Riffe D. ( 1996 ) ‘ Sampling Error and Selecting Intercoder Reliability Samples for Nominal Content Categories ’, Journalism & Mass Communication Quarterly , 73 / 4 : 963 – 73 . doi: 10.1177/107769909607300414 Google Scholar OpenURL Placeholder Text WorldCat Crossref Landoni M. ( 2017 ) ‘ Innovation Policy in Progress. Institutional Intermediation in Public Procurement of Innovation: Satellite Telecommunications in Italy ’, R&D Management , 47 / 4 : 583 – 94 . doi: 10.1111/radm.12246 Google Scholar OpenURL Placeholder Text WorldCat Crossref Lember V. , Kattel R. , Kalvet T. ( 2015 ) ‘ Quo Vadis Public Procurement of Innovation? ’, Innovation: The European Journal of Social Science Research , 28 / 3 : 403 – 21 . doi: 10.1080/13511610.2015.1043245 Google Scholar OpenURL Placeholder Text WorldCat Crossref Lember V. , Kalvet T. , Kattel R. ( 2011 ) ‘ Urban Competitiveness and Public Procurement for Innovation ’, Urban Studies , 48 / 7 : 1373 – 95 . doi: 10.1177/0042098010374512 Google Scholar OpenURL Placeholder Text WorldCat Crossref León H. C. M. , Farris J. A. ( 2011 ) ‘ Lean Product Development Research: Current State and Future Directions ’, Engineering Management Journal , 23 / 1 : 29 – 51 . doi: 10.1080/10429247.2011.11431885 Google Scholar OpenURL Placeholder Text WorldCat Crossref Li Y. ( 2017 ) ‘ Assessing Public Procurement of Innovation as a Cross-Domain Policy: A Framework and Application to the Chinese Context ’, Review of Policy Research , 34 / 3 : 421 – 46 . doi: 10.1111/ropr.12229 Google Scholar OpenURL Placeholder Text WorldCat Crossref Li Y. , Georghiou L. ( 2016 ) ‘ Signaling and Accrediting New Technology: Use of Procurement for Innovation in China ’, Science and Public Policy , 43 / 3 : 338 – 51 . doi: 10.1093/scipol/scv044 Google Scholar OpenURL Placeholder Text WorldCat Crossref Lichtenberg F. R. ( 1988 ) ‘ The Private R&D Investment Response to Federal Design and Technical Competitions ’, American Economic Review , 78 / 3 : 550 . Google Scholar OpenURL Placeholder Text WorldCat Lundvall B. -Å. ( 1985 ) Product innovation and user-producer interaction, industrial development. Research Series 31, Aalborg: Aalborg University Press. Malerba F. , Nelson R. , Orsenigo L. et al. ( 2008 ) ‘ Public Policies and Changing Boundaries of Firms in a “history-friendly” Model of the Co-evolution of the Computer and Semiconductor Industries ’, Journal of Economic Behavior & Organization , 67 / 2 : 355 – 80 . doi: 10.1016/j.jebo.2006.10.010 Google Scholar OpenURL Placeholder Text WorldCat Crossref Markovic-Hribernik T. , Detelj K. ( 2016 ) ‘ Simulation of Public Procurement’s Impact on Innovativeness of EU Countries ’, International Journal of Simulation Modelling/ , 2: 249 – 61 . doi: 10.2507/IJSIMM15(2)5.333 Google Scholar OpenURL Placeholder Text WorldCat Crossref Martin J. F. ( 1996 ) The E Public Procurement Rules: A Critical Analysis . Oxford : Clarendon Press . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Matelly S. , Lima M. ( 2016 ) ‘ The Influence of the State on the Strategic Choices of Defence Companies: the Cases of Germany, France, and the UK after the Cold War ’, Journal of Innovation Economics & Economics , 20 : 61 – 88 . doi: 10.3917/jie.020.0061 Google Scholar OpenURL Placeholder Text WorldCat Crossref Mazzucato M. ( 2013 ) The Entrepreneurial State: Debunking Public Vs. Private Sector Myths . Anthem Press : London. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC McHugh M. L. ( 2012 ) ‘ Interrater Reliability: the Kappa Statistic ’, Biochemia Medica , 22 / 3 : 276 – 82 . Google Scholar OpenURL Placeholder Text WorldCat Meehan J. , Menzies L. , Michaelides R. ( 2017 ) ‘ The Long Shadow of Public Policy; Barriers to a Value-Based Approach in Healthcare Procurement ’, Journal of Purchasing and Supply. Retrieved from Scopus , doi: 10.1016/j.pursup.2017.05.003 Google Scholar OpenURL Placeholder Text WorldCat Mowery D. C. , Langlois R. N. ( 1996 ) ‘ Spinning Off and Spinning On(?): The Federal Government Role in the Development of the US Computer Software Industry ’, Research Policy , 25 / 6 : 947 – 66 . doi: 10.1016/0048-7333(96)00888-8 Google Scholar OpenURL Placeholder Text WorldCat Crossref Myoken Y. ( 2010 ) ‘ Demand-Orientated Policy on Leading-Edge Industry and Technology: Public Procurement for Innovation ’, International Journal of Technology Management , 49 / 1–3 : 196 – 219 . Google Scholar OpenURL Placeholder Text WorldCat Nelson, R. R. (ed) ( 1993 ) National Innovation Systems: A Comaparative Analysis , Oxford University Press: Oxford Series 31, Aalborg: Aalborg University Press. Nelson R. R. , Langlois R. N. ( 1983 ) ‘ Industrial Innovation Policy: Lessons from American History ’, Science , 219 / 4586 : 814 – 8 . Google Scholar OpenURL Placeholder Text WorldCat Nissen M. E. ( 1996 ) ‘ A Focused Review of the Reengineering Literature: Expert Frequently Asked Questions ’, Quality Management Journal , 3 / 3 : 52 – 66 . doi: 10.1080/10686967.1996.11918744 Google Scholar OpenURL Placeholder Text WorldCat Crossref OECD ( 2011 ). ‘Demand-side Innovation Policies’. doi: 10.1787/9789264098886-en Page E. S. ( 1954 ) ‘ Continuous Inspection Schemes ’, Biometrika , 41 / 1 /2: 100 – 15 . doi: 10.2307/2333009 Google Scholar OpenURL Placeholder Text WorldCat Crossref Pavitt K. , Walker W. ( 1976 ) ‘ Government Policies Towards Industrial Innovation: A Review ’, Research Policy , 5 / 1 : 11 – 97 . doi: 10.1016/0048-7333(76)90017-2 Google Scholar OpenURL Placeholder Text WorldCat Crossref Pham M. T. , Rajić A. , Greig J. D. , et al. ( 2014 ) ‘ A Scoping Review of Scoping Reviews: Advancing the Approach and Enhancing the Consistency ’, Research Synthesis Methods , 5 / 4 : 371 – 85 . doi: 10.1002/jrsm.1123 Google Scholar OpenURL Placeholder Text WorldCat Crossref Phillips, W., Knight, L., Caldwell, N., and Warrington, J. (2007) Policy through procurement-The introduction of digital signal process (DSP) hearing aids into the English NHS. Health Policy, 80/ 1 : 77 – 85 . https://doi.org/10.1016/j.healthpol.2006.03.0082. Pickernell D. Kay A. , Packham, G. and Miller, C. ( 2011 ) Competing agendas public procurement: An empirical analysis of opportunities limits the UK for SMEs . Environment and Planning C: Government and Policy , 29/4: 641–658. https://doi.org/10.1068/c10164b3. Google Scholar OpenURL Placeholder Text WorldCat Rigby J. ( 2016 ) ‘The Impact of Pre-commercial Procurement on Innovation’. In: Handbook of Innovation Policy Impact , pp. 382 – 402 . Edward Elgar Publishing . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Rolfstam M. ( 2009 ) ‘ Public Procurement as an Innovation Policy Tool: The Role of Institutions ’, Science and Public Policy , 36 / 5 : 349 – 60 . doi: 10.3152/030234209X442025 Google Scholar OpenURL Placeholder Text WorldCat Crossref Rolfstam M. ( 2013 ) Public Procurement and Innovation: The Role of Institutions . Edward Elgar Publishing Cheltenham, UK. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Rolfstam M. ( 2015 ) ‘ Public Procurement of Innovation for A Better World: A Consolidation or a New Beginning? ’, Innovation: The European Journal of Social Science Research , 28 /3: 211 – 9 . Google Scholar OpenURL Placeholder Text WorldCat Rothwell R. ( 1981 ) ‘ Pointers to Government Policies for Technical Innovation ’, Futures , 13 / 3 : 171 – 83 . doi: 10.1016/0016-3287(81)90084-7 Google Scholar OpenURL Placeholder Text WorldCat Crossref Rothwell R. ( 1984 ) ‘ Technology-Based Small Firms and Regional Innovation Potential: The Role of Public Procurement ’, Journal of Public Policy , 4 / 4 : 307 – 32 . doi: 10.1017/S0143814X00002774 Google Scholar OpenURL Placeholder Text WorldCat Crossref Ruttan V. W. ( 2006 ) Is War Necessary for Economic Growth?: Military Procurement and Technology Development . Oxford University Press : New York. Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Ryan T. P. , Faddy B. J. ( 2001 ) The Effect of Non-Normality on the Performance of CUSUM Procedures . In Frontiers in Statistical Quality Control 6 (pp. 176–193). Physica, Heidelberg . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Shyu J. Z. , Chiu Y.-C. , Yuo C.-C. ( 2001 ) ‘ A Cross-National Comparative Analysis of Innovation Policy in the Integrated Circuit Industry ’, Technology in Society , 23 / 2 : 227 – 40 . doi: 10.1016/S0160-791X(01)00011-2 Google Scholar OpenURL Placeholder Text WorldCat Crossref Sorenson C. , Kanavos P. ( 2011 ) ‘ Medical Technology Procurement in Europe: A Cross-Country Comparison of Current Practice and Policy ’, Health Policy , 100 / 1 : 43 – 50 . doi: 10.1016/j.healthpol.2010.08.001 Google Scholar OpenURL Placeholder Text WorldCat Crossref Srinivas S. ( 2006 ) ‘ Industrial Development and Innovation: Some Lessons from Vaccine Procurement ’, World Development , 34 / 10 : 1742 – 64 . doi: 10.1016/j.worlddev.2006.02.004 Google Scholar OpenURL Placeholder Text WorldCat Crossref Torbica A. , Cappellaro G. ( 2010 ) ‘ Uptake and Diffusion of Medical Technology Innovation in Europe: What Role for Funding and Procurement Policies ’, Journal of Medical Marketing , 10 / 1 : 61 –6 9 . doi: 10.1057/jmm.2009.48 Google Scholar OpenURL Placeholder Text WorldCat Crossref Uyarra E. , Flanagan K. ( 2010 ) ‘ Understanding the Innovation Impacts of Public Procurement ’, European Planning Studies , 18 / 1 : 123 – 43 . doi: 10.1080/09654310903343567 Google Scholar OpenURL Placeholder Text WorldCat Crossref Uyarra, E., Flanagan K. , Magro, E., and Zabala-Iturriagagoitia J. M . ( 2017 ) Anchoring the innovation impacts of public procurement to place: The role of conversations. Environment and Planning C? Politics and Space, 35/5: 828. doi: 10.1177/2399654417694620 10.1177/2399654417694620 . Crossref Walker H. , Preuss L. ( 2008 ) ‘ Fostering Sustainability through Sourcing from Small Businesses: Public Sector Perspectives ’, Journal of Cleaner Production , 16 / 15 : 1600 – 9 . doi: 10.1016/j.jclepro.2008.04.014 Google Scholar OpenURL Placeholder Text WorldCat Crossref Walsh V. ( 1993 ) ‘ Demand, Public Markets and Innovation in Biotechnology ’, Science and Public Policy , 20 / 3 : 138 – 56 . doi: 10.1093/spp/20.3.138 Google Scholar OpenURL Placeholder Text WorldCat Crossref Yeow J. , Edler J. ( 2012 ) ‘ Innovation Procurement as Projects ’, Journal of Public Procurement , 12 / 4 : 472 – 504 . Google Scholar OpenURL Placeholder Text WorldCat Zelenbabic D. ( 2015 ) Fostering innovation through innovation friendly procurement practices: case study Danish local government procurement . Innovation : 28/3: 261–281. https://doi.org/10.1080/13511610.2015.10567244 Google Scholar OpenURL Placeholder Text WorldCat © The Author(s) 2020. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Public procurement and innovation: a systematic literature review
JF - Science and Public Policy
DO - 10.1093/scipol/scaa029
DA - 2020-08-01
UR - https://www.deepdyve.com/lp/oxford-university-press/public-procurement-and-innovation-a-systematic-literature-review-6SuCy0e4Hj
SP - 490
EP - 502
VL - 47
IS - 4
DP - DeepDyve
ER -