Abstract Political scientists, economists, legal scholars and others write widely on Bilateral Investment Treaties (BITs). A central (and ongoing) debate in the literature is an empirical question of whether or not BITs ‘work.’ Do BITs catalyze investments into the capital importing countries that sign them? In this article I make the case that one likely driver of the literature’s inability to come to a firm conclusion on this matter is that the FDI flow data typically used in it are poorly suited to the task. To make that case I note various ways in which those data are (a) poorly measured, and (b) poorly suited to the question even if they were measured well. I. INTRODUCTION Political scientists, economists, legal scholars and others have written widely on Bilateral Investment Treaties (BITs).2 BITs, as readers of the ICSID Review are surely aware, are treaties between two countries establishing for foreign direct investors a set of legal rights covering investments flowing from one signatory country to the other. Importantly, BITs typically allow those investors recourse to investor-state dispute settlement (ISDS) in the event that those treaty-defined rights are violated. For capital importing countries BITs represent a tradeoff: states give up a degree of policy autonomy, and do so in the hope it catalyzes foreign direct investment (FDI) by credibly assuring direct investors that their long term, illiquid investments will not be undermined by adverse policy actions in the future. Academia’s fascination with BITs is well grounded. BITs are an important part of the legal architecture underpinning global FDI, and an increasingly controversial one at that. We should know as much about these treaties as we can. But even apart from their substantive importance, BITs pose a series of interesting questions about the politics of international economics. To list a few: Why are some governments but not others willing to expose their countries to potential litigation in order to attract more FDI? Why is that tradeoff more popular at some times than others?3 What are the economic implications of being sued under ISDS?4 What drives variation in BIT design, and what are the consequences of that variation?5 Do BITs complement existing domestic political institutions, or do they substitute for them?6 But above all else, academic work on BITs asks a simple empirical question: Do BITs catalyze investments into the capital importing (ie developing) countries that sign them?7 In other words: Do BITs work? The answer to that question is of more than ‘just’ academic interest; it is also of obvious interest to the trade and investment professionals who must decide whether or not to pursue BITs as a tool for FDI promotion. Catalyzing inward FDI flows is not the only rationale for signing BITs, but the case for signing them (and especially for subjecting the state to ISDS) is substantially stronger if doing so can reasonably be expected to deliver economic benefits. Whether BITs work is a simple enough question to ask, but a surprisingly difficult one to answer. Social scientists have been unable to reach widely accepted conclusions. Indeed, that inability has become one of literature’s defining features. So much so that it is almost a cliché for contributions to this literature to be framed explicitly around settling debates stemming from ‘mixed results,’ turning those mixed results themselves into of object of academic curiosity; somewhat apart from the underlying phenomenon which they cannot quite seem to get a clear picture of.8 Multiple issues contribute to our inability as a field to come to a firm conclusion about BITs’ effects.9 This paper focuses on one of those: empirically testing theories of BITs effects on MNC behavior invariably requires a quantification of that behavior, and the FDI flow data typically used are unsuitable for the task. That may sound like a cheap critique: virtually all non-trivial macroeconomic data are at least slightly ill suited for social scientific purposes, and yet the vast majority of those necessarily imperfect data can nonetheless be used to generate reliable estimates of socio-economic processes.10 But the problems that FDI flows data pose to the accumulation of reliable knowledge about BITs effects are especially substantial, and it is important that they be better understood within academic and practitioner communities alike.11 This article describes why FDI flow data are poorly designed to test most theories relating to BITs, and does so from the perspective of empirical social science. It is not an exhaustive cataloging of the various imperfections of FDI flow data, but rather a contextualization of some of those imperfections in a broader framework about measurement error and it effects on quantitative empirical social science. In doing so, this article sets out to do three things, with each ‘task’ occupying its own section. Section II describes what makes macroeconomic data useful or not for social scientific purposes, without any specific references to FDI data. The briefly stated lesson from that section is that macroeconomic data need not be perfect to be useful—which is good, because a large portion of empirical social science would otherwise be rendered impossible—but that some categories of imperfections are more tolerable than others. Section III describes what FDI flow data measure, and how these data and their application to the study of BITs relate to the previously described concepts. Suffice it here to say that I argue that FDI flow data fall into the ‘less tolerably imperfect’ category. Section IV asks what, if anything, can be done to produce reliable knowledge and to interpret existing data in more fruitful ways. II. WHAT DO SOCIAL SCIENTISTS WANT OUT OF MACROECONOMIC DATA? Macroeconomic statistics are an unavoidable tool for empirical research in wide swathes of political science, economics, sociology and other fields invested in what can broadly be termed ‘political economy’ relationships. To know if a policy intervention or social attribute causes growth, we need some measure of growth. To know if a set of treaties or institutions promotes trade, we need some measure of trade volumes. And so on. To be sure, there has been innovation in these fields to get beyond reliance these data12, but for many important questions they are unavoidable and will remain so. The ubiquity of these data belies the difficulty and expense of generating reliable macroeconomic data suitable for making cross-national comparisons, and even more so the difficulty of doing so in a developing country context. The data that we rely on are inevitably imperfect, and sometimes wildly so.13 That is not a new revelation: these imperfections have long been known in academic and policy circles—going back at least to Morgenstern.14 However, the perfect need not be the enemy of the good. Imperfect data can be used to build up reliable social scientific knowledge. What is important is not whether the data are imperfect but in what ways. To that end, we can broadly categorize data imperfections as those that would generate ‘noise’ in a regression context, and those that would generate ‘bias.’ A. Noise Virtually all non-trivial economic data are subject to some degree of measurement error. To illustrate with an example, consider that macroeconomic statistics are often built on raw data derived from surveys of representative samples of firms. Some of those surveys will contain errors. Perhaps an accountant was sleepy and mistyped a numeral on the survey, or misunderstood which quantities the survey was requesting. Perhaps the modeling assumptions used to generate the representative sample were flawed in some way, and the sample of surveyed firms was unrepresentative of the broader population. The result of these inevitable errors is measurement error. In more exact terms, we can think of any particular data point Y* as decomposable into the true value Y plus some error component ɛ. Perhaps Y represents the actual but unobservable rate of fixed capital formation, while Y* represents the observable but slightly inaccurate version of that statistic, which in this case rendered inaccurate by the sleep-deprived accountant. While sleep deprived accountants might not be entirely random, the resulting imprecision very likely is. As long as sleep-deprived accountants are as likely to mistakenly replace a larger numeral for a smaller one, and vice-versa, the average value of ɛ should be 0 and the consequent measurement error is essentially ‘noise.’ The more significant the gap between Y and Y*, the ‘noisier’ is the data. But even in noisy data the mean value of Y* in large samples should converge towards Y. The dispersion of data around the mean is likely over-estimated, but not biased in one direction or another. Regressions that use noisy data produce imprecise estimates of the mis-measured variables’ relationships to other variables.15 The result is an increased probability of a false negative (a so-called type II error), wherein the researcher has posited an accurate theory but the data are so imprecise that they fail to produce convincing evidence of it. False negatives undermine the accumulation of social scientific knowledge. But as long as consumers of social scientific findings constrain the instinct to over-interpret null findings, they are not misleading. Empirical social science can and does live with a fairly significant degree of random measurement error. If anything, random measurement error can have a conservatizing effect, making it harder to find evidence of weaker relationships. A relationship that consistently reveals itself despite imperfectly measured data is likely a strong relationship. B. Bias A more threatening consequence of imprecisely measured data is bias. Bias occurs when the error component in a piece of data is systematic, rather than random. To illustrate, reconsider the above-noted parable of the sleepy accountant generating random measurement error by putting typos in firm surveys. In this reconsideration the errors are non-random. Some firms’ surveys are inaccurate because well-rested accountants had an incentive to underreport certain activities. And consider further the possibility that those incentives are indicative of political or economic conditions that exist in some countries but not others. That would results in non-random measurement error, and a biased measure. Put differently: the average gap between Y and Y*—ɛ—in the above scenario is negative, not zero. The mean value of Y* in large samples of data subject to non-random measurement error will not converge towards Y. Moreover, the average value of ɛ is not uniform. Rather, ɛ is systematically more negative in countries whose political-economic environments encourage underreporting. Regressions using biased measures can produce biased results. Consider the possibility that you wanted to measure the effect of ‘law and order’ on economic outcome Y. If the absence of ‘law and order’ is, or is correlated with, the qualities that lead to underreporting, regressions using the observable Y* as a proxy for Y will produce coefficient estimates that are biased upward. That is, even if there is no actual relationship between ‘law and order’ and Y, the positive relationship between ‘law and order’ and ɛ (and thus Y*) will generate the appearance of one in the regressions. If the true relationship between ‘law and order’ and Y is negative, the systematic measurement error in Y* would bias coefficient estimates upwards towards 0 and possibly higher. That sort of error (a so-called type I error) is serious, and renders the use of the data more problematic. The above discussion assumes that measurement error stems from an inability to perfectly measure complex concepts. That is an important source of measurement error, but it is not the only one. Sometimes the economic quantity we wish to observe is not actually measured by anyone, perfectly or imperfectly. That happens quite a lot: there is no limit to the number of economic phenomena that social scientists can theorize about, but only so many of them are actually recorded, let alone recorded consistently across multiple countries and over a sustained period of time. And that gap between even a perfectly measured piece of economic data and the slightly different economic concept it is meant to proxy for generates its own, potentially non-random, gap between Y and Y*, with it all of the same pathologies described above. In short, then, the ideal for political economists using macroeconomic data for empirical research is for that data to be perfectly measured. Because that ideal can very rarely (if ever) be reached, the practical ideal is for measurement error to be random and relatively small. The most vexing problems come from data for which measurement error is non-random, and when the source of that non-randomness is empirically or theoretically connected to the political object whose relationship to economic outcomes we wish to study. It is very hard to generate reliable estimates of politics’ effects on economics when politics also affects how economies are measured. And that difficulty underscores the importance of empirical researchers understanding—and grappling with—the process by which data are made and the extent to which that process intersects with politics or other concepts whose effects on economics we often wish to understand. I provide an (abridged) overview of those issues as they pertain to FDI flow data below. III. WHAT DO FDI FLOW DATA MEASURE? FDI stock (alternatively known as the FDI position) represents the total financial claims of foreign parent corporations on local affiliates, net of the claims that those same affiliates have on the parent corporation.16 FDI flows, which are more commonly used in research on BITs, measure the sum of capital transactions that affect that position occurring within a single year.17 For brevity’s sake, I focus discussion on the more directly relevant FDI flow data. But many of the same problems noted below apply equally to stock data. Indeed many of the developing countries whose FDI inflows are most relevant to testing BITs’ efficacy calculate the FDI stock as the over-time accumulation of FDI inflows.18 Inward FDI flow data aggregate across several categories of financial flows, which occur between affiliates firms in one country and their MNC parents abroad. More specifically, FDI flow data sum across flows of intercompany debt (including bonds, loans, trade credits and other forms of debt relationships), equity (shares, reserves and capital contributions) and reinvested earnings, which refer to the parent’s share of affiliate earnings that are retained locally, rather than remitted back to the parent MNC. Any of these components can flow in either direction. The equity portion of FDI inflows increases when the parent MNC establishes the affiliate, when it purchases additional shares in the affiliate, or when it makes other forms of capital contributions.19 The equity portion decreases if the parent divests its interest in the affiliate, or in the rare case of an affiliate taking an equity position in the parent. The debt portion of FDI inflows increases as the parent MNC loans money to the foreign affiliate, and decreases when the foreign affiliate loans money back to the parent MNC. The reinvested earnings component increases when revenue is earned and invested, but decreases when the affiliate loses money, or if dividends paid out to the investor are greater than the affiliate’s recorded income. While we typically use FDI flows to represent the scale of MNCs investment activity over the course of a year, it is more accurate to think of FDI flows as the net influence that MNCs’ foreign affiliates have on the host country’s capital account. FDI inflows are an important concept, and there are many important questions for which it is precisely the concept that analysts ought to be measuring. But it is not, strictly speaking, the concept that most theories relating to BITs implicate. If BITs ‘work’ it is by decreasing corporate perceptions of political risk, and catalyzing foreign corporations to undertake more political risk-sensitive investments in signatory countries. Those activities should, in turn, generate an increase in FDI inflows, but the link between the two is far more tenuous than is initially obvious. That is to say: FDI inflows are only ever imperfectly measured, and even if they were perfectly measured, many factors that have nothing whatsoever to do with the extension of political risk-sensitive investments can affect FDI inflows. At best those divergences create noisy measures and regression estimates that fail to validate accurate hypotheses; at worst they generate biased and more severely misleading regression estimates. Below I note a few of the more notable, and more problematic, drivers of random and non-random measurement error in the measurement of FDI inflows. A. Not All Assets Counted by FDI Flows Serve Long-Term or Political Risk Sensitive Investment The central premise of using FDI inflows to test BITs’ efficacy is that those flows represent illiquid investments meant to serve long-term projects that are especially vulnerable to political risk. The link between the two may not be as tight as commonly thought, however. In fact, much of what counts as FDI is neither illiquid, nor intended to serve long-term investments. Blanchard and Acalin show that FDI inflows are remarkably responsive to quarterly changes in US monetary policy.20 As US interest rates go up, FDI inflows to developing countries go down; as US interest rates go down, FDI inflows to developing countries go up. This is precisely what we would expect of portfolio flows,21 but it runs counter to expectations for direct flows. Kerner and Lawrence similarly shows that a change in the United States' tax law—the 2004 Homeland Investment Act—reduced US outbound FDI substantially,22 but had no perceptible impact on the actual operations of American MNCs' foreign affiliates. It suggests that much of what counts as FDI is neither illiquid nor part of any production process, let alone a production process that would be sensitive to the political risk reduction offers by BITs. The equation of FDI flows with long term-oriented capital flows is at best incomplete and at worst misleading. Much of what counts of FDI is better thought of as FPI, and subject to a whole other set of political pressures. Kerner and Lawrence note similar patterns with respect to the balance sheets of foreign affiliates of American MNCs.23 Excluding financial industry firms and holding companies, the 2004 benchmark survey of US MNCs indicates that 43 per cent of US MNCs’ affiliates balance sheets are accounted for by current assets—ie highly liquid assets that are either in case or expected to be converted into cash within a year under normal operating conditions. Plant, property and equipment, including the value of physical structures, land, machinery, equipment, and the book value of land, timber, mineral and similar rights made up 24 percent of assets. That is not to say that only plant, property should count as FDI—firms obviously need liquid forms of capital to operate—but that a substantial portion of the assets that we assume are illiquid and potentially unrecoverable in the event of an adverse regulatory event, are not, and may not be especially relevant to questions of political risk. At a minimum, the liquidity—and, apparently, short-term profit seeking—of much FDI creates a gap between what we want to measure—capital invested in the service of long-term political risk-sensitive projects—and the FDI inflows that we use to proxy for it. More problematically, the noise introduced by that gap varies non-randomly by industry and by country. PPE makes up a much higher percentage of the corporate balance sheet in the mining and utilities industries, for example, than it does in the service sector. And the extent to which FDI inflow is really FPI inflow in disguise varies by country as well, with the largest discrepancies being found in countries with amenable corporate taxes and the relevant corporate tax treaties. (Though, as Blanchard and Acalin show, the negative elasticity between FDI inflows and US interest rates is present even in countries not generally know as tax havens.) Neither the variation in a country’s industrial mix nor the extent to which governments have installed MNC-friendly policies exist in a vacuum, are very likely correlated with the extent of a country’s BIT programs, and are thus potential sources of bias. B. Not All Assets Meant to Serve Long-Term Investments are Financed Through FDI MNC’s foreign affiliates finance their activities through a variety of means. FDI—defined as financial transfers from their parent-corporation, plus reinvested earnings—are a main source, but many foreign affiliates rely on local financial markets to help meet their financing needs, or on capital raised from non-affiliated firms.24 This capital is not included in FDI flow data. This is not a problem with the data per se. Capital raised by foreign affiliates on local financial markets, or through investments from non-affiliated firms that do not implicate a lasting interest by a foreign entity quite clearly are not FDI, and should not be counted as such. But the distinction between capital raised on local capital markets or loaned from the parent corporation is not an obviously meaningful one to most substantive applications, suggesting that even a perfectly measured version of FDI inflows may inaccurately characterize the extent of capital flowing into foreign affiliates. I am unaware of any reason why the extent of BIT protection would be conditioned by the extent of external debt financing. The fact of such financing generates another divergence between the object we would like to observe—capital invested in foreign owned, political-risk sensitive projects—and what we can observe—the portion of the capital invested in foreign owned, political-risk sensitive projects financed via FDI. At a minimum this generates downward bias in the data. All FDI inflow data understate the object of interest by ignoring assets financed locally (or through other non-FDI means). But, as with the extent of liquid capital being counted as FDI, the extent to which foreign affiliates rely on local financing is not random, and very likely the product of the same political-economic milieu that gives rise to BITs. Firms make more use of external financial markets in countries with strong creditor rights.25 The extent of creditor’s rights—effectively a distribution of rents between creditors and borrowers—is assuredly endogenous to politics and related (albeit perhaps obliquely) to a country’s BIT program. That financing is also quite clearly a product of the interest rate environment, which is also and more directly related to the likelihood of countries signing and ratifying BITs.26 C. FDI Flow Data Focus on the Proximate Rather than the Ultimate Investor FDI flow data capture flows between the immediate source of the capital and the immediate recipient of FDI. That would be fine if the immediate sender and recipient of capital were also the ultimate sender and recipients of capital, but much of global FDI does not work like that. Rather, much FDI is routed through intermediate destinations for tax purposes. In the case of ‘round tripped’ capital, domestic investors route their domestic investments through a foreign entity to take advantage of the legal and tax protections available to investors there. The use of offshore investment vehicles in this way can lead FDI flow data to suggest warped views of where capital is being deployed and by whom.27 The result is the appearance of FDI in intermediate countries where no political risk-bearing investment actually exists, the confusion of domestic, but round-tripped capital with ‘real’ FDI, and the inability of dyadic datasets to accurately characterize FDI’s source country and, thus, whether or not an investment is BIT-protected. The scale of the problem is substantial. Holding companies accounted for 36 percent of the United States direct investment position abroad in 2008.28 Sutherland and Anderson note that between 2003 and 2010, around 80 percent of outbound Chinese FDI flows were destined for Hong Kong, the Cayman Island or the British Virgin Islands.29 Borga reports that reconstituting FDI statistics around the ultimate rather than the immediate investor suggests the Spanish firms are, in fact, the second largest source of Spanish FDI inflows, and that half—half!—of Russian FDI consists of round tripped capital originating in Russia but stored in tax havens such as Cyprus before returning to be invested.30 Those misattributions can be problematic in a number of ways. For one, the economic and social implications of round-tripped capital are different than the economic and social implications of foreign direct investment. Round tripped capital does not bring with it foreign managerial expertise, or technological spillover, not does it meaningfully contribute to the domestic capital stock. And, as such, it invites a situation in which BITs could ‘work’ in the sense that the extra legal protections granted by round tripping capital through a BIT-covered jurisdiction might make long-term, political risk-sensitive investment more feasible and thus more common, but it would not carry with it the benefits that FDI—and thus, BITs—are generally thought to have. More generally, the use of offshore holding companies makes it very difficult to properly know how much FDI a country is actually getting, which FDI flows are protected by BITs and which are not, and where a country’s FDI inflows are coming from. Even if those matters are clearer when the data can be inspected on a more granular level, empirical social science in this area has of necessity relied on national aggregates in which those patterns are obscured. This feature of FDI data has some non-BIT related consequences as well. Consider that a 2015 surge in Irish FDI distorted that country’s GDP data to the point that it indicated a 26.3 percent annual growth rate. The Irish economy did not grow at 26.3 percent in 2015. The FDI that warped GDP figures was due to corporate inversions and other vehicles through which firms took advantage of the Irish corporate tax rate. That surge of capital had virtually no relationship with real economic activity in Ireland, and certainly no bearing or perceptions of political risk there.31 The fact that FDI inflow data is by definition divorced from actual production by foreign firms rendered Irish GDP virtually useless as s descriptor of the Irish economy. The empirical literature on BITs should apply a similar lesson. FDI inflows are a poor descriptor of the extent and location of political risk-sensitive economic activities. It should be especially keen to learn that lesson because there is absolutely nothing random about the distribution of this particular form of measurement error. That it happened to Ireland, and not France, has everything to do with political-economic decisions made by the Irish government and not by the French government. There is no reason to assume that the distortions cause by FDI flow data’s failure to properly locate the ultimate investor is not correlated with BITs and other efforts to provide a pro-corporate business environment. D. Countries Vary Substantially in their Treatment of Reinvested Earnings Of the three components of FDI inflow—intercompany debt, equity and reinvested earnings—reinvested earning stand out for being collected through firm surveys rather than by central banks as a byproduct of monitoring the balance of payments.32 That distinction is intuitive: reinvested earnings never actually cross a border, even if the money nominally transfers from MNC parent to foreign affiliate. That, in and of itself, is not a problem—many of the macroeconomic statistics social scientists rely on are based on firm surveys. But firm surveys are expensive to administer and difficult to do well, and those difficulties are especially binding on developing countries whose data are most relevant to the question of BITs’ efficacy. And as shown in Jerven and Kerner and Crabtree, there is nothing random or apolitical about which countries choose (or are able to choose) to carry out meaningful surveys of their economies.33 Indeed, those capacities are profoundly tied to the nature of the political system and, especially, a government’s relationship to international financial institutions. Those works focus on the production of GDP data, but similar issues appear to affect the recording of reinvested earnings data as well. The Survey of Implementation of Methodological Standards for Direct Investment (SIMSDI) notes that many developing countries and several developed countries do not perform the surveys necessary to collect reinvested earnings data, or do not report the data that they do collect as part of their FDI flow data.34 This leads some countries to systematically underreport FDI figures.35 Trends in this area are encouraging, but substantially divergent reporting standards persist today and are especially problematic for data prior to the year 2000.36 That suggests that some countries’ failure to report reinvested earnings is less akin to the sleepy accountant whose random errors generate noise in the data (and regressions using that data) than it is to accountants whose underreporting stems in some way from the political-economic context in which they exist. That is not to say that there is a direct link between a robust BIT regime and a country’s propensity to collect reinvested earnings data, but the assumption that measurement error in FDI is unrelated to BITs seems unjustified. It is reasonable to suspect, for example, that a government’s decision to commit resources to accurately measuring FDI and to attracting FDI through BITs are jointly determined by an underlying and unobservable appreciation for FDI’s development potential. More prosaically, an active relationship with the international financial institutions that provide assistance to national statistical offices and who typically recommend policies liberalize FDI policies could jointly cause more BITs and (at least the appearance of) more FDI. To the extent that the current or historical distribution of capacities to collect reinvested earnings data reflects either of those it poses substantial problems for our ability to discern BIT’s effects on FDI from the broader context’s effect on how FDI is measured. IV. WHAT IS TO BE DONE? To summarize, all non-trivial macroeconomic data are flawed to some degree, but FDI inflow data are flawed in ways that make it difficult to use them to make conclusive statements about the relationship between BITs and the investment behaviors of MNCs. FDI inflow data measure a set of financial transaction that are only tangentially related to the behaviors BITs are meant to catalyze, they measure those concepts imperfectly, and both forms of measurement error are very plausibly correlated with the scale and nature of a country’s BIT program. That is a big problem. Not so big that studies using these data should be summarily ignored, but big enough that the details of the specific study—the sample, the estimations strategy, etc.—should be considered with the deficiencies of FDI inflow data in mind. To the extent that studies using these data corroborate findings using other data, so much the better. But these data should not, on their own, be used as the basis for forming non-trivial beliefs about BITs and their consequences. There are some proactive things we can do as well. Perhaps the easiest and best thing we (as researchers and as consumers of research) can do is to move away from large-N studies that rely on a global sample of host countries over many decades. The only available data that could populate those samples are FDI inflow data and those data simply are not up to the task. Smaller studies based on more granular data, often from a single host and/or single source country, can be an invaluable adjunct to the broader studies that we already have. Those studies can, for example, examine the actual operations of foreign affiliates active in a host country, how much to commit to fixed capital, etc. rather than rely on FDI inflow data to form a proxy. Such studies are not a panacea; they trade off a degree of external validity—ie the extent to which dynamics observed in a study can be generalized to the broader population—for a greater degree of internal validity—ie the extent to which the study’s findings are believable on their own terms. But to the extent that multiple smaller scale studies tell a similar story it presents a more reasonable basis to form beliefs about BITs. A second, and related, prescription is for more qualitative work on the topic, especially work that includes more direct engagement between the researchers and the firms whose behaviors are implicated by their theories. Firms take in and process information about political risk in a variety of ways, and when they perceive that the political risk environment has shifted they can react to that in a variety of ways. Not all of those reactions should be expected to cleanly manifest in the aggregate FDI data. By design such studies could not speak to broader trends, but by describing firm reactions (and non-reactions) to BITs in more detail such studies could help inform the research designs that are used on the broader data sets. A final prescription that, again, applies equally to researchers and consumers, is to not over-react to the findings of any one particular study. Meta-analyses such as Bellak37 can be helpful, but so too can a greater degree of humility. In my opinion, no single study employing FDI inflow data is capable of providing dispositive evidence of BITs’ effects. The data are just not good enough. But by the same token, the extent of measurement error in these data means that the failure to find convincing evidence of such an effect using these data is not terribly surprising either. The absence of evidence really and truly is not evidence of absence. There is only so much that can be very confidently said about BITs’ effects on FDI flows using traditional FDI flow data. Moving away from that data necessarily means utilizing smaller, and not necessarily representative, samples. The smaller scale undermines any one study’s claims to definitely characterize BITs effects, but that, in my opinion, is a small and in any event, necessary price to pay for confidence that the various forms of measurement error noted above do not warp the results. Footnotes 2 It’s a crude measure, to be sure, but entering the phrase ‘bilateral investment treaty’ into the Google scholar database reveals over 20,000 available pieces of scholarship at least mentioning that phrase, with over 15,000 of those authored just in the last 10 years. 3 See eg Timm Betz and Andrew Kerner, ‘The Influence of Interest: Real US Interest Rates and Bilateral Investment Treaties’ (2016) 11(4) Rev Intl Org 419–48; Zachary Elkins, Andrew T Guzman and Beth A Simmons, ‘Competing for Capital: The Diffusion of Bilateral Investment Treaties, 1960–2000’ (2006) 60(4) Intl Org 811–46; Zachary Elkins, Andrew T Guzman and Beth A Simmons, ‘Competing for Capital: The Diffusion of Bilateral Investment Treaties, 1960–2000’ (2008) U Illinois L Rev 265; Andrew T Guzman, ‘Why LDCs Sign Treaties That Hurt Them: Explaining the Popularity of Bilateral Investment Treaties’ (1997) 38 Va J Intl L 639; Beth A Simmons, ‘Bargaining over BITs, Arbitrating Awards: The Regime for Protection and Promotion of International Investment’ (2014) 66(1) World Politics 12–46. 4 See Todd Allee and Clint Peinhardt, ‘Contingent Credibility: The Impact of Investment Treaty Violations on Foreign Direct Investment’ (2011) 65(3) Intl Org 401–32. 5 See Todd Allee and Clint Peinhardt, ‘Delegating Differences: Bilateral Investment Treaties and Bargaining over Dispute Resolution Provisions’ (2010) 54(1) Intl Stud Q 1–26; Eric Neumayer, Peter Nunnenkamp and Martin Roy, ‘Are Stricter Investment Rules Contagious? Host Country Competition for Foreign Direct Investment Through International Agreements’ (2016) 152(1) Rev World Econ 177–213; Axel Berger, Matthias Busse, Peter Nunnenkamp and Martin Roy, ‘Do Trade and Investment Agreements Lead to More FDI? Accounting for Key Provisions Inside the Black Box’ (2013) 10(2), Intl Econ and Econ Poly 247–75. 6 See Tom Ginsburg, ‘International Substitutes for Domestic Institutions: Bilateral Investment Treaties and Governance’ (2005) 25(1) Intl Rev Law Econ 107–23; G Sirr, John Garvey and Liam A Gallagher, ‘Bilateral Investment Treaties and Foreign Direct Investment: Evidence of Asymmetric Effects on Vertical and Horizontal Investments’ (2017) 35(1) Develop Poly Rev 93–113; Eric Neumayer and Laura Spess, ‘Do Bilateral Investment Treaties Increase Foreign Direct Investment to Developing Countries?’ (2005) 33(10) World Development 1567–85; Mark Hallward-Driemeier, ‘Do Bilateral Investment Treaties Attract FDI?: Only a Bit … and They Could Bite’ (2003) World Bank, Development Research Group, Investment Climate; Karl P Sauvant and Lisa E Sachs (eds). The Effect of Treaties on Foreign Direct Investment: Bilateral Investment Treaties, Double Taxation Treaties, and Investment Flows (OUP 2009) 660. 7 Among many, many other examples see Eric Neumayer and Laura Spess, L (n 6); Andrew Kerner, ‘Why Should I Believe You? The Costs and Consequences of Bilateral Investment Treaties’ (2009) 53(1) Intl Studies Q 73–102; Rod Falvey and Neil Foster-McGregor, ‘Heterogeneous Effects of Bilateral Investment Treaties’ (2017) 153(4) Rev World Econ 631–56; Hallward-Driemeier (n 6); Jennifer L Tobin and Susan Rose-Ackerman, ‘When BITs Have Some Bite: The Political-Economic Environment for Bilateral Investment Treaties’ (2011) 6(1) Rev Intl Org 1–32.; Matthias Busse, Jens Königer and Peter Nunnenkamp, ‘FDI Promotion Through Bilateral Investment Treaties: More than a Bit?’ (2010) 146(1) Rev World Econ 147–77; Peter Egger and Michael Pfaffermayr, ‘The Impact of Bilateral Investment Treaties on Foreign Direct Investment’ (2004) 32(4) J Comp Econ 788–804; Yoram Z Haftel, ‘Ratification Counts: US Investment Treaties and FDI Flows into Developing Countries‘ (2010) 17(2) Rev Intl Pol Econ, 348–77; Jeswald W Salacuse, ‘BIT by BIT: The Growth of Bilateral Investment Treaties and Their Impact on Foreign Investment in Developing Countries’ (1990) The Intl Lawyer 655–75. 8 Andrew Kerner and Jane Lawrence, ‘What’s the Risk? Bilateral Investment Treaties, Political Risk and Fixed Capital Accumulation’ (2014) 44(1) B J Pol Science 2; Falvey and Foster-McGregor (n 7) 635; Jason W Yackee, ‘Bilateral Investment Treaties, Credible Commitment, and the Rule of (International) Law: Do BITs Promote Foreign Direct Investment?’ (2008) 42(4) L Soc Rev 805–32. 9 Two other reasons, neither of which are the focus of this paper, are worth briefly mentioning. The first is that BITs are not randomly assigned to governments. They are often responses to a perception among countries that there is a significant potential for bilateral FDI flows, and often come as part of a broader (and not always easily perceptible) shift towards policies meant to catalyze that FDI. It is difficult in most circumstances to convincingly separate BITs’ effects from its context. A second hurdle worth mentioning is the lack of cumulative theoretical development. BITs’ effects on MNCs’ behaviors appears to be highly contingent. That is as it probably should be. BITs vary in their treaty design, and BIT signatories vary in their wealth, the quality of their domestic politics and investment environment, the ex ante plausibility of significant FDI flows between them, and the industrial mix of investment opportunities in them. It would be odd if BITs’ effects were not similarly heterogeneous. A substantial part of clarifying the empirical evidence on BITs’ effects thus requires a broad acceptance of when we should expect those effects in the first place. And such a consensus has been slow to build. 10 See eg Morten Jerven, Poor Numbers: How We Are Misled by African Development Statistics and What to Do About It (Cornell University Press 2013); Angus Deaton and Alan Heston, ‘Understanding PPPs and PPP-based national accounts’ (2010) 2(4) Am Econ J: Macroeconomics 1–35. ; Andrew Kerner and Charles Crabtree, ‘The IMF and the Political Economy of GDP Data Production (SocArXiv Papers, January 2018) <osf.io/preprints/socarxiv/qsxae> accessed 16 January 2018; Alexander J Yeats, ‘On the Accuracy of Economic Observations: Do Sub-Saharan Trade Statistics Mean Anything?’ (1990) 4(2) World Bank Econ Rev 135–56; Jerzy Rozanski and Alexander Yeats, ‘On the (In)Accuracy of Economic Observations: An Assessment of Trends in the Reliability of International Trade Statistics’ (1994) 44(1) J Dev Econ 103–30; Yosiko M Herrera and Devesh Kapur, ‘Improving Data Quality: Actors, Incentives, and Capabilities’ (2007) 15(4) Pol Analysis 365–86; Andrew Kerner, Morten Jerven and Alison Beatty, ‘Does It Pay to be Poor? Testing for Systematically Underreported GNI Estimates’ (2017) 12(1) Rev Intl Org 1–38. 11 This is not the only piece to make a similar argument. See eg Andrew Kerner, ‘What We Talk About When We Talk About Foreign Direct Investment’ (2014) 58(4) Intl Stud Q 804–15; Sjoerd Beugelsdijk, Jean-Francois Hennart, A Slangen and R Smeets, ‘Why and How FDI Stocks are a Biased Measure of MNE Affiliate Activity’ (2010) 41(9) J Intl Business Stud 1444–59; D Sutherland and J Anderson, ‘The Pitfalls of Using Foreign Direct Investment Data to Measure Chinese Multinational Enterprise Activity’ (2015) 221 China Q 21–48; Karl Sauvant, ‘Beware of FDI statistics!’ Columbia FDI Perspectives on Topical Foreign Direct Investment Issues, No 215 (2017) <http://ccsi.columbia.edu/files/2016/10/No-215-Sauvant-FINAL.pdf> accessed 12 October 2017. 12 See eg Christopher S Magee and John A Doces, ‘Reconsidering Regime Type and Growth: Lies, Dictatorships, and Statistics’ (2015) 59(2) Intl Stud Q 223–37; Angus S Deaton, Instruments of Development: Randomization in the Tropics, and the Search for the Elusive Keys to Economic Development (No w14690, National Bureau of Economic Research 2009). 13 See Morten Jerven, ‘Random Growth in Africa? Lessons from an Evaluation of the Growth Evidence on Botswana, Kenya, Tanzania and Zambia, 1965–1995’ (2010) 46(2) J Dev Stud 274–94; ‘Step Change: Revised Figures Show that Nigeria is Africa’s Largest Economy’ The Economist (4 December 2014) <https://www.economist.com/news/finance-and-economics/21600734-revised-figures-show-nigeria-africas-largest-economy-step-change> accessed 12 October 2017. 14 Oskar Morgenstern, On the Accuracy of Economic Observations (Princeton University Press 1950). 15 More precisely, increasing random measurement error in the dependent variable increases the standard errors associated with the regressions estimate; increasing random measurement error in the independent variable biases the coefficient estimate to zero. 16 Investment counts as foreign direct investment, as opposed to foreign portfolio investment, if the parent firm as at least a 10 percent stake in the foreign affiliate. Ten percent is regarded as sufficient to indicate a lasting interest, even if it does not necessarily indicate control. 17 Outward FDI flows represent FDI from parent firms within a country to their affiliates abroad. A country’s inward and outward FDI flows have (at least in a computational sense) nothing to do with each other. 18 While many countries calculate stock in that way, the value of the FDI stock is not, according to a strict application of the balance of payments manual, the same thing as the accumulation of FDI flows over time. If anything, FDI stock data is harder to calculate consistently across different contexts, and the resulting data are more problematic than flow data for social scientific purposes. See Kerner (n 11), Karl Sauvant, ‘Beware of FDI Statistics!’ Columbia FDI Perspectives, No 215 (18 December 2017) <https://ssrn.com/abstract=3089794> accessed 12 October 2017. Citations therein for references to problems that are unique to stock data. 19 Share purchasing provokes a bit of an accounting dilemma. Consider a parent MNC that acquires a lasting interest by purchasing 1 percent of a foreign firm’s shares every year for 11 years. The first 10 years of share acquisition do not count as FDI, because those acquisitions were made in the absence of a lasting interest in the firm. Only share purchases occurring after the 10 percent threshold is met count as FDI. 20 Oliver Blanchard and Julien Acalin, What Does Measured FDI Actually Measure? (No PB 16-17, Peterson Institute for International Economics 2016). 21 Robin Koepke, ‘Fed Policy Expectations and Portfolio Flows to Emerging Markets (2015) <https://ssrn.com/abstract=2456288> or <http://dx.doi.org/10.2139/ssrn.2456288> accessed 12 October 2017. 22 Kerner and Lawrence (n 8). 23 ibid. 24 See Sauvant (n 11) and citations therein. 25 Mihir A Desai, C Fritz Foley and James R Hines Jr, ‘A Multinational Perspective on Capital Structure Choice and Internal Capital Markets’ (2004) 59(6) J Finance 2451–87. 26 Betz and Kerner (n 3). 27 See eg Robert E Lipsey, ‘Foreign Direct Investment and the Operations of Multinational Firms: Concepts, History, and Data’ in E Kwan Choi, and James Harrigan (eds), Handbook of International Trade (Blackwell 2003); Marilyn Ibarra and Jennifer Koncz, ‘Direct Investment Positions for 2007’ (2008) 88(7) Survey of Current Bus 20–35; Beugelsdijk and others (n 11); Sutherland and Anderson (n 11); Sauvant (n 11). 28 Ibarra and Koncz (n 27). 29 Sutherland and Anderson (n 11). 30 Maria Borga, ‘New FDI Statistics: Looking Through Complex Ownership Structures to the Ultimate Source of FDI’ in Austrian Central Bank, Focus on External Trade (ACB 2017) 165. 31 Vincent Boland, ‘Irish Tell a Tale of 26.3% Growth Spurt’ Financial Times (London, 12 July, 2016) <https://www.ft.com/content/8a1ebc9c-4846-11e6-8d68-72e9211e86ab> accessed 12 October 2017. 32 For an alternative (and more detailed) exploration of some non-obvious aspects of collecting and counting reinvested earnings see International and Financial Accounts Branch Australian Bureau of Statistics, ‘IMF Committee on Balance of Payments Statistics and OECD Workshop on Interantional Investment, Issue Paper # 5A Reinvested Earnings’ (2004) <https://www. imf.org/External/NP/sta/bop/pdf/diteg5A.pdf> accessed 12 October 2017. 33 Morten Jerven, Africa: Why economists get it wrong (Zed Books 2015) Kerner and Crabtree (n 10). 34 IMF, Foreign Direct Investment Statistics: How Countries Measure FDI (IMF Press 2003). 35 See eg IMF (n 34); John Dunning and Sarianna Lundan, Multinational Enterprises and the Global Economy (Edward Elgar 2008) 12–15; UNCTAD, ‘Methods of Data Collection and National Policies in the Treatment of FDI’ (2002) <http://www.unctad.org/templates/Page.asp?intItemID=3157andlang=1> accessed 12 October 2017. 36 IMF (n 34). 37 Christian Bellak, ‘How bilateral investment treaties impact on foreign direct investment: A meta-analysis of public policy’ (MAER Network Colloquium, Vienna, August 2013). © The Author(s) 2018. Published by Oxford University Press on behalf of ICSID. All rights reserved. For Permissions, please email: firstname.lastname@example.org This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)
ICSID Review: Foreign Investment Law Journal – Oxford University Press
Published: Apr 12, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
All the latest content is available, no embargo periods.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud