Abstract In this article, we introduce and showcase how social media can be used to implement experiments in public administration research. To do so, we pre-registered a placebo-controlled field experiment and implemented it on the social media platform Facebook. The purpose of the experiment was to examine whether government funding to nonprofit organizations has an effect on charitable donations. Theories on the interaction between government funding and charitable donations stipulate that government funding of nonprofit organizations either decreases (crowding-out), or increases (crowding-in) private donations. To test these competing theoretical predictions, we used Facebook’s advertisement facilities and implemented an online field experiment among 296,121 Facebook users nested in 600 clusters. Through the process of cluster-randomization, groups of Facebook users were randomly assigned to different nonprofit donation solicitation ads, experimentally manipulating information cues of nonprofit funding. Contrary to theoretical predictions, we find that government funding does not seem to matter; providing information about government support to nonprofit organizations neither increases nor decreases people’s propensity to donate. We discuss the implications of our empirical application, as well as the merits of using social media to conduct experiments in public administration more generally. Finally, we outline a research agenda of how social media can be used to implement public administration experiments. Introduction Using social media has become an increasingly common activity as platforms like Facebook have become ubiquitous. For instance, to date there are 218 million Facebook users in the United States (Statistia 2017) with one-third of the US adult population logging into Facebook at least once a day (Public Religion Research Institute 2012). Social media platforms like Facebook or Twitter provide brand-new opportunities to implement online experiments studying citizen–state interactions. However, within public administration this potential has been largely untapped.1 Therefore, the purpose of this article is to introduce and showcase how social media, particularly Facebook, can be used to implement experiments in public administration research. As a consequence of digitization, large-scale social media experiments have been used increasingly in neighboring fields like political science, economics, or marketing (for an overview see Aral 2016). Examples include a wide range of topics, including political behavior (e.g., Bond et al. 2012), advertising (e.g., Bakshy et al. 2012a), product pricing (e.g., Ajorlou, Jabdabaie, and Kakhbod 2016), information propagation (e.g., Bakshy et al. 2012b), or emotional contagion (e.g., Kramer, Guillory and Hancock 2014). This trend is encouraging but it misses an important component of people’s online behavior, namely citizen–state interactions. Indeed, the advent of e-government and an increased presence of government agencies on social media platforms has led to the rise of online citizen–state interactions (Thomas and Streib 2003; Wukich and Mergel 2015). These interactions range from gathering information, for example, about how to fill out online applications, to crisis communication, and even complaints about poor services. In this study, we illustrate how to implement large-scale social media experiments on Facebook by examining interactions between nonprofit organizations seeking donations and their potential donors. In particular, we assess whether changes in government funding affect the levels of charitable income of nonprofit organizations. Theoretically, two competing mechanisms have been distinguished in explaining why levels of government funding may have an effect on private donations to nonprofit organizations. The crowding-out perspective argues that government funding would decrease people’s willingness to donate because donors as taxpayers perceive government funding as a substitution to their donations (Andreoni 1989; Warr 1982; Kim and Van Ryzin 2014). If they contributed already via taxes, why should they give in addition to them? Therefore, the crowding-out model predicts a decrease in private donations as a result of government funding. In contrast, the signaling model of crowding-in suggests that government funding is used by potential donors as an imperfect signal of an organization’s effectiveness (e.g., Borgonovi and O’Hare 2004; Rose-Ackerman 1981). In the absence of complete information about how a nonprofit organization will perform and operate with the funds at its disposal, government funding serves as an organization’s “quality stamp,” signaling the organization is not only trustworthy but also effective because it managed to receive competitive government grants. The crowding-in perspective, therefore, predicts an increase in private donations as a result of government funding. In this study, we test these somewhat competing claims in the context of a large-scale social media experiment. Conducted in a naturalistic setting, social media experiments, similar to conventional field experiments, combine high levels of internal validity with external validity. This allows us to test whether government funding crowds-in, or crowds-out, private donations. We implemented a field experiment on the social media platform Facebook by assigning clusters of approximately 300,000 Facebook users to donation solicitations of groups of real food banks. Using a pre-registered, placebo-controlled between-subjects design, groups of users were randomly allocated to three experimental conditions: (1) the control group (i.e., no funding information), (2) the placebo group (i.e., donation funded), and (3) the treatment group (i.e., government funded). As outcome measures, we monitored people’s revealed donation intentions by their click-through-rates (i.e., the frequency people clicked on the links in the ad solicitations), but also other behavioral measures such as website visits. Consistently, we find no direct evidence for either model, suggesting that public and private funding streams of nonprofit organizations do not seem to interact in the real world. In addition to these findings, we provide an overview of social media experiments and how they can be implemented in public administration research, including an agenda for studying online citizen–state interactions using large-scale social media experiments. The remainder of the study is as follow: in the next section, we review empirical applications of social media experiments in neighboring fields to provide an overview of the applicability of social media to conduct experiments in public administration. We then discuss our empirical application by first reviewing the literature on the crowding-out and crowding-in hypotheses. On this basis, we introduce our experimental research design and report the results of the experiment subsequently. In the final section, we draw implications for public administration research and practice from both, our empirical application and the review of innovative social media experiments. Conducting Social Media Experiments Before turning to the empirical application in this article, we provide an overview of the potential of social media to conduct experiments. Recent years have seen an increase in online field experiments implemented on social media platforms (Aral 2016). Indeed, companies like Amazon, Google, and Facebook constantly perform small experiments on their clients, for example through randomly altered website content where two different versions of the same website, online ad, or any other online parameter are randomly assigned to service users—a procedure marketers commonly refer to as A/B testing. In the past, social scientists have collaborated with major social media platforms to implement experiments. For example, Bond et al. (2012) implemented a 61-million-person political mobilization experiment on Facebook; similarly, Kramer, Guillory, and Hancock (2014) have implemented an online experiment to study emotional contagion among 690,000 Facebook users. Most studies on Facebook include advertisement related topics, however. For example, Bakshy and colleagues (2012a) study the effectiveness of social cues (i.e., peers’ associations with a brand) on consumer responses to ads for more than 5 million Facebook users. In another study, Bakshy et al. (2012b) look at the randomized exposure of links shared by peers of more than 250 million Facebook users and how it affects information sharing behavior on Facebook. In all of these cases, researchers had to work closely with Facebook to implement the process of randomization at the individual level of Facebook users. But, this would also mean that experimenting on Facebook would be limited to those with industry contacts. In the following, we report from recent experiments that have been conducted without industry collaboration. We aim to showcase how social media platforms like Facebook or Twitter can be used by scholars or government agencies to implement experiments in a relatively straightforward manner. Ryan (2012) was one of the first social scientists to use Facebook’s advertisement facilities to conduct research without having to collaborate with Facebook directly. Similar to the empirical application we report in this article, he randomly assigned clusters of individuals to different advertisements instead of randomizing on the user level (see Teresi and Michelson 2015 for an alternative approach2). To do so, he used Facebook’s advertisement facilities, which allow targeting ads to Facebook users on a number of demographic characteristics, such as age and gender, but also zip code (see also Ryan and Brockman 2012). Based on these user parameters, researchers can predetermine clusters of users and randomly allocate them to varying ad content. This is what Ryan did in his study. In particular, he looked at how advertisements that evoke emotions such as anger or anxiety affect information seeking behavior. He then used cluster-level click-through-rates as a dependent variable. Across three studies and more than 1.8 million impressions grouped into 360 clusters in total, he found consistent evidence that political advertisements that ought to evoke anger increase users’ proclivity to click through to a website. In other words, anger makes us click. A similar methodological approach was also used by Ryan and Brader (2017) who studied partisan selective exposure to election messages during the 2012 US presidential elections, using a total of 846 clusters of Facebook users. Similar applications also exist in fields like marketing or economics. Aral and Walker (2014) for instance report from an experiment conducted with 1.3 million users of a Facebook application to test how social influence in online networks affects consumer demand. They experimentally manipulated users’ network embeddedness and the strength of their social ties, finding that both increase influence in social networks. Wang and Chang (2013) studied a similar topic looking at whether social ties and product-related risks influence purchase intentions of Facebook users that were recruited via an online application. Although they found that online tie strength leads the higher purchase intentions, product-related risks had no direct effect on purchase intentions. In another Facebook study by Broockman and Green (2013), users were exposed to different types of political candidate advertisements over the course of 1 week. Like Ryan (2012), they randomized clusters of individuals, instead of individuals themselves. However, since they had access to public voter registries, they targeted 32,000 voters, which they assigned to 1,220 clusters across 18 age ranges, 34 towns, and 2 genders. These clusters of Facebook users were assigned to one of four experimental conditions: a control group with no advertisement, and three different types of advertisements that ought to increase Facebook users’ name recognition of the candidate. The innovation that Brookman and Green’s study introduced was that they used contact information from public voter records to gather public opinion data from these individual voters through telephone interviews later on. Since the creation of clusters was done on the basis of assigning 32,000 registered voters to 1,220 clusters, they had detailed contact information of registered voters that belong to each respective cluster. In other words, they were able to link cluster assignment on Facebook with attitudinal outcomes from survey data, such as candidate’s name recognition, positive impression of the candidate, whether people voted for the candidate, and whether they recall having seen the ad. Social media experiments exist outside of Facebook also. Gong et al. (2017) conducted a large-scale experiment among the social microblogging service Sina Weibo (i.e., the “Chinese Twitter”). They examined the return-on-investment of company tweets on viewers of TV shows. To do so, they randomly assigned 98 different TV shows into three experimental conditions: (1) the control group, where there is no tweet sent out about the particular TV shows; (2) the tweet condition, where each show is tweeted by the company; and (3) a tweet and retweet condition, where each show is tweeted by a company and retweeted by a so-called “influencer.” TV show viewing percentages were used as an outcome measure, finding that both tweeting and tweeting coupled with retweeting boost TV show views relative to the shows in the control group. In other words, social media efforts of TV companies result in a significant increase in viewers. They also found that retweets of influencers are more effective in generating new viewers than tweets by the companies. In another Twitter experiment, Coppock, Guess, and Ternovski (2016) looked at online mobilization behavior. In particular, authors were interested in whether Twitter users could be encouraged to sign a petition. To do so, they randomly divided 8,500 followers of a US nonprofit advocacy group into three experimental conditions. In the first stage, the nonprofit organization published a tweet in which its followers were encouraged to sign a petition. All three groups were exposed to the public tweet. In the second stage, a treatment condition received a direct message with a similar request, referring to them as “followers,” another treatment condition got the same direct message, but they were referred to as “organizers,” whereas the control group received no direct message. On this basis, authors checked whether subjects either retweeted or signed the petition. Other notable examples of using social media platforms like Twitter to implement experiments involve studying social media censorship in China (King, Pan, and Roberts 2013), the effectiveness of repeated political messages on twitter followers of politicians (Kobayashi and Ichifuji 2015), the effectiveness of news media (King, Schneer, and White 2017), or racist online harassment (Munger 2017). The aforementioned examples provide rich inspiration for conducting social media experiments in public administration research. Social media experiments have the distinct advantage that they combine the internal validity of experiments with an increased realism and external validity. In this sense, they are a subtype of conventional field experiments, which are conducted in an online environment where people interact via social media. In addition, social media experiments can easily be conducted on large-scale samples using a variety of unobtrusive outcome measures to assess respondents’ revealed behaviors. They are therefore a viable option to complement survey-based experiments that often employ stated preferences (i.e., attitudes, evaluative judgments, or behavioral intentions), which make up the majority type of experiments implemented in public administration to date (Li and Van Ryzin 2017). People increasingly interact with government and government agencies using social media platforms like Twitter and Facebook (Mergel 2012). Scholars and government agencies alike can implement social media experiments to test the effectiveness of using these relatively new channels of communication and information provision. Examples may include assessing whether providing information on social media about the performance of government agencies affect citizen trust in those agencies, or may lead citizens to desirable behaviors, including coproduction. Indeed, implementing such innovative experimental designs in the context of online citizen–government interactions may be a viable avenue for future experimentation in public administration research. In the following, we introduce an empirical application of an online social media experiment that examines in how far government funding and charitable donation intentions interact. Empirical Application: How Government Funding and Private Donations Interact An impressive body of literature has emerged from various disciplines that focus on the issue of whether government funding would displace (crowding-out effect) or leverage (crowding-in effect) private contributions to nonprofit organizations. The impact of government funding on private donations to nonprofits largely rests on how potential donors and nonprofits themselves strategically respond to government funding of nonprofit activities (Tinkelman 2010; Lu 2016). We focus on the strategic responses of private donors in the present analysis: how would donors change their levels of charitable giving when a nonprofit organization is supported by government funding. The literature distinguishes two contrasting models of how government-funded nonprofits are perceived, which ultimately affects charitable donations via processes of crowding-in or crowding-out. Early crowding-out theory assumes that private donors are altruistic and care about the optimal level of public goods provision. Donors as taxpayers would consider government funding as their contributions through taxation and thus perceive it as a perfect substitute for voluntary donations. In this way, increases in government support would lower the need for additional private contributions. Therefore, when a nonprofit receives more support from the government, private donors would consciously reduce their giving to this organization. As a result, there is a dollar-for-dollar replacement between private giving and government funding (Roberts 1984; Warr 1982). This pure altruism assumption was later challenged by Andreoni’s (1989) model of impure altruism, which predicts that donors are also motivated by a “warm-glow”—the utility from the act of giving to help others. In this impure altruism line of reasoning, government funding and private giving would not completely substitute each other. As a result, there may exist a crowding-out effect between these two funding sources, but the magnitude of the effect is less than the dollar-for-dollar model that pure altruism would predict. On the other hand, private donors might consider government funding favorably and become more willing to contribute to government-funded nonprofits because they perceive them as more competent and/or needy. Crowding-in theory proposes that government funding may stimulate charitable contributions in two ways. First, when donors do not have complete knowledge concerning beneficiary nonprofits and their programs, government funding serves as a direct signal of the nonprofit’s quality and reliability (Rose-Ackerman 1981). Indeed, to be funded by government agencies, nonprofit organizations have to go through a competitive merit-based selection process and meet financial and programmatic requirements (Lu 2015; Suárez 2011). Therefore, the receipt of government funding can be perceived by uninformed donors as an indicator of trustworthiness and competence. Second, government funding also is considered as a signal of unmet social needs, calling for more donor attention and further facilitating the leveraging effect of government funding (Brooks 1999; Okten and Weisbrod 2000). There exist a rich body of empirical studies in support of the crowding-out hypothesis (e.g., Andreoni 1993; Andreoni and Payne 2011; Brooks 2000; De Wit, Bekkers, and Broese van Groenou 2017; Dokko 2009; Hughes, Luksetich, and Rooney 2014; Kingma 1989; Steinberg 1987) and the crowding-in model (e.g., Borgonovi and O’Hare 2004; De Wit and Bekkers 2016; Heutel 2014; Khanna and Sandler 2000; Lu 2016; Okten and Weisbrod 2000; Smith 2007). Most recently, De Wit and Bekkers (2016) and Lu (2016) respectively employed meta-analytical techniques to aggregate existing empirical evidence on crowding-in and crowding-out. Both studies conclude a significant positive association between government funding and private donations, even though the magnitude of the relationship is trivial. The above-mentioned body of literature on crowding-out and crowding-in greatly advances our understanding of the complex interaction between government funding and private donations. However, it generally suffers from two limitations. First, the majority of existing empirical literature testing the crowding-in or crowding-out effect employs observational data. Although observational studies enable scholars to explore the association between the two revenue sources, drawing causal inferences remains challenging (Blom-Hansen, Morton, and Serritzlew 2015; James, Jilke, and Van Ryzin 2017a). Second, both crowding-in and crowding-out lines of reasoning assume that potential donors possess perfect information about the nonprofits they might want to donate to, especially whether these organizations are funded by government and to what extent. However, this assumption might not be true in the real world (De Wit et al. 2017; Horne, Johnson, and Van Slyke 2005; Krasteva and Yildirim 2013). For example, Horne, Johnson, and Van Slyke (2005) used public opinion data to demonstrate that individual donors do not necessarily have complete information on the financial structures of their beneficiary organizations and subsequently link donation decisions to the levels of government funding. In recent years, scholars began to employ experimental designs to address these two limitations (i.e., endogeneity and imperfect information) in the crowding-out and crowding-in literature (e.g., Eckel, Grossman, and Johnston 2005; Kim and Van Ryzin 2014; Ottoni-Wilhelm, Vesterlund, and Xie 2017; Wasif and Prakash 2017). Existing experimental studies testing crowding-out/-in effects usually include manipulations of the existence and the level of direct government support to beneficiaries, and then measure variations in of subjects’ donations. Methodologically, experimental designs are advantageous over observational settings in terms of their internal validity when testing crowding-out/-in effects because experimental studies create a controlled environment of information exchange to rule out confounding factors. As a result, scholars are provided with more direct evidence of the causal linkage between government support and charitable giving (Blom-Hansen, Morton, and Serritzlew 2015; James, Jilke, and Van Ryzin 2017a). Table 1 reviews the experimental studies of crowding-out/-in effects to date, including type, setting, and results. As can be seen in Table 1, most experimental studies employ laboratory experimental designs, primarily using two specific experimental paradigms. One is the public goods game (e.g., Andreoni 1993; Eckel et al. 2005; Isaac and Norton 2013) and the other type of experimental setting employed in the literature is the dictator game (e.g., Blanco, Lopez, and Coleman 2012; Konow 2010; Korenok, Millner, and Razzolini 2012). Despite different experimental paradigms, most of the laboratory experiments report a partial crowding-out effect between government funding and charitable contributions (see also De Wit and Bekkers 2016). Table 1. Experimental Studies on Crowding-in and Crowding-out Effects Reference Type of experiment Experimental setting Subjects Finding Andreoni (1993) Lab Public goods game Students Crowding-out Chan et al. (1996) Lab Public goods game Students No effect Bolton and Katok (1998) Lab Dictator game Students Crowding-out Chan et al. (2002) Lab Public goods game Students Crowding-out Sutter and Weck-Hannemann (2004) Lab Public goods game Students Crowding-out Eckel et al. (2005) Lab Public goods game Students Crowding-out Güth et al. (2006) Lab Public goods game Students Crowding-out Galbiati and Vertova (2008) Lab Public goods game Students Crowding-in Hsu (2008) Lab Public goods game Students Crowding-out Reeson and Tisdell (2008) Lab Public goods game Students Crowding-out Konow (2010) Lab Dictator game Students Crowding-out Blanco et al. (2012) Lab Dictator game Tourists No effect Gronberg et al. (2012) Lab Public goods game Students Crowding-out Korenok et al. (2012) Lab Dictator game Students Crowding-out Issac and Norton (2013) Lab Public goods game Students Crowding-out Lilley and Slonim (2014) Lab Contribution decision making Students Crowding-out Galbiati and Vertova (2014) Lab Public goods game Students Crowding-in Kim and Van Ryzin (2014) Survey Hypothetical vignette General population sample (US) Crowding-out Wasif and Prakash (2017) Survey Hypothetical vignette General population sample (Pakistan) No effect Ottoni-Wilhelm et al. (2017) Lab Contribution decision making Students Crowding-out Reference Type of experiment Experimental setting Subjects Finding Andreoni (1993) Lab Public goods game Students Crowding-out Chan et al. (1996) Lab Public goods game Students No effect Bolton and Katok (1998) Lab Dictator game Students Crowding-out Chan et al. (2002) Lab Public goods game Students Crowding-out Sutter and Weck-Hannemann (2004) Lab Public goods game Students Crowding-out Eckel et al. (2005) Lab Public goods game Students Crowding-out Güth et al. (2006) Lab Public goods game Students Crowding-out Galbiati and Vertova (2008) Lab Public goods game Students Crowding-in Hsu (2008) Lab Public goods game Students Crowding-out Reeson and Tisdell (2008) Lab Public goods game Students Crowding-out Konow (2010) Lab Dictator game Students Crowding-out Blanco et al. (2012) Lab Dictator game Tourists No effect Gronberg et al. (2012) Lab Public goods game Students Crowding-out Korenok et al. (2012) Lab Dictator game Students Crowding-out Issac and Norton (2013) Lab Public goods game Students Crowding-out Lilley and Slonim (2014) Lab Contribution decision making Students Crowding-out Galbiati and Vertova (2014) Lab Public goods game Students Crowding-in Kim and Van Ryzin (2014) Survey Hypothetical vignette General population sample (US) Crowding-out Wasif and Prakash (2017) Survey Hypothetical vignette General population sample (Pakistan) No effect Ottoni-Wilhelm et al. (2017) Lab Contribution decision making Students Crowding-out View Large Table 1. Experimental Studies on Crowding-in and Crowding-out Effects Reference Type of experiment Experimental setting Subjects Finding Andreoni (1993) Lab Public goods game Students Crowding-out Chan et al. (1996) Lab Public goods game Students No effect Bolton and Katok (1998) Lab Dictator game Students Crowding-out Chan et al. (2002) Lab Public goods game Students Crowding-out Sutter and Weck-Hannemann (2004) Lab Public goods game Students Crowding-out Eckel et al. (2005) Lab Public goods game Students Crowding-out Güth et al. (2006) Lab Public goods game Students Crowding-out Galbiati and Vertova (2008) Lab Public goods game Students Crowding-in Hsu (2008) Lab Public goods game Students Crowding-out Reeson and Tisdell (2008) Lab Public goods game Students Crowding-out Konow (2010) Lab Dictator game Students Crowding-out Blanco et al. (2012) Lab Dictator game Tourists No effect Gronberg et al. (2012) Lab Public goods game Students Crowding-out Korenok et al. (2012) Lab Dictator game Students Crowding-out Issac and Norton (2013) Lab Public goods game Students Crowding-out Lilley and Slonim (2014) Lab Contribution decision making Students Crowding-out Galbiati and Vertova (2014) Lab Public goods game Students Crowding-in Kim and Van Ryzin (2014) Survey Hypothetical vignette General population sample (US) Crowding-out Wasif and Prakash (2017) Survey Hypothetical vignette General population sample (Pakistan) No effect Ottoni-Wilhelm et al. (2017) Lab Contribution decision making Students Crowding-out Reference Type of experiment Experimental setting Subjects Finding Andreoni (1993) Lab Public goods game Students Crowding-out Chan et al. (1996) Lab Public goods game Students No effect Bolton and Katok (1998) Lab Dictator game Students Crowding-out Chan et al. (2002) Lab Public goods game Students Crowding-out Sutter and Weck-Hannemann (2004) Lab Public goods game Students Crowding-out Eckel et al. (2005) Lab Public goods game Students Crowding-out Güth et al. (2006) Lab Public goods game Students Crowding-out Galbiati and Vertova (2008) Lab Public goods game Students Crowding-in Hsu (2008) Lab Public goods game Students Crowding-out Reeson and Tisdell (2008) Lab Public goods game Students Crowding-out Konow (2010) Lab Dictator game Students Crowding-out Blanco et al. (2012) Lab Dictator game Tourists No effect Gronberg et al. (2012) Lab Public goods game Students Crowding-out Korenok et al. (2012) Lab Dictator game Students Crowding-out Issac and Norton (2013) Lab Public goods game Students Crowding-out Lilley and Slonim (2014) Lab Contribution decision making Students Crowding-out Galbiati and Vertova (2014) Lab Public goods game Students Crowding-in Kim and Van Ryzin (2014) Survey Hypothetical vignette General population sample (US) Crowding-out Wasif and Prakash (2017) Survey Hypothetical vignette General population sample (Pakistan) No effect Ottoni-Wilhelm et al. (2017) Lab Contribution decision making Students Crowding-out View Large In addition to laboratory experiments, there are a few experimental studies that employ survey experiments to test the crowding-in and crowding-out propositions. For example, Kim and Van Ryzin (2014) conducted an online survey experiment with 562 participants and found that an arts nonprofit with government funding would receive about 25% less private donations than an identical hypothetical organization without government funding. In contrast, Wasif and Prakash’s (2017) survey experiment with 530 respondents in Pakistan reported that federal funding would not change respondents’ willingness to donate to a hypothetical faith-based educational nonprofit. When meta-analyzing results from experimental studies only, De Wit and Bekkers (2016) find substantially different results compared to observational studies, with experimental studies showing a considerable crowding-out effect and nonexperimental studies a very small crowding-in effect. There are two potential explanations for these differences. A first possibility would be that observational studies on crowding-out/-in are plagued by endogeneity, and hence the discrepancies in results may be a product of the comparatively poor internal validity of observational research designs. A second possibility would be that findings predominately produced within stylized settings such as economic games or hypothetical scenarios may hardly extrapolate beyond the laboratory. Or in other words, people may behave differently in lab and survey experiments as in the real world. Indeed, a recent systematic comparison between laboratory and field experiments concluded that the ability of stylized experiments to extrapolate social preferences from the lab to the field is limited, at best (Galizzi and Navarro-Martinez, Forthcoming; see also Levitt and List 2007). We respond to these competing interpretations by implementing a naturalistic field experiment using social media. Method In the following, we report the experimental design for this study. Doing so, we follow James, Jilke, and Van Ryzin’s (2017b) recommended reporting guidelines for experiments in public administration research. Experimental Design, Subject Recruitment, and Treatment We designed a large-scale placebo-controlled online field experiment on the social networking platform Facebook by purchasing ads using Facebook’s advertisement facilities. These ads were designed to solicit donations to human service nonprofits in New York City (NYC). We randomly allocated groups of Facebook users to three different ads of local human service organizations. Depending on experimental conditions, these ads included information cues about (1) government funding (treatment group), (2) donation funding (placebo group), or (3) no such information (control group) as depicted in Figure 1. The experimental design was pre-registered at the AEA RCT registry3 and received ethics approval from Rutgers University’s Institutional Review Board. Figure 1. View largeDownload slide Experimental Design. Figure 1. View largeDownload slide Experimental Design. We chose to target Facebook users who live in NYC because this allowed us to leverage local information about government funding of human service organizations (i.e., real food banks in NYC) in the experimental design. All ads included a donation appeal, encouraging Facebook users to donate to local food banks through a webpage they could access by clicking on the ad. Once clicked, and depending on the experimental condition, users were directed to one of three web pages where Food Banks from NYC4 were listed and hyperlinked. Facebook users in the control condition (i.e., no funding information) were directed to the Rutgers Observatory of Food Banks webpage. Here, all NYC Food Banks were listed and hyperlinked. Those subjects who were allocated to the placebo condition (i.e., donation-funded), were directed to the Rutgers Observatory of Donation-Funded Food Banks webpage. The presence of a placebo condition (i.e., donation-funded) allows us to test whether any differences between treatment and control is due to the mere provision of funding information. On this website, all donation-funded5 food banks from NYC were listed and hyperlinked. In the treatment condition (i.e., government-funded), Facebook users were sent to the Rutgers Observatory of Government-Funded Food Banks website, where all Food Banks that were classified as government-funded were listed and hyperlinked. Unique ads were developed for the experiment. Figure 2 shows the ads that were randomly presented to groups of Facebook users. Taking the ad of the control condition as a baseline, the treatment and placebo ads look exactly like the control ad, except for four important differences: Figure 2. View largeDownload slide Facebook Ads (Control versus Placebo versus Treatment Conditions). Figure 2. View largeDownload slide Facebook Ads (Control versus Placebo versus Treatment Conditions). 1) A different sponsor was used: i. Rutgers Observatory of Food Banks for the control group; ii. Rutgers Observatory of Donation-Funded Food Banks for the placebo group; iii. Rutgers Observatory of Government-Funded Food Banks for the treatment group. 2) The ad’s headline was changed from “Donate to Food Banks in your area” to “Donate to government-funded Food Banks in your area,” or “Donate to donation-funded Food Banks in your area” respectively. 3) The heading under the image changed from “Food Banks in NYC” to “Government-funded Food Banks in NYC,” or “Donation-funded Food Banks in NYC” respectively. 4) The two-sentence-description used “government-funded Food Banks in NYC” and “donation-funded Food Banks in NYC” instead of “Food Banks in NYC.” Cluster Randomization Randomization into experimental conditions was not performed on the individual level, but on the level of groups of Facebook users (see also Ryan 2012 for a similar approach). This was done because Facebook’s commercial advertisement facilities do not permit different ads to be randomized on the individual level (Ryan and Brockman 2012). Instead, users were classified into groups. This approach makes use of Facebook’s ability to target users based on demographics. More precisely, we stratified Facebook users into clusters of users by zip code, age, and gender. Cluster-randomized experiments are widely used across the social and medical sciences where groups of individuals—for example, people living in an area, or classmates—are jointly randomized into experimental conditions (e.g., Campbell and Walters 2014; Hayes and Moulton 2017). For our experiment, we randomized demographic clusters of Facebook users, rather than users nested within these clusters. Doing so, we first collected all 177 NYC zip codes and randomly chose 100. In a second step, we grouped respondents of 18 to 60 years into three age categories (18–27, 28–38, and 39–60). We composed these three groups to have an approximately equal “potential reach” across strata (potential reach refers to the approximate number of Facebook users that can be exposed to an ad). Each of the three groups has a potential reach of about 1.5 to 1.6 million users who live in NYC. Next, we created 600 clusters by taking 100 zip codes * 3 age categories * 2 genders. These 600 clusters were randomized into one of the three experimental conditions, so that each experimental condition ended up with 200 clusters with a potential reach of about 1 million Facebook users6. An important concern when analyzing data from cluster-randomized experiments is that clusters vary in size so that the conventional difference-in-means estimator would be biased (Middleton and Arronow 2015). Indeed, the 600 clusters we have come up with vary in (anticipated) size, with a potential reach between fewer than 1,000 to 110,000 Facebook users (mean potential reach of approximately 5,000 users). Therefore, we followed Gerber and Green’s (2012) suggestion and blocked on cluster size (200 groups of 3 clusters each) in the randomization procedure, so that the difference-in-means estimator can be used without risk of bias. The blocked cluster-randomization was performed using Stata 14. The actual implementation of the experiment was done using Facebook’s Adverts Manager interface. We purchased 600 ads and targeted them to specific demographic clusters (stratified by zip code, age categories, and gender). A total of 600 clusters (i.e., 200 per experimental condition) were purchased for $10 each and our ads were constructed to pay per impression7. All ads ran on the same day for 24 h, from 6 a.m. EST on August 21, 2017 until 6 a.m. the following day. Although the sample size of the clusters was pre-determined, the number of Facebook users who have been actually exposed to the ads was not; however, the potential reach across experimental conditions was approximately the same (see footnote 5). This is so because Facebook uses its own “Optimization for Ad Delivery” algorithm (which is the same across experimental conditions) to determine who will be exposed to an ad. In essence, Facebook users are shown ads which—according to Facebook’s algorithm—have the highest probability of resulting in an impression or click.8 The 600 clusters had a joint potential reach of 2,972,500, but 296,121 Facebook users were actually exposed to the ads as a result the Facebook’s ad bidding procedure. Outcomes Our primary outcome of interest is subjects’ revealed intention to donate. Or in other words, we look at whether the ads encourage people to click on them. We interpret link clicks as people’s intention to donate because the ads explicitly solicit donations by encouraging people to click. Indeed, clicking is commonly seen as a crucial antecedent for purchase intentions in the marketing literature (Zhang and Mao 2016). Against this background and given the explicit ad solicitations for donations we used in the ads (i.e., “click on one of the listed websites to make a donation”), we assume that ad clicks are a motivational precursor for donation intentions. Doing so, we use the so-called unique-outbound-link-click-rate per cluster as our primary dependent variable. The unique-outbound-click-rate (also referred to as click-through-rate) denotes a cluster’s actual reach (i.e., the number of people who saw our ads at least once) divided through its unique outbound clicks (i.e., the number of people who performed a click that took them off Facebook-owned property). In our case, the outbound click led users to one of three web pages—depending on the experimental condition they were assigned to—we have developed for this study. On these web pages, which were labeled as either the Rutgers Observatory of Food Banks (control group), the Rutgers Observatory of Food Banks (placebo group), or the Rutgers Observatory of Government-Funded Food Banks (treatment group), subjects were informed about the purpose of the study and provided with a hyperlinked list of either government-funded, donation-funded, or all food banks in NYC. As a secondary, though not pre-registered, measure, we analyze unique page views (i.e., the number of times people visited their assigned Observatory webpage at least once) as provided by Google Analytics for the three web pages. We calculated the percentage of unique page views relative to the number of people who saw the ads at least once. This measure serves as a robustness check for our primary measure of interest as it provides the actual number of users who actually ended up on the web pages. Results The ads we ran on Facebook reached a total 296,121 NYC Facebook users, resulting in a mean click-through-rate (CTR) of 0.48 (standard deviation of 0.49) for a total of 600 clusters (which is our unit of analysis). This means that of the approximately 300,000 Facebook users who saw the ads at least once, 0.48% performed at least one click which was intended to take them off Facebook and to one of the respective Observatory web pages that were designed for this study. To determine whether we find evidence for a crowding-out/-in effect of government funding on donation intentions, we analyzed whether there are systematic differences in users’ click-through-rates between experimental conditions. The left panel in Figure 3 depicts the results of tests on the equality of means between control, placebo (i.e., donation), and treatment (i.e., government) groups. Independent two-sample t-tests were conducted to compare (1) control and treatment conditions (mean difference of 0.00, p = 0.99; Cohen’s d = 0.00), (2) placebo and treatment conditions (mean difference of 0.06, p = 0.21; Cohen’s d = −0.13), and (3) control and placebo conditions (mean difference of −0.06, p = 0.17; Cohen’s d = −0.14) (see also Table A1 in the Appendix). In sum, all three experimental conditions are statistically indistinguishable from each other in terms of their CTRs. In addition, they are of trivial magnitude with very small values in terms of effect sizes (i.e., Cohen’s d). This means that we neither find support for the crowding-out nor the crowding-in hypothesis. In addition, there is also no significant difference between control and placebo conditions. Figure 3. View largeDownload slide Experimental Results. Figure 3. View largeDownload slide Experimental Results. When analyzing unique page views of the Observatory web pages that were created for this study, we used page visit data from Google Analytics, which provided us with the total number of unique page visits for each experimental condition. Therefore, the unit of analysis for this measure is individual Facebook users (n = 296,121). In total, 524 page views were recorded representing 0.18% of our sample of Facebook users.9 In the control condition, there were a total of 162 page visits, representing 0.16% of Facebook users that have been assigned to the control condition. In the placebo and treatment conditions there were respectively a total of 168 (0.17%) and 194 (0.19%) page views (see also the right panel on Figure 3). When analyzing whether these somewhat-minor differences are of systematic nature, we performed x2 tests. The percentage points difference in page visits between control and treatment condition is 0.03 (x2(1) = 2.30, p = 0.13; Odds Ratio = 1.18), and the difference between placebo and treatment condition is 0.02 (x2(1) = 1.16, p = 0.28; Odds Ratio = 1.12). Between control and placebo conditions, the percentage point difference is even smaller (i.e., 0.01) (x2(1) = 1.87, p = 0.66; Odds Ratio = 1.05). In sum, all three experimental conditions are statistically indistinguishable from each other. In addition, differences are small in magnitude with small effect sizes (i.e., Odds Ratios), confirming prior results from analyzing Facebook’s CTRs. We can therefore confidently reject both, the crowding-out and the crowding-in hypotheses. Conclusion This study has developed and implemented a large-scale but relatively low-cost online experiment in the context of public administration research, where about 300,000 Facebook users were randomly assigned to different donation solicitation ads. The study has found no evidence for crowding-in, nor the crowding-out model. In this sense, our results differ from prior experimental studies that stem from the behavioral laboratory or have been produced in the context of hypothetical vignette designs. While our study is certainly not the first that finds neither evidence for the crowding-out, nor the crowding-in hypotheses (see Blanco et al. 2012; Chan et al. 1996; Wasif and Prakash 2017), we provide—to our knowledge—the first experimental evidence that has been produced within a naturalistic context. However, our results have to be taken with a fair grain of salt. Our effective sample size of 600 clusters may not be able to have detected a difference in groups for our secondary outcome measure. Future studies are advised to replicate our design using a larger number of clusters, possibly by stratifying clusters for single age categories (i.e., one for each year) instead of grouping them as we did. In this sense, our findings do not provide a strict theory test of crowding-in versus crowding-out but provide some suggestive evidence that government funding does not seem to matter in the case of online donation solicitations of local food banks. Future studies may extend this to other areas of human service delivery, including more direct cues of government funding because sometimes people do not pay appropriate attention to informational cues. Although there is a long-standing scholarly debate on crowding-out and crowding-in between government funding and private donations, our study adds to this literature that in an online setting, whether a nonprofit is supported by government funding seems not to affect donors’ behaviors. Therefore, it is possible that, from donors’ perspective, government funding and charitable donations are two independent revenue streams. Donors do not necessarily consider a nonprofit’s revenue structure, especially whether it is supported by government funding when making donation decisions. If so, the crowding-out versus crowding-in debate over the last several decades might be overstated. Certainly, we did not examine in the present study how nonprofits strategically modify their fundraising efforts when they receive government funding, which might actually contribute to the crowding-in or crowding-out effect. For example, Andreoni and Payne (2011) argued that the crowding-out effect is mostly driven by nonprofits’ decreased fundraising efforts as a response to government funding. Again, we cannot guarantee the generalizability of our findings from food banks to other service areas. The primary aim of this study was to showcase how public administration scholars and government agencies can utilize social media to conduct experimental research in the realm of online citizen–government interactions. Social media experiments enable public administration researchers to ensure both internal and external validity because they have the distinct advantage that they combine the internal validity of experiments with an increased realism and ecological validity. Furthermore, as we demonstrate in this study, industry contacts are no longer necessary for implementing large-scale social media experiments. Social media service platforms such as Twitter or Facebook can be easily used to test a variety of important questions to public administration scholars and practitioners. Here, online experiments can be conducted on large-scale samples using a variety of unobtrusive outcome measures to assess respondents’ revealed behaviors. A research agenda for applying social media experiments in studying online citizen–state interactions has implications for a wide range of areas. Possible applications may include assessing the effectiveness of different public communication strategies on social media and how these strategies affect, for example, agency reputation. Indeed, bureaucratic reputation is a core topic area in public administration, with studies examining how news media shape agencies’ reputation, including trust (e.g., Grimmelikhuijsen, de Vries, and Zijlstra 2018). Little is known about how social media communication strategies may alter agency reputation. Social media experiments may assess the effect of different types of communication strategies on reputation—examples may include diffusing responsibility attributions for service failure across actors, active agency branding, or just randomly publishing information about administrative procedures. Another possible area of application may include assessing the effectiveness of different targeted job advertisements for government agencies, using, for instance, Facebook’s advertisement facilities to get underrepresented groups to apply. Recent scholarship has exemplified the importance of signaling personal benefits to increase the number of women and people of color to apply for government jobs (e.g., Linos 2017). Testing and refining such targeted job advertisements and/or messages via social media platforms like Facebook may be a great way to further this promising line of scholarship, but also be an excellent evaluation tool in seeking the most effective online recruitment strategy to increase the demographic representation of government agencies. A more obvious area of application may include assessing whether different ways of how public performance information is presented online affects citizens’ attitudes and subsequent behaviors. Indeed, a growing literature in behavioral public administration examines the psychology of performance information (e.g., James and Olsen 2017), assessing whether different ways of presenting numbers affects perceptions of public service performance or satisfaction evaluations of government agencies. This agenda may be extended to social media experiments where large-scale experiments could be conducted on social media platforms to probe for the external validity of prior survey experiments. Social media experiments may also be useful in simply testing the effectiveness of governmental online crisis warning systems by assessing which types of messages on social media are most likely to be shared across users. Such an agenda would have important practical implications for the effectiveness of early crisis warning systems. The emergence of Internet-based social media has made it possible for researchers to easily interact with thousands or even millions of citizens. Implementing such innovative experimental designs in the context of online citizen–government interactions may be a viable avenue for future experimentation in public administration research both, for academics and practitioners alike. We hope this contribution sparks a wider implementation of public administration experiments on social media platforms. FUNDING This study received funding from the Rutgers SPAA Faculty Research Funds AY16-17. This study was supported by a National Research Foundation of Korea Grant from the Korean Government [NRF-2017S1A3A2065838]. Footnotes 1 Despite a number of important studies on social media use in government agencies and online citizen–government interactions (e.g., Mergel 2012; Grimmelikhuijsen and Meijer 2015; Porumbescu 2015; Im et al. 2014), to date no experimental applications on social media platforms exist in public administration scholarship (for a notable exception see a recent conference contribution by Olsen 2017). 2 Teresi and Michelson (2015) randomized individual Facebook users with whom they connected via a Facebook profile (i.e., becoming “friends”) into experimental conditions. While one group of “friends” received mainly apolitical status updates from the host account, the treatment group received political messages about the upcoming 2010 elections. After the election, authors searched for each “friend” in the state list of registered voters using information provided via Facebook’s profile (i.e., names, age, gender, etc.) to examine whether these online get-out-the-vote messages distributed through social media encouraged subjects to vote. 3 AEARCTR-0002345 (see https://www.socialscienceregistry.org/trials/ 2345). 4 The food banks listed on the websites are all real food banks that operate in NYC, as collected through the National Center for Charitable Statistics (NCCS) database. 5 We distinguished donation-funded and government-funded food banks based on the revenue information provided in food banks’ latest 990 forms in the NCCS database. For this study, we defined government-funded food banks as the ones that receive government funding in addition to private donations, and donation-funded food banks as the ones that do not receive government funding but private donations. 6 988,200 in the control group; 985,300 in the placebo group; 999,000 in the treatment group. 7 This makes a total project cost of $6,000. However, from the purchased $6,000, Facebook ran our ads for a total value of $3,299, or a mean value of $5.50 per cluster. 8 Further information can be found here: https://www.facebook.com/business/help/1619591734742116. 9 There is an inconsistency between the number of unique outbound clicks reported by Facebook and actual unique page visits as provided by Google Analytics. This may be the result of some Facebook clicks not reaching its intended destination with users potentially aborting connecting with a website outside Facebook, possibly because of fear of virus-infected websites. References Ajorlou , A. , A. Jadbabaie , and A. Kakhbod . 2016 . Dynamic pricing in social networks: The word-of-mouth effect . Management Science 64 : 971 – 79 . Google Scholar CrossRef Search ADS Andreoni , J . 1989 . Giving with impure altruism: Applications to charity and Ricardian equivalence . The Journal of Political Economy 97 : 1447 – 58 . Google Scholar CrossRef Search ADS ———. 1993 . An experimental test of the public-goods crowding-out hypothesis . The American Economic Review 83 : 1317 – 27 . Andreoni , J. , and A. A. Payne . 2011 . Is crowding out due entirely to fundraising? Evidence from a panel of charities . Journal of Public Economics 95 : 334 – 43 . Google Scholar CrossRef Search ADS Aral , S . 2016 . Networked experiments . In The Oxford Handbook of the Economics of Networks , ed. Yann Bramoullé , Andrea Galeotti , and Brian Rogers , New York : Oxford University Press . Aral , S. , and D. Walker . 2014 . Tie strength, embeddedness, and social influence: A large-scale networked experiment . Management Science 60 : 1352 – 70 . Google Scholar CrossRef Search ADS Bakshy , E. , D. Eckles , R. Yan , and I. Rosenn . 2012a . Social influence in social advertising: evidence from field experiments . arXiv:1206.4327v1. Bakshy , E. , I. Rosenn , C. Marlow , and L. Adamic . 2012b . The role of netwroks in information diffusion . arXiv:1207.4145v5. Blanco , E. , M. C. Lopez , and E. A. Coleman . 2012 . Voting for environmental donations: Experimental evidence from Majorca, Spain . Ecological Economics 75 : 52 – 60 . Google Scholar CrossRef Search ADS Blom-Hansen , Jens , Rebecca Morton , and Soren Serritzlew . 2015 . Experiments in public management research . International Public Management Journal 18 : 151 – 70 . Google Scholar CrossRef Search ADS Bolton , G. E. , and E. Katok . 1998 . An experimental test of the crowding out hypothesis: The nature of beneficent behavior . Journal of Economic Behavior & Organization 37 ( 3 ): 315 – 31 . Google Scholar CrossRef Search ADS Bond , R. M. , C. Fariss , J. Jones , A. Kramer , C. Marlow , J. Settle , and J. Fowler . 2012 . A 61-million-person experiment in social influence and political mobilization . Nature 489 : 295 – 98 . Google Scholar CrossRef Search ADS Borgonovi , F. , and M. O’Hare . 2004 . The impact of the national endowment for the arts in the United States: Institutional and sectoral effects on private funding . Journal of Cultural Economics 28 : 21 – 36 . Google Scholar CrossRef Search ADS Broockman , David and Donald Green . 2014 . Do online advertisements increase political candidates’ name recognition or favorability? Evidence from randomized field experiments . Political Behavior 36 : 263 – 89 . Google Scholar CrossRef Search ADS Brooks , A. C . 1999 . Do public subsidies leverage private philanthropy for the arts? Empirical evidence on symphony orchestras . Nonprofit and Voluntary Sector Quarterly 28 : 32 – 45 . Google Scholar CrossRef Search ADS ———. 2000 . Is there a dark side to government support for nonprofits ? Public Administration Review 60 : 211 – 18 . CrossRef Search ADS Campbell , Michael and Stephen Walters . 2014 . How to design, analyse and report cluster randomised trials in medicine and health related research . Hoboken : Wiley . Google Scholar CrossRef Search ADS Chan , K. S. , S. Mestelman , R. Moir , and R. A. Muller . 1996 . The voluntary provision of public goods under varying income distributions . Canadian Journal of Economics 29 ( 1 ): 54 – 69 . Google Scholar CrossRef Search ADS Chan , K. S. , R. Godby , S. Mestelman , and R. A. Muller . 2002 . Crowding-out voluntary contributions to public goods . Journal of Economic Behavior & Organization 48 ( 3 ): 305 – 17 . Google Scholar CrossRef Search ADS Coppock , A. , A. Guess and J. Ternovski . 2016 . When treatments are tweets: A network mobilization experiemnt over twitter . Political Behavior 38 : 105 – 2018 . Google Scholar CrossRef Search ADS De Wit , A. , and R. Bekkers . 2016 . Government support and charitable donations: A meta-analysis of the crowding-out hypothesis . Journal of Public Administration Research and Theory 27 : 301 – 19 . De Wit , A. , R. Bekkers , and M. Broese van Groenou . 2017 . Heterogeneity in crowding-out: When are charitable donations responsive to government support ? European Sociological Review 33 : 59 – 71 . Dokko , J. K . 2009 . Does the NEA crowd out private charitable contributions to the arts ? National Tax Journal 62 : 57 – 75 . Google Scholar CrossRef Search ADS Eckel , C. C. , P. J. Grossman , and R. M. Johnston 2005 . An experimental test of the crowding out hypothesis . Journal of Public Economics 89 : 1543 – 60 . Google Scholar CrossRef Search ADS Galbiati , R. , and P. Vertova . 2008 . Obligations and cooperative behaviour in public good games . Games and Economic Behavior 64 ( 1 ): 146 – 70 . Google Scholar CrossRef Search ADS Galbiati , R. , and P. Vertova . 2014 . How laws affect behavior: Obligations, incentives and cooperative behavior . International Review of Law and Economics 38 : 48 – 57 . Google Scholar CrossRef Search ADS Galizzi , Mateo and Daniel Navarro-Martinez . Forthcoming . On the external validity of social preference games: A systematic lab-field study . Management Science . Gerber , Alan and Donald Green . 2012 . Field experiments: design, analysis, and interpretation . New York : Norton . Gong , S. , J. Zhang , P. Zhao , and X. Jiang . 2017 . Tweeting as a marketing tool: A field experiment in the TV industry . Journal of Marketing Research 54 : 833 – 50 . Google Scholar CrossRef Search ADS Grimmelikhuijsen , Stephan and Albert Meijer . 2015 . Does twitter increase perceived police legitimacy ? Public Administration Review 75 : 598 – 607 . Google Scholar CrossRef Search ADS Grimmelikhuijsen , S. , F. de Vries , and W. Zijlstra . 2018 . Breaking bad news without breaking trust: The effects of a press release and newspaper coverage on perceived trustworthiness . Journal of Behavioral Public Administration 1 . Gronberg , T. J. , R. A. Luccasen III , T. L. Turocy , and J. B. Van Huyck . 2012 . Are tax-financed contributions to a public good completely crowded-out? Experimental evidence . Journal of Public Economics 96 ( 7–8 ): 596 – 603 . Google Scholar CrossRef Search ADS Güth , W. , M. Sutter , and H. Verbon . 2006 . Voluntary versus compulsory solidarity: Theory and experiment . Journal of Institutional and Theoretical Economics 162 ( 2 ): 347 – 63 . Google Scholar CrossRef Search ADS Hayes , Richard and Lawrence Moulton . 2017 . Cluster randomised trials . Boca Raton : Chapman & Hall/CRC . Heutel , G . 2014 . Crowding out and crowding in of private donations and government grants . Public Finance Review 42 : 143 – 75 . Google Scholar CrossRef Search ADS Horne , C. S. , J. L. Johnson , and D. M. Van Slyke . 2005 . Do charitable donors know enough—and care enough—About government subsidies to affect private giving to nonprofit organizations ? Nonprofit and Voluntary Sector Quarterly 34 : 136 – 49 . Google Scholar CrossRef Search ADS Hsu , L. C . 2008 . Experimental evidence on tax compliance and voluntary public good provision . National Tax Journal 61 ( 2 ): 205 – 23 . Google Scholar CrossRef Search ADS Hughes , P. , W. Luksetich , and P. Rooney . 2014 . Crowding‐Out and fundraising efforts . Nonprofit Management and Leadership 24 ( 4 ): 445 – 64 . Im , T. , W. Cho , G. Porumbescu , and J. Park . 2014 . Internet, trust in government, and citizen compliance . Journal of Public Administration Research and Theory 24 ( 3 ): 741 – 63 . Google Scholar CrossRef Search ADS Isaac , R. M. , and D. A. Norton . 2013 . Endogenous institutions and the possibility of reverse crowding out . Public Choice 156 : 253 – 84 . Google Scholar CrossRef Search ADS James , Oliver , Sebastian Jilke , and Gregg Van Ryzin , eds. 2017a . Experiments in public management research: challenges and opportunities . Cambridge : Cambridge University Press . Google Scholar CrossRef Search ADS James , Oliver , Sebastian Jilke , and Gregg Van Ryzin . 2017b . Causal inference and the design and analysis of experiments . In Experiments in public management research: Challenges and opportunities , ed. Oliver James , Sebastian Jilke , and Gregg Van Ryzin , 59 – 88 . Cambridge : Cambridge University Press . Google Scholar CrossRef Search ADS James , Oliver , and Asmus Leth Olsen . 2017 . Citizens and public performance measures: making sense of performance information . In Experiments in public management research: Challenges and opportunities , ed. Oliver James , Sebastian Jilke , and Gregg Van Ryzin , 270 – 89 . Cambridge : Cambridge University Press . Google Scholar CrossRef Search ADS Khanna , J. , and T. Sandler . 2000 . Partners in giving: The crowding-in effects of UK government grants . European Economic Review 44 : 1543 – 56 . Google Scholar CrossRef Search ADS Kim , Mirae , and Gregg G. Van Ryzin . 2014 . Impact of government funding on donations to arts organizations: A survey experiment . Nonprofit and Voluntary Sector Quarterly 43 : 910 – 25 . Google Scholar CrossRef Search ADS King , G. , J. Pan , and M. E. Roberts . 2013 . How censorship in China allows government criticism but silences collective expression . American Political Science Review 107 ( 2 ): 1 – 18 . Google Scholar CrossRef Search ADS King , G. , B. Schneer , and A. White . 2017 . How the news media activate public expression and influence national agendas . Science 358 : 776 – 80 . Google Scholar CrossRef Search ADS Kingma , B. R . 1989 . An accurate measurement of the crowd-out effect, income effect, and price effect for charitable contributions . Journal of Political Economy 97 : 1197 – 207 . Google Scholar CrossRef Search ADS Kobayashi , T. and Y. Ichifuji . 2015 . Tweets that matter: Evidence from a randomized field experiment in Japan . Political Communication 32 : 574 – 93 . Google Scholar CrossRef Search ADS Konow , J . 2010 . Mixed feelings: Theories of and evidence on giving . Journal of Public Economics 94 (3–4): 279 – 97 . Google Scholar CrossRef Search ADS Korenok , O. , E. L. Millner , and L. Razzolini . 2012 . Are dictators averse to inequality ? Journal of Economic Behavior & Organization 82 : 543 – 47 . Google Scholar CrossRef Search ADS Kramer , A. , J. Guillory and J. Hancock . 2014 . Experimental evidence of massice-scale emotional contagion through social networks . Proceedings of the National Academy of Science of the United States of America 111 : 8788 – 90 . Google Scholar CrossRef Search ADS Krasteva , S. , and H. Yildirim . 2013 . (Un) informed charitable giving . Journal of Public Economics 106 : 14 – 26 . Google Scholar CrossRef Search ADS Ledyard , J . 1995 . Public goods: A survey of experimental research . In Handbook of experimental economics , ed. J. Kagel and A. Roth , 111 – 94 . Princeton : Princeton University Press . Levitt , Steven and John List . 2007 . What do laboratory experiments measuring social preferences reveal about the real world ? Journal of Economic Perspectives 21 : 153 – 74 . Google Scholar CrossRef Search ADS Li , H. and G. Van Ryzin . 2017 . A systematic review of experimental studies in public management journals . In Experiments in public management research: challenges and opportunities , ed. James , O. , S. Jilke , and G. Van Ryzin . Cambridge : Cambridge University Press . Google Scholar CrossRef Search ADS Lilley , A. , and R. Slonim . 2014 . The price of warm glow . Journal of Public Economics 114 : 58 – 74 . Google Scholar CrossRef Search ADS Linos , E . 2017 . More than public service: A field experiment on job advertisements and diversity in the police . Journal of Public Administration Research and Theory 28 ( 1 ): 67 – 85 . Google Scholar CrossRef Search ADS Lu , J . 2015 . Which nonprofit gets more government funding ? Nonprofit Management and Leadership 25 : 297 – 312 . Google Scholar CrossRef Search ADS ———. 2016 . The Philanthropic consequence of government grants to nonprofit organizations . Nonprofit Management and Leadership 26 : 381 – 400 . CrossRef Search ADS Mergel , I . 2012 . Social media in the public sector: A guide to participation, collaboration and transparency in the networked world . Hoboken : Wiley: Jossey-Bass . Middleton , Joal and Peter Arronow . 2015 . Unbiased estimation of the average treatment effect in cluster-randomized experiments . Statistics, Politics and Policy 6 : 39 – 75 . Google Scholar CrossRef Search ADS Munger , K . 2017 . Tweetment effects on the tweeted: experimentally reducing racist harassment . Political Behavior 39 ( 3 ): 629 – 49 . Google Scholar CrossRef Search ADS Okten , C. , and B. A. Weisbrod . 2000 . Determinants of donations in private nonprofit markets . Journal of Public Economics 75 : 255 – 72 . Google Scholar CrossRef Search ADS Olsen , A . 2017 . Adventures into the negativity bias . Paper presented at EGPA conference in Milan , Italy . Ottoni-Wilhelm , M. , L. Vesterlund , and H. Xie . 2017 . Why do people give? Testing pure and impure altruism . American Economic Review 107 : 3617 – 33 . Google Scholar CrossRef Search ADS Porumbescu , Gregory . 2015 . Comparing the effects of E-Government and social media use on trust in government: Evidence from Seoul, South Korea . Public Management Review 18 : 1308 – 34 . Google Scholar CrossRef Search ADS Public Religion Research Institute . 2012 . Few Americans use social media to connect with their faith communities . http://publicreligion.org/research/2012/08/july-2012-tracking-survey/ (accessed April 30, 2018) . Reeson , A. F. , and J. G. Tisdell . 2008 . Institutions, motivations and public goods: An experimental test of motivational crowding . Journal of Economic Behavior & Organization 68 ( 1 ): 273 – 81 . Google Scholar CrossRef Search ADS Roberts , R. D . 1984 . A positive model of private charity and public transfers . The Journal of Political Economy 92 : 136 – 48 . Google Scholar CrossRef Search ADS Rose-Ackerman , S . 1981 . Do government grants to charity reduce private donations ? In Nonprofit firms in a three sector economy , ed. White M. J ., 95 – 114 . Washington, DC : Urban Institute . Ryan , T . 2012 . What makes us click? Demonstrating incentives for angry discourse with digital-age field experiments . The Journal of Politics 74 : 1138 – 52 . Google Scholar CrossRef Search ADS Ryan , T. J. , and Brader , T . 2017 . Gaffe appeal a field experiment on partisan selective exposure to election messages . Political Science Research and Methods 5 : 667 – 87 . Google Scholar CrossRef Search ADS Ryan , T. and D. Broockman . 2012 . Facebook: A new frontier for field experiments . The Experimental Political Scientist: Newsletter of the ASPA Experimental Section 3 : 2 – 8 . Smith , T. M . 2007 . The impact of government funding on private contributions to nonprofit performing arts organizations . Annals of Public and Cooperative Economics 78 : 137 – 60 . Google Scholar CrossRef Search ADS Statistia . 2017 . Number of Facebook users by age in the U.S. as of January 2017 . https://www.statista.com/statistics/398136/us-facebook-user-age-groups/ (accessed April 30, 2018) . Steinberg , R . 1987 . Voluntary donations and public expenditures in a federalist system . The American Economic Review 77 : 24 – 36 . Sutter , M. , & H. Weck-Hannemann . 2004 . An experimental test of the public goods crowding out hypothesis when taxation is endogenous . FinanzArchiv: Public Finance Analysis 60 ( 1 ): 94 – 110 . Google Scholar CrossRef Search ADS Suárez , D. F . 2011 . Collaboration and professionalization: The contours of public sector funding for nonprofit organizations . Journal of Public Administration Research and Theory 21 : 307 – 26 . Google Scholar CrossRef Search ADS Teresi , H. and M. Michelson . 2015 . Wired to mobilize: The effect of social networking messages on voter turnout . The Social Science Journal 52 : 195 – 204 Google Scholar CrossRef Search ADS Thomas , J. C. , and G. Streib . 2003 . The new face of government: Citizen-initiated contacts in the era of E-Government . Journal of public administration research and theory 13 : 83 – 102 . Google Scholar CrossRef Search ADS Tinkelman , D . 2010 . Revenue interactions: crowding out, crowding in, or neither ? In Handbook of research on nonprofit economics and management , ed. Seaman , B. A. , and D. R. Young , pp. 18 – 41 . Northampton, MA : Edward Elgar . Google Scholar CrossRef Search ADS Wang , J. C. , and C. H. Chang . 2013 . How online social ties and product-related risks influence purchase intentions: A Facebook experiment . Electronic Commerce Research and Applications 12 : 337 – 46 . Google Scholar CrossRef Search ADS Warr , P. G . 1982 . Pareto optimal redistribution and private charity . Journal of Public Economics 19 : 131 – 38 . Google Scholar CrossRef Search ADS Wasif , R. , and A. Prakash . 2017 . Do government and foreign funding influence individual donations to religious nonprofits? A survey experiment in Pakistan . Nonprofit Policy Forum 8 ( 3 ): 237 – 73 . Google Scholar CrossRef Search ADS Wukich , C. , and I. Mergel . 2015 . Closing the citizen-government communication gap: Content, audience, and network analysis of government tweets . Journal of Homeland Security and Emergency Management 12 : 707 – 35 . Google Scholar CrossRef Search ADS Zhang , J. , and E. Mao . 2016 . From online motivations to ad clicks and to behavioral intentions: An empirical study of consumer response to social media advertising . Psychology & Marketing 33 : 155 – 64 . Google Scholar CrossRef Search ADS Appendix Table A1. Mean difference tests CTR (independent two-sample t-test; n = 600) Control Placebo Treatment Mean Standard error Mean Standard error Mean Standard error Mean difference; control vs. treatment p value Mean difference; placebo versus treatment p value Mean difference; control versus placebo p value CTR 0.50 0.03 0.44 0.03 0.50 0.04 0.00 0.99 0.06 0.21 −0.06 0.17 Control Placebo Treatment Mean Standard error Mean Standard error Mean Standard error Mean difference; control vs. treatment p value Mean difference; placebo versus treatment p value Mean difference; control versus placebo p value CTR 0.50 0.03 0.44 0.03 0.50 0.04 0.00 0.99 0.06 0.21 −0.06 0.17 View Large Table A1. Mean difference tests CTR (independent two-sample t-test; n = 600) Control Placebo Treatment Mean Standard error Mean Standard error Mean Standard error Mean difference; control vs. treatment p value Mean difference; placebo versus treatment p value Mean difference; control versus placebo p value CTR 0.50 0.03 0.44 0.03 0.50 0.04 0.00 0.99 0.06 0.21 −0.06 0.17 Control Placebo Treatment Mean Standard error Mean Standard error Mean Standard error Mean difference; control vs. treatment p value Mean difference; placebo versus treatment p value Mean difference; control versus placebo p value CTR 0.50 0.03 0.44 0.03 0.50 0.04 0.00 0.99 0.06 0.21 −0.06 0.17 View Large © The Author(s) 2018. Published by Oxford University Press on behalf of the Public Management Research Association. All rights reserved. For permissions, please e-mail: firstname.lastname@example.org. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)
Journal of Public Administration Research and Theory – Oxford University Press
Published: May 12, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera