TY - JOUR
AU - Nair, Harikesh, S
AB - Abstract We develop a field experiment that assesses whether advertising can serve as a signal that enhances consumers’ evaluations of advertised goods. We implement the experiment on a mobile search platform that provides listings and reviews for an archetypal experience good, restaurants. In collaboration with the platform, we randomize about 200,000 users in 13 Asian cities into exposure of ads for about 600+ local restaurants. Within the exposure group, we randomly vary the disclosure to the consumer of whether a restaurant’s listing is a paid-ad. This enables isolating the effect on outcomes of a user knowing that a listing is sponsored—a pure signalling effect. We find that this disclosure increases calls to the restaurant by 77%, holding fixed all other attributes of the ad. The disclosure effect is higher when the consumer uses the platform away from his typical city of search, when the uncertainty about restaurant quality is larger, and for restaurants that have received fewer ratings in the past. On the supply side, newer, higher rated and more popular restaurants are found to advertise more on the platform; and ratings of those that advertised during the experiment are found to be higher two years later. Taken together, we interpret these results as consistent with a signalling equilibrium in which ads serve as implicit signals that enhance the appeal of the advertised restaurants to consumers. Both consumers and advertisers seem to benefit from the signalling. Consumers shift choices towards restaurants that are better rated (at baseline) in the disclosure group compared to the no disclosure group, and advertisers gain from the improved outcomes induced by disclosure. 1. Introduction Despite its prominent influence on how social scientists think about the role of advertising, Nelson ’s (1970, 1974) celebrated idea that advertising can serve as a signal of product quality has proven difficult to test empirically. Consequently, more than 40 years since it was originally articulated, credible empirical evidence in favor of the signalling view of advertising has remained rare. Understanding whether advertising actually plays a signalling role and how this role materializes is important to assess the welfare consequences of advertising: if advertising can serve as a signal, it can improve the efficiency of markets with search frictions by helping buyers and sellers communicate. It also has implications for firms targeting their ads. If ads convey demand-enhancing information about products beyond informing users of their existence and product attributes, ads could be targeted to users already aware of the product, when there is considerable uncertainty about the products’ quality. This article describes a field experiment implemented in collaboration with a large restaurant search platform that enables a test of the “signalling hypothesis”. We find results consistent with signalling. Nelson’s work postulates an indirectly informative view of advertising, suggesting that one role played by advertising is to signal to market participants that the advertising firm is of high quality. Nelson’s suggestion is formalized in several canonical models that followed (e.g.Kihlstrom and Riordan 1984; Milgrom and Roberts 1986). In these models, consumers are ex ante uncertain about the quality of a good, which is revealed to them upon consumption. Firms with higher quality benefit more from advertising, for example, when high quality firms obtain more repeat purchase after consumption. A separating equilibrium is achieved when (1) the gain from the repeat purchase relative to the cost of advertising is higher for high quality firms at optimally chosen prices and (2) lower quality firms do not gain from mimicking the strategies of the high quality firms. In the equilibrium, the act of advertising itself conveys information about quality. Because direct claims about quality in an ad cannot be verified prior to purchase of an experience good, the indirect way by which advertising reveals quality is relevant. Therefore, costly advertising serves as a credible signal. Subsequently, several empirical studies have built on Nelson’s ideas and attempted to test the signalling hypothesis. One group of studies on the “supply-side” have investigated the predicted equilibrium associations amongst the key components of the model—advertising, prices, and quality. Another set of studies on the “demand-side” have used micro-data on consumer exposure to advertising to explore patterns suggestive of signalling. Researchers using either strategy have faced challenges in establishing a signalling role for advertising. A supply-side empirical test of the theory requires (a) a measure of quality that is being signaled by the firm and (b) a way to match its association with firms’ observed behaviour in a manner that is falsifiable by the theory. Both these steps are difficult. The main challenge in step (a) is in obtaining a measure of quality as conceptualized in the theory, which is a construct that is observed to the firm but is unobserved to the consumer. This is non-trivial when the consumers’ actual information sets are unobserved. Even if a researcher obtains a measure of quality unavailable to consumers, such as a performance metric in a technical report, it is hard to rule out that the measure is uncorrelated with some component of consumers’ unobserved information-sets. By implication, any covariation between such a quality metric and advertising could reflect the mediation of such omitted variables. Evidence for correlations between observed metrics of quality—like peer ratings, consumer reports or time spent at the firm—with advertising actions, thus present a weaker test of the theory (e.g. see Archibald et al. 1983; Kwoka 1984; Caves and Greene 1996; Thomas et al. 1998; Kirmani and Rao 2000; Horstmann and MacDonald 2003; Horstmann and MacDonald 2003 for further discussion). The challenge in step (b) is that observed patterns of firm-level advertising can often be explained by reasons other than those postulated by signalling. For instance, the canonical model predicts that ad-intensity starts high for new goods and falls for established goods as information about unobserved quality diffuses in the market. A subset of studies focus on whether observed data are consistent with these life-cycle predictions (e.g.Tellis and Fornell 1988, Horstmann and MacDonald 2003). However, such patterns could also be produced by changing consumers’ awareness about the product, changes in competitive intensity due to entry, changes over time in the costs of advertising in media-markets, all of which have to be ruled out to establish the empirical relevance of the signalling mechanism. Testing whether advertising signals quality on the “demand-side” by directly exploring consumer-level response to ads is also difficult. The main challenge is to disentangle the signalling effect from other effects of advertising. This concern is severe in data where all relevant aspects of the ad-message are not observed. Hence, it is usually impossible to say whether consumer response to advertising is due some aspect of the ad’s message that is unobserved by the econometrician, or due to the ad reminding the consumer of the product; or due to the consumer’s knowledge that the firm has advertised. Only the last is a pure signalling effect. A separate concern is the endogeneity of advertising exposure due to the targeting of advertising by firms or from user self selection into viewing ads. A final difficulty with consumer-level analysis involves issues of statistical power arising from the large noise-to-signal ratio of ad-effects at the individual level, requiring large datasets that are difficult to collect.1 We design a field-experiment that attempts to address these difficulties in a more direct way compared to the previous literature. Our experiment is implemented in collaboration with Zomato, a worldwide restaurant-search portal. Zomato provides an online platform for consumers to search and browse through information on restaurants in local markets. The experiment is implemented on the platform’s mobile app and introduces search advertising on Zomato’s mobile platform. The experiment randomizes users into treatment groups in which they see search ads for local restaurants. The groups are similar on all dimensions except the manner in which advertising is disclosed. Users in the first see the advertiser’s listing without any disclosure, whereas users in the second see the listing with an indication disclosing that the listing is an ad. Therefore, users in both groups are exposed to the same set of ads for the same restaurants with and without disclosure that the ads are paid for by the advertiser. In the data, we observe the user-level browsing behaviour, and the restaurants they call.2 This design enables assessing the effect of a consumer’s knowledge that the firm is advertising, separately from the ads’ effect on the awareness of the existence of the firm, and from the effect of the content of the ads. Thus, it facilitates a demand-side test of the signalling effect. The empirical setting has advantages as a field laboratory to assess signalling. First, signalling is most relevant in markets for experience-goods, in which consumers are information-constrained, quality sensitive, have uncertainty about the product prior to consumption, and show repeat purchase for firms revealed to have high quality from their visitation. Restaurants are examples of such goods. Second, search ads on restaurant search platforms are matched to user intent and served in response to users who are searching for information about the goods being advertised, which reduces the chance the ads are annoying and will be skipped. This makes detection of the signalling effect more likely. Third, the mobile application environment has the advantage that one obtains a persistent user identifier defined within a closed system that logs engagement with the ads as well as demonstrated interest in the product. This makes randomization and behaviour-tracking at the individual user-level possible. Finally, a large number of users (around 200,000) and advertisers (around 600) enables exploring differential patterns of response along dimensions of consumer- and restaurant-heterogeneity that facilitate additional tests of the signalling theory, and serve as consistency checks on the effect. The experimental design leverages aspects of the institutional environment. Firstly, the Zomato platform had an active advertising market on its website at the time of the experiment, but no advertising on the mobile app prior to it. Therefore, our experiment estimates the signalling effect based on the initial beliefs the consumers hold from their experience on the website. Since the mobile app had no advertising prior to the experiment, users assigned to the no-disclosure group continue using the app as though there is no advertising on the app. Second, for each search query originating in the mobile app during the experiment, the experiment retrieves the restaurants that have contracted to show their ad for the same search query on the Zomatowebsite, and shows their ads to users. By ensuring that ads shown are for restaurants that choose to advertise on the platform, we estimate effects for a relevant set of advertising restaurants, whose identities are consistent with consumer beliefs about advertisers on Zomato. Third, the experimental design incorporates features to minimize concerns that treatments induce “Hawthorne effects” or “randomization bias” that confound the signalling effect. In particular, the treatment of showing an advertised listing without disclosing to consumers that it is paid for, is harder to conceptualize in other media, such as TV. The presence of such an ad would seem odd, and may evoke a corresponding consumer response. This treatment is more natural as part of an experiment on a search platform, where ads without disclosure look exactly like “organic” listings. Further, we implement our experiment in a manner such that the ads introduced by our experiments are consistent with the user’s expectations from search. More details on this are discussed later. Our main results report the effect of disclosure of users’ first ad impressions on their behaviour in the session.3 We find that disclosing to a consumer that a listing on the platform’s search results is an ad increases calls to the restaurant by 77% relative to no disclosure. This represents the causal effect of disclosure, because it keeps all aspects of the ad, including its content and position on the listings page fixed, and holds consumer type fixed in the comparison on account of the randomization. This effect is also comparable to the impact of other informational attributes. An average advertiser in our data obtains similar benefit from ad disclosure, as a two decile increase in the number of its ratings on the platform. We find that the likelihood of calling the advertiser conditional on visiting its page is 76% higher among users who see the ad disclosure, relative to users who see the listing without ad disclosure. Further, we find that exposure to the listing mainly drives visits to the restaurant’s page, whereas disclosing that the listing is an ad helps convert the page-visits into calls to the restaurant. These results suggest that the ad disclosure implicitly conveys a cue to exposed users, which increases the restaurant’s appeal amongst them, and improves their evaluation of the advertised product. Information from the search platform cannot completely eliminate consumer’s uncertainty about product quality. Advertising serves as a signal of the advertiser’s unobserved “type”, and reduces uncertainty. There would be no role of signalling in a world where the search platform perfectly matches consumers to restaurants. Therefore, our results also imply that in our empirical setting the Zomato platform does not perfectly match consumers to restaurants.4 A stylized theoretical model (discussed in Section 5) explains precisely our interpretation of these results. Analysing the data further, we test whether the effect of the disclosure is higher for subpopulations of users that have more uncertainty about the restaurants they are searching for, as the signalling hypothesis would suggest. We find that users searching in a city different from where they usually search, visit the listed restaurant’s pages at a significantly higher rate if it is placed with ad disclosure. We also see that restaurants that have been rated fewer times on the platform—presumably, those about which consumers a priori have more uncertainty—benefit more from the ad-disclosure. These effects are consistent with predictions from signalling theory. Exploring whether consumers are made better or worse off under disclosure, we find that consumers’ choices shift systematically towards restaurants that are better rated (at baseline) in the disclosure group compared to the no disclosure group. Finally, exploring patterns of covariation on the supply-side, we find that restaurants with higher appeal to consumers (with better ratings), with higher prices and which are newer and not part of a restaurant-chain (presumably ones that consumers have more uncertainty about) are more likely to advertise on Zomato. We also find that restaurants that chose to advertise on Zomato during the experiment are likely to receive higher ratings in the next two years, compared to those that did not advertise. This is consistent with these restaurants having higher unobserved quality that is revealed over time through actual consumer experiences. These supply-side patterns, similar to those reported in some of the past literature, are broadly consistent with a signalling equilibrium the demand-side results suggest. This article is related to an empirical literature on digital advertising (e.g.Manchanda et al. 2006; Yang and Ghose 2010; Chan et al. 2011; Yao and Mela 2011; Rutz and Bucklin 2011; Johnson 2013; Lewis and Reiley 2014), and more pointedly, to a burgeoning literature that leverages experimental or quasi-experimental variation on search platforms to address issues related to search advertising (e.g.Goldfarb and Tucker 2011; Nosko et al., 2015; Sahni 2015; Narayanan and Kalyanam 2015; Ursu 2018). To our knowledge though, none have focused explicitly on the role of signalling or disclosure. Outside of advertising, our analysis is closest to the empirical literature in education that has tested for the signalling role of education (e.g.Tyler et al. 2000 in particular for a quasi-experiment; and Weiss 1995 for a survey of empirical work). The difficulties in that literature in distinguishing between human capital versus signalling (“sheepskin”) explanations for education have parallels to the difficulties here in distinguishing between direct and indirect informative effects of advertising. This study is also related to recent empirical papers that have investigated signalling via round-numbered asking prices by sellers on eBay.com (Backus et al. 2019) and via posted interest rates by loan seekers on Prosper.com (Zhang and Liu 2012; Kawai et al. 2014). Finally, both this paper and a companion paper (Sahni and Nair 2018) use data from a broad experiment we implemented to study two distinct issues related to advertising. Sahni and Nair (2018) are focused on assessing an experimental design to test whether sponsorship disclosure in “native” advertising materially deceives consumers as defined under U.S. law; while this paper’s focus is on using the experimental data (as well as other supply-side information) to test signalling theories of advertising. This paper is in the domain of assessing the economic mechanisms by which advertising works; while Sahni and Nair (2018) is in the domain of assessing legal aspects of digital advertising. The key effects in that paper are pinned down by the contrast between different formats of disclosure within the ad disclosure group, which is not the focus here. Our results obtained here show the consistency of the results obtained therein with the use of advertising as a quality signal. The reader should note that the design and data have two shortcomings. First, the design might understate the size of the signalling effect. If the search engine’s algorithm is working well, a searching consumer will believe that organic listings are a good match for his taste. Hence, what is measured as the difference in outcomes between an ad with disclosure versus without, is the sum total of a (potentially negative) “not-organic” effect and a (positive signalling-derived) “is-ad” effect. The not-organic effect reduces the estimated effect. So our test of signalling is conservative. Second, our main outcome variable—calls to the restaurant—represents only a proxy for restaurant demand. We do not have access to actual demand/expenditures. We present later in the paper supporting data suggesting that this is a reasonable proxy in the experimental markets we consider; nevertheless, in the absence of actual demand data, this remains a limitation. The rest of the article is outlined as follows. The next section briefly describes our empirical strategy. The following section describes the Zomato platform and details of the field experiment. The sections after describe market level advertising patterns, the stylized theoretical model, the main results from the experiment, assessment of heterogeneity, robustness, and aggregate effects. The last section concludes. 2. Empirical Strategy To understand our empirical strategy, consider the behaviour of a consumer |$i$| who is contemplating buying from a seller of an archetypical experience good, such as a restaurant. Let |$r$| index the seller, |$q_{r}$| the seller’s appeal to the consumer and |$b_{i}\left(q_{r}\right)$| represent the consumer’s ex ante belief about |$q_{r}$|⁠. For the purpose of this discussion, assume that |$i$| is aware of the existence of |$r$|⁠. Prior to a possible purchase, suppose the consumer is exposed to information |$\mathcal{I}\left(x_{r},a\right)$| about the seller. |$\mathcal{I}\left(.\right)$| includes a vector of product attributes denoted by |$x_{r}$| which we will refer to as “content,” that informs the consumer about the seller’s appeal. In the context of a search platform, |$x_{r}$| includes attributes like the seller’s average rating, and reviews by other consumers. |$\mathcal{I}\left(.\right)$| is also indexed by a binary variable |$a$| that denotes whether the information is part of an advertisement or not. When |$a=1$|⁠, the consumer realizes that that information is paid for by the vendor (i.e. “paid listing”); when |$a=0$|⁠, the information is presented by the platform (i.e. “organic listing”). On receiving the information, the consumer’s prior beliefs are updated to a posterior |$b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a\right)\right)$|⁠. The observed action (⁠|$y$|⁠) representing the behaviour of the consumer towards the seller are a function of his beliefs and are denoted |$y_{i}\left[b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a\right)\right)\right]$|⁠. The observed action of a consumer who is exposed to no information is denoted |$y_{i}\left[b_{i}\left(q_{r}\right)\right]$|⁠. Our empirical strategy measures the causal effect of |$a$| on |$y$|⁠. As noted, empirical researchers face two main difficulties in assessing the signalling effect of advertising from field data on the demand side. A first fundamental difficulty is that information in an ad arrives as a bundle of |$x_{r}$| and |$\left(a=1\right)$|⁠, which makes it difficult to separate the effect of the content of the ad from the fact that the seller is advertising. In terms of our notation, a researcher may compare the behaviour of the same individual |$i$| in situations when he is exposed to the bundle versus not, $$\begin{equation} \triangle_{1i}^{\left(r\right)}=y_{i}\left[b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a_{i}=1\right)\right)\right]-y_{i}\left[b_{i}\left(q_{r}\right)\right]. \end{equation}$$ (2.1) However, |$\triangle_{1i}^{\left(r\right)}$| is not the right treatment effect because it does not hold content |$x_{r}$| constant in comparing the situations with and without exposure to information. The second difficulty is related to self-selection: generally, the set of consumers who get exposed to advertising is different from the set of consumers who do not, because ads are targeted, and typically get displayed to users who are more likely to generate the desired action. A researcher might compare the behaviour of consumers who are exposed to the information (indexed by |$i)$| to those who are not (indexed by |$k)$|⁠, $$\begin{equation} \triangle_{2ik}^{\left(r\right)}=y_{i}\left[b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a_{i}=1\right)\right)\right]-y_{k}\left[b_{k}\left(q_{r}\right)\right]. \end{equation}$$ (2.2) In comparison |$\triangle_{2ik}^{\left(r\right)}$| we face the same issue as in |$\Delta_{1i}^{\left(r\right)}$|⁠, plus that fact that the treatment |$a_{i}$| is correlated with |$y_{i}$| due to selection into exposure. It is clear that both do not deliver the valid causal effect of |$a_{i}$| on |$y_{i}$|⁠. 2.1. The empirical strategy in this article We aim to compare the behaviour of an individual exposed to the same content either as part of a paid advertisement by the seller or not. That is, we aim to construct the comparison, $$\triangle_{i}^{\left(r\right)}=y_{i}\left[b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a_{i}=1\right)\right)\right]-y_{i}\left[b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a_{i}=0\right)\right)\right]$$ representing the causal effect on buying behaviour of the consumers’ knowledge that the content is paid for by the seller. This contrast is implemented on the Zomato platform by randomizing across users the same content with and without revealing the content is paid for by the advertising restaurant. By randomizing |$a$| across users, we are able to estimate an average treatment effect of the ad-disclosure across all participating advertisers on the platform, $$\triangle=\mathbb{E}_{r}\triangle^{\left(r\right)}=\mathbb{E}_{r}\left[\mathbb{E}_{i}y_{i}\left\{ b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a=1\right)\right)\right\} -\mathbb{E}_{i}y_{i}\left\{ b_{i}\left(q_{r}|\mathcal{I}\left(x_{r},a=0\right)\right)\right\} \right].$$ In Section 5 below, we present a stylized model that relates this comparison to signalling theory in a consumer-search setting like ours, and explains our interpretation. Based on the predictions of this theory, we analyse heterogeneity in these treatment effects. In the next two sections, we describe the empirical setting, the field-experiment, and the control and treatment groups in more detail. 3. Application Setting and Field Experiment 3.1. Zomato.com Pursuant to the acquisition of urbanspoon.com in 2015, the Zomato platform hosts searchable listings on about 1.4 million restaurants in 22 countries, counting approximately 90 million visits each month across its website and mobile applications. As comparison, Yelp, the market leader for online listings of local businesses (not just restaurants), is present in 32 countries and is visited by approximately 142 million users monthly (Yelp.com 2015). Compared to competing restaurant platforms, Zomato is differentiated by having a strong presence in South Asia and the Middle East, in large cities traditionally under served by online restaurant search platforms, and by having a more comprehensive and reliable database about restaurant attributes than traditional crowd sourced content platforms.5 In 2014, 30 million unique users used Zomato every month to search for restaurants. The Zomato platform is accessible via an internet website or via a mobile application available on Android or AppleiOS smartphones. The website was launched in July 2008, the Android app in February 2010, and the iOS app in May 2011. On accessing the platform, users can search for restaurants by inputing a set of text-based keywords (e.g. some combination of the restaurant name, location, cuisine, or other attribute), or by searching by pre-established categories (e.g. a list of recommended restaurants in the users’ location that are open for service at the time of search). A variety of filters based on geographic location, cuisine, and intention (as defined as “home-delivery, dine-out or night-life”) can be applied as part of the search as desired. In response to the search, a list of restaurants that are determined by the platform to be relevant to the search criteria are displayed to the user on a search results page. Following the online search literature, we refer to these as “organic” listings. If an advertiser or a set of advertisers have contracted with Zomato to show ads for the search criteria and filters used, a set of advertised listings are also displayed on the search results page. We refer to these as “paid” listings or simply “ads”. The user can click on any of the displayed listings and subsequently browse a set of pages containing additional information specific to the listed restaurant. At any point of time, thousands of restaurants advertise on the Zomato website. Ads were served on the website from its launch days, but were officially launched on the mobile apps only in November 2014. Our experiment (described in more detail below) is implemented in August–September 2014 on the Android version of the Zomato app. Thus, as noted in the introduction, in the pre-experiment period, users are exposed to ads on the Zomato website, but see no ads on their mobile apps. 3.1.1. App search experience To understand how the experiment works, we describe a user’s pre-experiment search experience on the ZomatoAndroid mobile app in more detail. Figure (1a) shows a series of snapshots of a search session on the app. Applying a search criteria takes the individual to a search results page that displays listings that satisfy the user’s criteria. The search results are sorted by the search engine’s measure of “popularity” of a restaurant, unless the user specifies an alternate sorting criterion. Each listing on the search results page presents the name of the restaurant, its cuisine, its location, a flag for whether or not the menu is available, the number of photos available, along with the number and average value (on a five point scale: 0 lowest, 5 highest) of ratings given to the restaurant by past Zomato users (see Figure (1a)). Clicking on a listing takes the user to an “info” page that provides more information about the restaurant as shown in Figure (1b). This page has tabs that allow the user to view the restaurants’ menu, its photos, its location on a map, and to add reviews and photos if desired. It also shows the restaurants’ rating information and allows the user to browse through reviews provided by other users on the platform. Users interested in the restaurant can click on a button on this page to call the restaurant directly from their mobile phone. All user actions are tracked on the app. Figure 1 Open in new tabDownload slide Screen shots of search experience on ZomatoAndroid App. (a) Search flow for a user. (b) Information screen for user upon clicking on a restaurant listing Notes: The top panel shows a snapshot of the search experience of a user on the ZomatoAndroid app. The user searches for restaurants by inputing a set of text-based keywords (including for example, some combination of the restaurant name, location, cuisine, or other attribute), or by searching by pre-established categories (like requesting a list of recommended restaurants in the users location that are open for service at the time of search), while applying a variety of search filters (including limiting the search by geographic location, cuisine, and intention as defined as “home-delivery, dine-out, or night-life”). In response to the search, a list of restaurants that are determined by the platform to be most relevant to the search criteria are displayed on a search results page. The user can click on any of the displayed listings and is led to an info page that contains information specific to the listed restaurant including users reviews, the restaurant’s menu, photos, and a map. The bottom panel shows a snapshot of the information screen a user sees when he clicks on the search listing of a restaurant. The user is led to a screen from which he can browse several tabs to view the menu, read the reviews, see a map of the restaurant, or add his own review. The user can also call the restaurant from within the app, which we track as an activity key to conversion. Figure 1 Open in new tabDownload slide Screen shots of search experience on ZomatoAndroid App. (a) Search flow for a user. (b) Information screen for user upon clicking on a restaurant listing Notes: The top panel shows a snapshot of the search experience of a user on the ZomatoAndroid app. The user searches for restaurants by inputing a set of text-based keywords (including for example, some combination of the restaurant name, location, cuisine, or other attribute), or by searching by pre-established categories (like requesting a list of recommended restaurants in the users location that are open for service at the time of search), while applying a variety of search filters (including limiting the search by geographic location, cuisine, and intention as defined as “home-delivery, dine-out, or night-life”). In response to the search, a list of restaurants that are determined by the platform to be most relevant to the search criteria are displayed on a search results page. The user can click on any of the displayed listings and is led to an info page that contains information specific to the listed restaurant including users reviews, the restaurant’s menu, photos, and a map. The bottom panel shows a snapshot of the information screen a user sees when he clicks on the search listing of a restaurant. The user is led to a screen from which he can browse several tabs to view the menu, read the reviews, see a map of the restaurant, or add his own review. The user can also call the restaurant from within the app, which we track as an activity key to conversion. 3.1.2. Advertising on Zomato.com website As mentioned, there is no mobile advertising on Zomato prior to the experiment, but restaurants actively advertise on the Zomato.com website. To advertise, restaurants contract with Zomato to buy ads for a specific set of search criteria. A typical contract outlines the target location specified by the user that initiates the search, the category of the search, the day, and the position on which the advertised listing will be displayed. To specify location, Zomato divides each city it operates in into a set of non-overlapping (approximately) 5 miles |$\times$| 5 miles zones and allows restaurants to buy ads at any level of aggregation over these zones. To specify search intent, Zomato specifies three search categories viz., “home-delivery,” “dine-out,” or “night-life” as described previously. The contracting advertiser can specify the criteria for his ads narrowly (e.g. “shows ad at position |$X$| for any user who searches for home-delivery with target location specified as |$Y$| on Fridays for the next |$2$| weeks”), or broadly (“show ad at position |$X$| for any user who searches with target location specified as |$Y+Z$| on Fridays for the next |$2$| weeks”), depending on its needs. When a search is initiated on the Zomato website that satisfies a desired criteria, the contracted ad is shown. If more than one advertiser has contracted for that search criteria, Zomato uses its own proprietary algorithms to vary across users the positions at which these advertisers will be shown that day. Advertisers can negotiate separately to be exclusively featured at a given position for a given search criteria to avoid this. All ads are local. 3.2. Description of supply-side patterns To explore these, we look at the set of 142,934 restaurants in our data which received at least one review as of baseline.6 The database includes for these restaurants the average rating across reviewing users, the number of reviews, the estimate average cost for two people to have a meal at the restaurant (a price-index), as well as the date on which the restaurant is entered into the Zomato database. We use the values for these variable reported just prior to the beginning of the experiment and compare these in the cross-section. We therefore caution the reader these comparisons are suggestive and may be picking up unobserved differences in restaurant characteristics. We describe how the probability that a restaurant advertises on Zomato during the study is related to the average rating, number of ratings it receives, cost for two, and days since it was added to the database. Since these variables are market-specific and difficult to compare across regions, we compute the decile of each restaurant’s value on each of these variables within the zone in which it is located, and use the deciles as covariates in the regression below. We interpret the average rating as a proxy for the appeal of the restaurant, the number of ratings as proxy for its popularity, and/or a measure of uncertainty around the average, and the days-since-added as a proxy for the age of the restaurant since its market entry.7 Table 1 presents a multivariate analysis, regressing whether or not a restaurant advertised on restaurant characteristics including market fixed effects. Column (1) uses the absolute value of the characteristics, while columns (2) and (3) use within-market deciles. We also include an indicator of whether the restaurant is a part of a chain.8 Both columns (1) and (2) show that holding other characteristics constant, the likelihood of advertising increases with ratings, cost and the number of times the restaurant is rated, and decreases with days since the restaurant was added. Chain restaurants are also seen to be less likely to advertise.9 Column (3) investigates how the advertising behaviour changes with characteristics of other restaurants in the focal restaurant’s zone. For each zone, we calculate the average characteristics of restaurants in that zone, and categorize the zone into one of ten deciles based on the rank of its average within its city on that characteristic.10 Including these characteristics in the regression shows that while a chain restaurant is less likely to advertise, existence of chain restaurants in a restaurant’s zone is positively associated with advertising. A restaurant is also more likely to advertise if it is in a zone with older restaurants and with more ratings. Supplementary Appendix A presents additional plots that show these correlations continue to hold in univariate analyses. Table 1. Regressing restaurant advertising decision on restaurant characteristics Coeff: |$t$|-stat Coeff: |$t$|-stat Coeff: |$t$|-stat Restaurant rating 0.00784 18.32 Days since restaurant added –0.0000403 –14.49 Cost for two (USD) 0.000613 8.74 Number of ratings 0.000184 9.66 Restaurant is in a chain –0.00824 –6.95 –0.00962 –8.09 –0.0134 –10.13 Decile restaurant rating 0.00158 6.36 0.0024 9.89 Decile date restaurant added 0.00306 15.18 0.0033 12.89 Decile cost for two (USD) 0.00291 16.54 0.0027 12.03 Decile number of ratings 0.00351 14.01 0.0031 9.76 Zone decile: Avg rating –0.0003 –0.55 Zone decile: Avg Num Ratings 0.0014 2.73 Zone decile: Avg Date added –0.0009 –2.61 Zone decile: Avg Price Index 0.0007 1.94 Zone decile: Prop. of chain res 0.001 3.18 Zone decile: Num. of res –0.0004 –1.44 Intercept 0.0158 11.98 –0.0302 –20.78 –0.0392 –7.97 Market fixed effects Yes Yes Yes Num observations 65,192 65,192 65,192 Coeff: |$t$|-stat Coeff: |$t$|-stat Coeff: |$t$|-stat Restaurant rating 0.00784 18.32 Days since restaurant added –0.0000403 –14.49 Cost for two (USD) 0.000613 8.74 Number of ratings 0.000184 9.66 Restaurant is in a chain –0.00824 –6.95 –0.00962 –8.09 –0.0134 –10.13 Decile restaurant rating 0.00158 6.36 0.0024 9.89 Decile date restaurant added 0.00306 15.18 0.0033 12.89 Decile cost for two (USD) 0.00291 16.54 0.0027 12.03 Decile number of ratings 0.00351 14.01 0.0031 9.76 Zone decile: Avg rating –0.0003 –0.55 Zone decile: Avg Num Ratings 0.0014 2.73 Zone decile: Avg Date added –0.0009 –2.61 Zone decile: Avg Price Index 0.0007 1.94 Zone decile: Prop. of chain res 0.001 3.18 Zone decile: Num. of res –0.0004 –1.44 Intercept 0.0158 11.98 –0.0302 –20.78 –0.0392 –7.97 Market fixed effects Yes Yes Yes Num observations 65,192 65,192 65,192 Notes: The table reports results from regressing an indicator of whether a restaurant advertised on restaurant characteristics and market fixed effects. The unit of observation is a restaurant for which we observe the regression variables. Column (1) uses the absolute value of the restaurant characteristics as explanatory variables while columns (2) and (3) use the restaurant’s within-market decile. Column (3) adds Zone deciles as explanatory variables to check how advertising behaviour changes with the average characteristic of restaurants in the advertiser’s zone. The Zone deciles are created by first creating average characteristics of restaurants within a zone and assigning the zone to a decile based on its average relative to other zones. Open in new tab Table 1. Regressing restaurant advertising decision on restaurant characteristics Coeff: |$t$|-stat Coeff: |$t$|-stat Coeff: |$t$|-stat Restaurant rating 0.00784 18.32 Days since restaurant added –0.0000403 –14.49 Cost for two (USD) 0.000613 8.74 Number of ratings 0.000184 9.66 Restaurant is in a chain –0.00824 –6.95 –0.00962 –8.09 –0.0134 –10.13 Decile restaurant rating 0.00158 6.36 0.0024 9.89 Decile date restaurant added 0.00306 15.18 0.0033 12.89 Decile cost for two (USD) 0.00291 16.54 0.0027 12.03 Decile number of ratings 0.00351 14.01 0.0031 9.76 Zone decile: Avg rating –0.0003 –0.55 Zone decile: Avg Num Ratings 0.0014 2.73 Zone decile: Avg Date added –0.0009 –2.61 Zone decile: Avg Price Index 0.0007 1.94 Zone decile: Prop. of chain res 0.001 3.18 Zone decile: Num. of res –0.0004 –1.44 Intercept 0.0158 11.98 –0.0302 –20.78 –0.0392 –7.97 Market fixed effects Yes Yes Yes Num observations 65,192 65,192 65,192 Coeff: |$t$|-stat Coeff: |$t$|-stat Coeff: |$t$|-stat Restaurant rating 0.00784 18.32 Days since restaurant added –0.0000403 –14.49 Cost for two (USD) 0.000613 8.74 Number of ratings 0.000184 9.66 Restaurant is in a chain –0.00824 –6.95 –0.00962 –8.09 –0.0134 –10.13 Decile restaurant rating 0.00158 6.36 0.0024 9.89 Decile date restaurant added 0.00306 15.18 0.0033 12.89 Decile cost for two (USD) 0.00291 16.54 0.0027 12.03 Decile number of ratings 0.00351 14.01 0.0031 9.76 Zone decile: Avg rating –0.0003 –0.55 Zone decile: Avg Num Ratings 0.0014 2.73 Zone decile: Avg Date added –0.0009 –2.61 Zone decile: Avg Price Index 0.0007 1.94 Zone decile: Prop. of chain res 0.001 3.18 Zone decile: Num. of res –0.0004 –1.44 Intercept 0.0158 11.98 –0.0302 –20.78 –0.0392 –7.97 Market fixed effects Yes Yes Yes Num observations 65,192 65,192 65,192 Notes: The table reports results from regressing an indicator of whether a restaurant advertised on restaurant characteristics and market fixed effects. The unit of observation is a restaurant for which we observe the regression variables. Column (1) uses the absolute value of the restaurant characteristics as explanatory variables while columns (2) and (3) use the restaurant’s within-market decile. Column (3) adds Zone deciles as explanatory variables to check how advertising behaviour changes with the average characteristic of restaurants in the advertiser’s zone. The Zone deciles are created by first creating average characteristics of restaurants within a zone and assigning the zone to a decile based on its average relative to other zones. Open in new tab These patterns seem broadly consistent with a signalling equilibrium in which restaurants of higher-than-average “quality” advertise more to signal their appeal to consumers. The higher advertising propensity of newer restaurants, especially those of higher rating, and its gradual reduction over time can be explained with canonical signalling setups in which firms use ads as signal early in their life-cycle and scale back on dissipative advertising as information diffuses in the market and uncertainty about the restaurant’s appeal reduces. Signalling arguments can justify the higher advertising propensity of independent restaurants (not part of a chain); presumably, consumers have more uncertainty about these restaurants. The fact that higher priced restaurants tend to advertise more is also consistent with signalling equilibria in product markets where marginal costs increase with quality.11 While we do not have access to cost data, most components of marginal costs including food, labour, and service, tend to be higher for better restaurants. We further ask, are consumers likely to have a better experience at advertising restaurants relative to similarly rated non-advertising restaurants? We expect this to be true in a signalling equilibrium. Specifically, the restaurants that advertise are expected to be better along dimensions that are unobserved to consumers at the time of purchase. The higher unobserved quality could be revealed over time through consumption experiences. We use the ratings information to assess evidence for this. If we view ratings as summarizing consumption experiences, we expect ratings in the future (as well as the improvement in those ratings relative to baseline) to be higher for restaurants that advertised during the experiment relative to those that did not. To check whether this is the case, we pick at random 200 restaurants that advertise during our experimental time period, and 200 restaurants that do not. We search for their ratings on Zomato in September 2016 (two years after our experiment). We found 137 that advertised and 146 that had not advertised in 2014 (the probability of finding a restaurant is the same whether or not the restaurant had advertised; p = 0.32). Table 2 reports regressions comparing the 2016 ratings of restaurants depending on whether they advertised in 2014. Looking at the first column, we see the restaurants that advertised in 2014 have a higher rating in 2016. The second column reports the same regression but controlling for 2014 ratings. Among restaurants with the same rating in 2014, we see those that advertised also have a higher rating in 2016, i.e., the fact that a restaurant advertised is able to predict a better consumption experience for consumers beyond what is predicted by its contemporaneous ratings.12 Table 2. Ratings two-years after experiment Dependent variable: rating in 2016 Restaurant advertised in 2014 (0/1) 0.411 (0.094) 0.176 (0.076) Rating in 2014 0.241 (0.061) Intercept 3.066 (0.080) 2.479 (0.208) N 283 283 Dependent variable: rating in 2016 Restaurant advertised in 2014 (0/1) 0.411 (0.094) 0.176 (0.076) Rating in 2014 0.241 (0.061) Intercept 3.066 (0.080) 2.479 (0.208) N 283 283 Notes: The table shows coefficients and standard errors in parentheses from regressions comparing ratings of restaurants in 2016 depending on whether they advertised in 2014. Column 1 regresses 2016 ratings on whether the restaurant advertised in 2014. The coefficient is positive and significant suggesting that restaurants that advertised in 2014 have a higher rating in 2016. Column 2 adds control for ratings in 2014. Both coefficients are positive and statistically significant, suggesting that restaurants with higher ratings in 2014 tend to have higher rating in 2016; and that among restaurants with the same rating in 2014, those that advertised had a higher rating in 2016. Open in new tab Table 2. Ratings two-years after experiment Dependent variable: rating in 2016 Restaurant advertised in 2014 (0/1) 0.411 (0.094) 0.176 (0.076) Rating in 2014 0.241 (0.061) Intercept 3.066 (0.080) 2.479 (0.208) N 283 283 Dependent variable: rating in 2016 Restaurant advertised in 2014 (0/1) 0.411 (0.094) 0.176 (0.076) Rating in 2014 0.241 (0.061) Intercept 3.066 (0.080) 2.479 (0.208) N 283 283 Notes: The table shows coefficients and standard errors in parentheses from regressions comparing ratings of restaurants in 2016 depending on whether they advertised in 2014. Column 1 regresses 2016 ratings on whether the restaurant advertised in 2014. The coefficient is positive and significant suggesting that restaurants that advertised in 2014 have a higher rating in 2016. Column 2 adds control for ratings in 2014. Both coefficients are positive and statistically significant, suggesting that restaurants with higher ratings in 2014 tend to have higher rating in 2016; and that among restaurants with the same rating in 2014, those that advertised had a higher rating in 2016. Open in new tab Overall, these descriptive facts on the “supply-side” are broadly consistent with signalling theory. However, we note that they are only suggestive, because tests of signalling in across-restaurant comparisons are confounded with unobservable product and market-specific differences as cautioned in the Introduction. For this reason, we base a test for signalling primarily on more carefully constructed comparisons on the demand-side.13 4. Field Experiment The field experiment adds search ads into the Zomato mobile platform. The collaboration with Zomato was motivated by the firm’s desire to assess the viability of advertising on its mobile platform through pre-launch “A/B” testing (i.e. randomized controlled trials). The experiment starts in July 2014, when a new update of the android app was launched with the experiments encoded in it and made available on Google Play app store. The 13 cities in our data are: (India) Delhi, Kolkata, Mumbai, Bangalore, Pune, Hyderabad, Chennai, Lucknow, Ahmedabad; (UAE) Sharjah, Abu Dhabi; (Philippines) Manila; (Indonesia) Jakarta.14 Our raw data starts on 9 August 2014 and ends on 26 September 2014, when the next update of the app was launched. Following this, advertisements were formally launched on the Zomato mobile platform in November 2014.15 4.1. Users and randomization Any user who downloads the updated app with the experiment in it becomes part of the experiment and is allocated to one of several groups (described below). Randomization is induced at the user level and is persistent across all sessions by that user. Every user is assigned a unique id, and all activities of the user on the app subsequent to allocation to one of the experimental groups are tracked. 4.2. Experimental treatments Users are randomized into three experimental groups: Control: Users in the control group are shown no advertising and are exposed to only organic listings. These users experience no difference between the experimental and pre-experimental regimes, and serve as a baseline Treatment A (“Ad with no disclosure”): Users in group are shown the same organic listings as those in the control group, along with additional paid listings, but without any indication that these paid listings are ads. Treatment B (“Ad with disclosure”): Users in Group B are shown the same organic listings as those in the control group, along with exactly the same additional paid listings as those in Group A, but with an indication that the paidlistings are ads. Apart from the ad-indication, all other aspects of the ad, including the identity of the advertising restaurant, the content of the listing, and its position in the search results is the same between Groups A and B. The contrast between Groups A and B helps estimate the signalling effects of advertising. We include the control group because it benchmarks the firm’s no-advertising regime. Figure 2 shows an example of the three experimental groups. The left panel shows the control group, in which users are shown no advertising. The middle panel shows Group A, wherein users are shown additional paid listings (here for the restaurant “Smoke House Deli”), but without any indication that these paid listings are ads. The right panel shows Group B, in which users are shown exactly the same additional paid listings as those in Group A, but with a yellow label indicating that the paid listings are ads. Apart from the label, all other aspects of the ad is the same between Groups A and B.16 Figure 2 Open in new tabDownload slide Experimental conditions Notes: The figure shows a snapshot of the three experimental groups. In the control group, users are shown no advertising and are exposed to only organic listings. In Group A, users are shown additional paid listings (here for the restaurant “Smoke House Deli”), but without any indication that these paid listings are ads. In Group B, users are shown exactly the same additional paid listings as those in group A, but with an indication—here, a yellow label—indicating that the paid listings are ads. Apart from the indication, all other aspects of the ad, including the identity of the advertising restaurant, the content of the listing, and its position in the search results is the same between Groups A and B. In a typical search in the data, the first ad appears after four organic listings, which is likely to be below the page-fold (i.e. a user has to scroll down after arriving at a search-result page to see an ad). Figure 2 Open in new tabDownload slide Experimental conditions Notes: The figure shows a snapshot of the three experimental groups. In the control group, users are shown no advertising and are exposed to only organic listings. In Group A, users are shown additional paid listings (here for the restaurant “Smoke House Deli”), but without any indication that these paid listings are ads. In Group B, users are shown exactly the same additional paid listings as those in group A, but with an indication—here, a yellow label—indicating that the paid listings are ads. Apart from the indication, all other aspects of the ad, including the identity of the advertising restaurant, the content of the listing, and its position in the search results is the same between Groups A and B. In a typical search in the data, the first ad appears after four organic listings, which is likely to be below the page-fold (i.e. a user has to scroll down after arriving at a search-result page to see an ad). 4.3. Choice of advertisers included in experiment The inclusion of advertisers reflects the logic explained in the Introduction section. We show ads for restaurants that choose to advertise in equilibrium. When a user searches on the Zomatomobile app, his search criterion (the set of filters applied) is directed to an algorithm that reveals the restaurants whose ads the person would have seen if he was on the website rather than the mobile app. These restaurants are then advertised on the mobile app’s search listings page for users in Groups A and B. This way, we include in the experiment a set of advertisers that are interested in advertising on the platform in response to the search criteria applied by the user, and also mirror on the app the profile of advertisers on the website. The advertisers enjoy the potential benefits of the added mobile exposure for free during the duration of the experiment.17 The ads are all served on the search results page shown to a user. A search results page consists of 20 listings. The page may show up to three ads, placed in slots among the organic links as shown in the figures above. The order and position of the slots is decided by Zomato’s algorithm (i.e. we do not randomize over these; discussed below). The first ad appears after four organic listings on average across the searches in the data. Clicking on a restaurant’s ad takes a user to its info page. Ads are displayed only on the search results pages, and not on the restaurant info pages. There is no change in a restaurant’s info pages across the experimental groups, regardless of whether or not the restaurant is advertised. To keep the scale manageable and to reduce experimental interference, we do not show mobile ads for all possible search criteria. Ads in the experiment appear in search results only for broadly defined search criteria; only when the search is based on (a) target location and/or (b) the search categories “home-delivery,” “dine-out,” or “night-life”. If a consumer includes a cuisine in the search filter (or any other narrow factor apart from target location and search category) he does not see any ads in the search results, by design. Since advertising is sold on the basis of the customer’s target location and these three category filters, this helps ensure the search criteria for which ads are shown are aligned with those desired by advertisers (if an ad is shown as part of the experiment, there will for sure exist an advertiser that desired to advertise to that search). Also, by showing ads based on only these broadly defined searches, we reduce the chance the advertised restaurants are unrelated to the users’ search intent. For example, we avoid a situation where a user searches for a specific cuisine (e.g. Chinese restaurants) and sees restaurants in the search results that serve that cuisine, but ads that may not.18 Consider an individual who applied a search criterion for which a restaurant r’s ad is to be placed. If he is in the control group, he may see r’s link once, as a part of the organic listings. If he is in group A, he may see r’s link at least once if r does not appear in the organic listings, or twice if r does appear in the organic listings. Finally, if he is in group B, he may see r’s link once as an ad, and again if r appears in the organic listings.19 4.4. Order and positions On the mobile app, the order and position of advertised restaurants in both the paid and organic listings is determined by the platform on the basis of its own proprietary algorithms and its contractual arrangements. While ad-position is not under our control nor is randomized, what should be noted is the experiment ensures that if a restaurant’s ad is shown in a given position in response to a particular search criteria in Group A, its ad will be shown in response that same search criteria in the same position in Group B as well. This implies that position is held fixed in comparisons between Groups A and B. Further, while the sequence of organic listings are determined by the platform on the basis of its own proprietary algorithms, the experiment ensures that the sequence of organic listings shown for a given search criteria are the same in the control, A and B. This facilitates interpreting the difference between the groups as driven by the manipulations we induced to paid listings. To fix ideas, suppose the advertising returned in Figure 2—“Smoke House Deli”—has contracted for its ads to appear in response to searches from the “Hauz Khas Village” location in New Delhi City on 9 August 2014, and Zomato displays its ad in Group A as shown. Then, all searches in Group B from the “Hauz Khas Village” location in New Delhi City on 9 August 2014 with the “Smoke House Deli” ad served will also feature it on the same position. Further, the same sequence of organic listings that appeared in response to the search in Groups A and B appear in the control group as well. 5. A Simple Two Firm Model How would signalling change consumer behaviour between treatments A and B? We present a stylized theoretical model that explains our view on how signalling occurs in a setting like above, and helps interpret the difference in behaviour between treatments. The exposition is as follows. We first set up a model corresponding to Treatment B; then for Treatment A, and finally, contrast consumer behaviour between the two. The main difference between the model in Treatments A and B is in the information structure faced by the agents. From the consumer’s perspective, while ads and organic listings can be discerned in Treatment B, both listings are perceived as organic in Treatment A. We use this model to interpret our data. Supplementary Appendix B reviews the elements of the model in more detail. The model works as follows. While considering a purchase, a consumer is uncertain whether a product will satisfy his need. Some uncertainty can be resolved by searching on a search engine, which reveals information it has about the products. Firms selling the products have the information that can resolve the remaining uncertainty. In equilibrium, the firm with the better product advertises, paying the ad cost to signal high quality. The consumer visits the search engine, makes inferences and decides to purchase optimally. 5.1. Model setup There are two firms, |$j=1,2,$| one search engine and one consumer in the model. Each firm produces one product. The search engine is a stylized version of Zomato. It lists the two products on its search listings page, with the advertiser’s listing on top. The search engine also hosts two information pages, one for each product. A product’s information page can be accessed by clicking its listing, which might be identified as an ad by the consumer (when subject to Treatment B), or otherwise as an organic listing. The consumer has a need that may be satisfied by consuming a product. The chance that either product will satisfy the need depends on the product’s “quality,” which is ex ante uncertain to the consumer. A product |$j$| satisfies the need if the quality of |$j$| = |$s_{j}+e_{j}>0$|⁠, where |$s_{j}$| is the quality of |$j$| ’s search attribute and |$e_{j}$| is the quality of |$j$|’s experience attribute. The experience attribute is not verifiable by the consumer before purchase; it is revealed only after purchase. The freshness of a restaurant’s food is an experience attribute in the restaurant context. The search attribute is potentially verifiable before purchase. In the restaurant context it could be the restaurant’s menu. To make the analysis below easier, we assume that |$s_{j}=\bar{s}_{j}+v_{j}$|⁠, where |$\bar{s}_{j}$| is the mean, and |$v_{j}$| is mean-zero and symmetrically distributed with cdf |$\mathcal{F}$|⁠. Therefore, given beliefs about |$\bar{s}_{j}$| and |$e_{j}$|⁠, the probability of |$j$| satisfying the consumer’s need is |$\mathbb{P}(s_{j}+e_{j}>0)$| = |$\mathcal{F}\left(e_{j}+ \bar{s}_{j}\right)$|⁠.20|$^{,}$|21 One of the firms is higher on the experience attribute with |$e_{j}$| = |$H_{E}$|⁠, and the other is low with value |$L_{E}$| (⁠|$H_{E}>L_{E}$|⁠). Without loss of generality, let firm 1 be high-type on the experience attribute and firm 2 be low-type on the experience attribute. Similarly, one of the firms has a relatively high |$\bar{s}_{j}=H_{S}$| (e.g. in expectation, the consumer is more likely to be satisfied by the menu offered by the restaurant). The other has a lower |$\bar{s}_{j}=L_{S}$| (⁠|$H_{S}>L_{S}$|⁠). Nature allocates the search attribute randomly such that, with probability |$\tfrac{1}{2}$|⁠, firm 1 is |$H_{S}$| and firm 2 is |$L_{S}$| and, with probability |$\tfrac{1}{2}$|⁠, firm 1 is |$L_{S}$| and firm 2 is |$H_{S}s$|⁠. The search engine has information on the products’ search attributes, which it displays on the info pages. The search engine knows which product’s search attribute has a higher quality on average, through its algorithms and databases. That is, it knows |$\bar{s}{}_{j}$| (not |$s_{j}$|⁠). The search engine has no information on the experience attributes; it does not know, and cannot learn |$e_{j}$|⁠. Therefore, when the search engine shows more than one organic listing, it sorts them by |$\bar{s}_{j}$|⁠. The analogy in the restaurant context is as follows. The search engine may know the menus for both restaurants, but not the freshness of the ingredients they use. It may also be able to predict which restaurant’s menu is more popular, among the searching consumers. But it might not be able to predict whose menu is preferred by a given consumer. If his need is satisfied, the consumer obtains a payoff of |$1$|⁠. Otherwise, he gets |$0$|⁠. The consumer faces two kinds of costs. Visiting the product-page on the search engine and perusing it incurs a search cost of |$c.$| He also incurs a transaction cost |$t$| while buying the product. In our setting, the transaction cost could be the cost of traveling to a restaurant, or waiting for the food to arrive in case of home delivery.22 The firm with |$L_{E}$| gets a normalized payoff of 1 on selling the product, and the firm with |$H_{E}$| gets 1+|$\pi$| (⁠|$\pi>0$|⁠). The difference |$\pi$| represents the additional payoff due to repeat purchase, or higher order-size, that accrues to the product with the higher experience attribute. 5.2. Information structure and timing of the model Values |$H_{S},L_{S},H_{E},L_{E}$|⁠, distribution |$\mathcal{F}\left(.\right)$|⁠, consumer payoffs and costs |$c,t$|⁠, firm’s payoffs |$\pi$| are common knowledge. In the beginning, each player knows different pieces of the remaining information. Each firm knows the mean quality of its own and its competing product’s search attributes; it also knows the experience attribute qualities for both firms. That is, each firm knows |$\bar{s}_{1},e_{1},\bar{s}_{2},e_{2}$|⁠. The search engine knows the mean search attribute qualities of both products i.e., |$\bar{s}_{1},\bar{s}_{2}$|⁠, but not |$e_{1},e_{2}$|⁠. The consumer does not know |$\bar{s}_{1},e_{1},\bar{s}_{2},e_{2}$|⁠. No player knows the consumer’s exact quality value of the search attributes (⁠|$s_{1}$| and |$s_{2}$|⁠). The purchase decision unfolds in three stages. In Stage 1 the search engine assigns the ad to firm 1 or 2 via a second price auction with no reserve price. In Stage 2, the consumer visits the search engine, sees the search listings, and engages in optimal search. Consumer information acquisition in Stage 2 is as follows. If he is in treatment B, he infers |$e_{1},e_{2}$| on seeing the identity of the advertiser. If he is in treatment A, he infers |$\bar{s}_{1}$|⁠, |$\bar{s}_{2}$| on seeing the ordering of the listings. After making his inferences, the consumer can choose to click on any product listing, and explore its product info page. On arriving and perusing product |$j$|’s info page, |$s_{j}$| is revealed, and search cost |$c$| is incurred. Upon obtaining the additional information, the consumer can choose to proceed to purchase the product; if not, the consumer can incur the search cost again to browse the info page of the other product, and proceed to purchase it if desired. Finally, in Stage 3, if the consumer purchases |$j$|⁠, he incurs transaction cost |$t$|⁠. After purchase of |$j$|⁠, |$e_{j}$| is revealed. 5.3. Consumer beliefs and behaviour in Treatment B The following occurs in a separating equilibrium (detailed in Supplementary Appendix B). Firm 1 (type |$H_{E}$|⁠) advertises and is placed on top with a disclosure indicating the ad. The consumer believes it to be high on the experience attribute. Therefore, he believes |$e_{1}=H_{E}$| and |$e_{2}=L_{E}$|⁠. He is not able to infer the search attribute by looking at the listings. Hence, the consumer expects firm 1 to be more likely to fulfill his need relative to firm 2 based on the information he has. Therefore, he visits the advertiser’s page first with probability 1, and is likely to buy with probability |$\mathcal{F}(\bar{s}_{1}+H_{E})$|⁠. 5.4. Consumer beliefs and behaviour in Treatment A Treatment A experimentally manipulates only the disclosure of the ad, holding other factors such as the firms’ advertising behaviour constant.23 Therefore, much of the set up remains the same as in treatment B, with two exceptions. First, since the ad is not disclosed, it appears as the top organic link in the listings. Second, the consumer in Treatment A believes there is no advertising on the platform.24 Hence, the following occurs in equilibrium. Firm 1’s ad is placed on top without disclosure. The consumer perceives all listings as organic, and believes the top organic listing, which is firm 1, is high-type on the search attribute. He is unable to make inferences about the experience attribute. Since the consumer believes firm 1 (advertiser) to be more likely to satisfy his need, he visits firm 1 with probability 1. On visiting firm 1’s page |$s_{1}$| becomes known. Given the existence of a transaction cost, we show in Supplementary Appendix B that, in equilibrium, the consumer buys only when |$s_{1}+L_{E}>0$|⁠, otherwise he gets a negative expected payoff from purchase. Therefore the probability the consumer buys is |$\mathcal{F}(\bar{s}_{1}+L_{E})$| (see Supplementary Appendix B). 5.5. Comparing Treatment B versus Treatment A Page-visits to the advertiser do not change between A and B, and, the purchase probability is higher in treatment B by |$\mathcal{F}(\bar{s}_{1}+H_{E})-\mathcal{F}(\bar{s}_{1}+L_{E})>0$|⁠. This is the main prediction of the model. Further, noting that the value of advertising as a signal of quality to the consumer is higher ceteris paribus when the uncertainty about the experience attribute is higher, we consider a comparative static implied by the model: i.e., How does the difference between treatment A and B change when the variance in ex ante consumer beliefs about the experience attribute increases? In the model, the variance in ex ante consumer beliefs about the experience attribute across the two firms is |$\frac{1}{4}(H_{E}-L_{E})^{2}$|⁠.25 A mean preserving increase in variance occurs when |$H_{E}\rightarrow H_{E}+\delta$| and |$L_{E}\rightarrow L_{E}-\delta$| for |$\delta>0$|⁠. The model predicts this increases the effect of disclosure on purchase. This is because the change in purchase probability from treatment A to B is |$\mathcal{F}(\bar{s}_{1}+H_{E}+\delta)-\mathcal{F}(\bar{s}_{1}+L_{E}-\delta)>$||$\mathcal{F}(\bar{s}_{1}+H_{E})-\mathcal{F}(\bar{s}_{1}+L_{E})$| as |$\mathcal{F}(.)$| is increasing in its arguments. 5.5.1. Discussion The intuition behind the model’s predicted change in consumer behaviour from ad disclosure is as follows. Without ad disclosure, the consumer in Treatment A is uncertain about the experience attribute, but believes the advertiser is high type on the search dimension and visits the advertiser first. When the ad is disclosed, in Treatment B, the consumer is uncertain about the search attribute, but believes the advertiser is high type on the experience dimension, and visits it first. So page visits do not change between the treatments. On visiting the advertiser’s page, the search attribute is fully revealed in both treatments. But the consumer infers the experience attribute due to ad disclosure in Treatment B, and believes the advertiser to be higher on the experience dimension. So the consumer purchases from the advertiser with higher probability in Treatment B, relative to A. The model clarifies what we mean by “signalling”: it is the change in the consumer’s belief about the product’s experience attribute due to advertising—causing the consumer to believe that the advertiser is high-type on the experience attribute. In this model, the search engine does not display any search attribute information on the search listings page. Therefore, the consumer visits the advertiser’s page with probability |$1$|⁠. In reality, search engines display some information (e.g.Zomato displays a restaurant’s cuisine) along with organic listings. Consumers may have an idiosyncratic response to this information, which induces a probability < 1 on visiting the advertiser’s page in the two treatments. In a revised model that accommodates this, we can show that visits weakly increase when going from Treatment A to B, and the extent to which visits increase is smaller than the extent to which purchases increase. See Supplementary Appendix C. 6. Results from the Field-Experiment 6.1. Outcome measures Based on the above rationale, we use calls and page-visits as dependent measures.26 We use calls instead of actual demand because we are unable to track actual restaurant visits or spending. Both Zomato and restaurant advertisers regard calls to be sales-leads that are a good proxy for sales in the markets we consider. To check this, we analyse historical data on calls made to restaurants. We report this analysis in Supplementary Appendix E, and provide a short summary here. We use a sample of 1,033 recorded calls made by users to 28 restaurants that advertised on Zomato in October–November 2010 and content-code them manually. 69.5% of the calls involve the caller placing an order for home delivery. 8.5% involve reserving a table at the restaurant. Other calls related to purchase involve those placed to “takeout” food (1.8%) or to arrange for catering (1.3%). The remaining 18.9% are unrelated to purchase. Specifically, 13.4% of these ask for information without expressed purchase intent (e.g. details on the buffet, inquiries about whether the restaurant is open on a holiday). The others include marketing calls, wrong number dials etc. Overall, we find that 81.1% of the calls involve purchase intent from the caller. These data suggest that calls proxy for demand.27 6.2. Analysis based on first ad exposure The main results reported below utilize data only on users’ first exposures to experimental advertising. We do this to address the following econometric issue. Consider an individual in Group A or B who searches on the platform and is exposed to an ad. All of the user’s subsequent search behaviour on the platform—which drives his propensity to be exposed to more ads—could be influenced by the initial ad-exposure. For example, if advertising in group B is more effective than A, users in B might end up browsing fewer pages and be exposed to fewer (and possibly different) ads after the first. Therefore, comparing users conditional on subsequent searches or subsequent ads seen is subject to selection. To mitigate this, we base our tests on the effects of the first exposure to experimental ads on consumer-decisions. We implement the following steps to generate a dataset with such first exposures. We start by defining a session as constituting all actions starting with a users’ opening of the app on his phone up-to the beginning of a continuous period of inactivity that is longer than 3 hours. A session could comprise one or several searches, page visits and calls and represents a user trying to find a restaurant for a particular consumption occasion. For each individual (across all three groups), we examine the sequence of his searches in the data, and determine the first search for which ads would be served. For this search, we determine the restaurants who advertised, using the search-to-advertiser mapping.28 We restrict our analysis to the users’ actions related to the first restaurant that is advertised on this search. This is the first advertiser that a user sees (or could have seen). If multiple restaurants advertised at this position (which happens if Zomato decides to draw from a set of multiple restaurants given its contractual agreements), we consider the user’s actions related to each such restaurant. Further, if the advertising restaurant is a part of a chain, we consider the user’s actions related to all restaurants of that chain in the area. This analysis plan minimizes the bias from feedback effects by basing inference on the response to the first ad exposure for each user. The above procedure results in a dataset in which each observation in groups A & B is an individual |$\times$| the restaurants that are advertised at the first ad-slot in a users’ search results page. In the control group, each observation is an individual |$\times$| the first restaurant or restaurants that counterfactually would have advertised in response to his search if the user had been treated. The dataset is unbalanced (different restaurants |$r$| for each user |$i$|⁠). Each observation tracks as dependent variables indicators of whether the user visits the advertiser’s page and/or calls the advertiser during the session when the exposure occurs. Henceforth, we refer to this dataset as the “analysis dataset”. Supplementary Appendix F presents more details about the creation of this dataset. 6.2.1. Description of the analysis dataset We observe 265,975 users who download the app update, and apply at least once a search for which ads are served and thereby get included in the analysis dataset. These users are split across the three experimental groups as 44,233 (in control); 44,637 (in A); and 177,105 (in B). There are roughly four times more users in group B than A or the control, because B is comprised of additional sub-conditions as noted in Section 4.29 These users are spread across 321 zones. There are 622 advertisers that occur in the analysis dataset. On average, an advertiser occurs 589 times, with a standard deviation of 972. The distribution of occurrence of the advertisers in the data is skewed: the 10th, 50th, and 90th percentiles are 4, 128, and 1,711 respectively, for groups A and B. 61.7% of the user-sessions in the analysis dataset involve a visit to a restaurant’s information page, counting not just the advertisers. The average number of restaurant pages browsed in a session is 2.39. The 10, 25, 50, 75, 90th percentiles for page-visits are: 0 , 0, 1, 3, 6. 7.9% of sessions call a restaurant. Supplementary Appendix F reports means and balancing tests in user characteristics across the groups, including ANOVA tests on a variety of pre-experimental variables such as users’ past engagement with Zomato, search activity prior to the experiment, and the characteristics of restaurants visited in the past. There, we find the null of equal means across the three groups is not rejected for any of the variables, showing that randomization is induced properly. Table 3 reports the same for characteristics of advertising restaurants. We find no systematic differences across treatment groups. Supplementary Appendix F also documents that the occurrence of searches and advertisers are balanced across treatment groups as expected by randomization. Table 3. Advertiser characteristics in within-market deciles across experimental groups Rating decile Number of ratings decile Cost decile Date added decile Chain indicator Control 7.708 8.537 7.944 5.757 0.306 Treatment A 7.710 8.533 7.931 5.766 0.307 Treatment B 7.710 8.535 7.935 5.770 0.306 p-value 0.98 0.91 0.46 0.71 0.96 Rating decile Number of ratings decile Cost decile Date added decile Chain indicator Control 7.708 8.537 7.944 5.757 0.306 Treatment A 7.710 8.533 7.931 5.766 0.307 Treatment B 7.710 8.535 7.935 5.770 0.306 p-value 0.98 0.91 0.46 0.71 0.96 Note: Table shows sample averages and tests whether the baseline characteristics of the advertisers (in terms decile within market) that appear in our analysis dataset, differ across the three experimental groups. Open in new tab Table 3. Advertiser characteristics in within-market deciles across experimental groups Rating decile Number of ratings decile Cost decile Date added decile Chain indicator Control 7.708 8.537 7.944 5.757 0.306 Treatment A 7.710 8.533 7.931 5.766 0.307 Treatment B 7.710 8.535 7.935 5.770 0.306 p-value 0.98 0.91 0.46 0.71 0.96 Rating decile Number of ratings decile Cost decile Date added decile Chain indicator Control 7.708 8.537 7.944 5.757 0.306 Treatment A 7.710 8.533 7.931 5.766 0.307 Treatment B 7.710 8.535 7.935 5.770 0.306 p-value 0.98 0.91 0.46 0.71 0.96 Note: Table shows sample averages and tests whether the baseline characteristics of the advertisers (in terms decile within market) that appear in our analysis dataset, differ across the three experimental groups. Open in new tab Table 4. Average restaurant characteristics: advertisers and other restaurants (1) Advertisers (2) Non-advertisers Average rating 3.408 2.321 Number of ratings 193.57 49.90 Days since restaurant was added (as of 1 September 2015) 897.36 927.49 Cost index (USD) 15.23 9.80 Indicator of a chain restaurant 0.357 0.442 (1) Advertisers (2) Non-advertisers Average rating 3.408 2.321 Number of ratings 193.57 49.90 Days since restaurant was added (as of 1 September 2015) 897.36 927.49 Cost index (USD) 15.23 9.80 Indicator of a chain restaurant 0.357 0.442 Note: Column (1) shows mean characteristics in absolute term for advertisers that appear in our analysis dataset. Column (2) shows means for all the restaurants in the advertisers’ zones. For column (1) we use data for 569 (out of 622) advertisers for which characteristics data is not missing. Data on “Days since restaurant was added” is available for 472 restaurants. For Column (2) we use data on 33,067 restaurants, except the third row, which is a mean of 27,406 restaurants. Open in new tab Table 4. Average restaurant characteristics: advertisers and other restaurants (1) Advertisers (2) Non-advertisers Average rating 3.408 2.321 Number of ratings 193.57 49.90 Days since restaurant was added (as of 1 September 2015) 897.36 927.49 Cost index (USD) 15.23 9.80 Indicator of a chain restaurant 0.357 0.442 (1) Advertisers (2) Non-advertisers Average rating 3.408 2.321 Number of ratings 193.57 49.90 Days since restaurant was added (as of 1 September 2015) 897.36 927.49 Cost index (USD) 15.23 9.80 Indicator of a chain restaurant 0.357 0.442 Note: Column (1) shows mean characteristics in absolute term for advertisers that appear in our analysis dataset. Column (2) shows means for all the restaurants in the advertisers’ zones. For column (1) we use data for 569 (out of 622) advertisers for which characteristics data is not missing. Data on “Days since restaurant was added” is available for 472 restaurants. For Column (2) we use data on 33,067 restaurants, except the third row, which is a mean of 27,406 restaurants. Open in new tab 6.3. Main effects: visit and call rates Table 5 reports the change in page visit probability and the call probability in response to disclosure and represents the main results of the article. Table 5. Effect of ad-disclosure on call and page-visit Group Description Call Page-visit Mean Std. Err. Mean Std. Err. A Ad with no disclosure 0.031% 0.006% 1.10% 0.03% B Ad with disclosure 0.055% 0.004% 1.18% 0.02% p-value of test, H|$_{0}$|⁠: Equal means 0.002 0.084 t-stat 3.09 1.73 Group Description Call Page-visit Mean Std. Err. Mean Std. Err. A Ad with no disclosure 0.031% 0.006% 1.10% 0.03% B Ad with disclosure 0.055% 0.004% 1.18% 0.02% p-value of test, H|$_{0}$|⁠: Equal means 0.002 0.084 t-stat 3.09 1.73 Notes: There are 366,330 observations, 73,714 in A; and 292,616 in B corresponding to 44,637 unique users in A; and 177,105 unique users in B. Means represent the average of call and page-visit indicators across all users in each group in the analysis dataset. p-values computed by running a regression of the call/page-visit indicators for users in A and B on an indicator for group B, with standard-errors clustered at the user-level. Call probability is 77% higher when disclosing relative to the no-disclosure group. The visit rate with and without disclosure are statistically indistinguishable at 95% confidence level. Disclosure that a listing is a paid ad thus drives the incremental conversion. Open in new tab Table 5. Effect of ad-disclosure on call and page-visit Group Description Call Page-visit Mean Std. Err. Mean Std. Err. A Ad with no disclosure 0.031% 0.006% 1.10% 0.03% B Ad with disclosure 0.055% 0.004% 1.18% 0.02% p-value of test, H|$_{0}$|⁠: Equal means 0.002 0.084 t-stat 3.09 1.73 Group Description Call Page-visit Mean Std. Err. Mean Std. Err. A Ad with no disclosure 0.031% 0.006% 1.10% 0.03% B Ad with disclosure 0.055% 0.004% 1.18% 0.02% p-value of test, H|$_{0}$|⁠: Equal means 0.002 0.084 t-stat 3.09 1.73 Notes: There are 366,330 observations, 73,714 in A; and 292,616 in B corresponding to 44,637 unique users in A; and 177,105 unique users in B. Means represent the average of call and page-visit indicators across all users in each group in the analysis dataset. p-values computed by running a regression of the call/page-visit indicators for users in A and B on an indicator for group B, with standard-errors clustered at the user-level. Call probability is 77% higher when disclosing relative to the no-disclosure group. The visit rate with and without disclosure are statistically indistinguishable at 95% confidence level. Disclosure that a listing is a paid ad thus drives the incremental conversion. Open in new tab It shows that disclosure has a positive and statistically significant effect on the probability of calling the advertised restaurant. This implies a 77% increase in B relative to A (from 0.031% to 0.055%).30 This represents the causal effect of disclosure on calls and is consistent with the signalling prediction from the model in Section 5. To gauge the magnitude of the disclosure effect, we compare it to the call-probabilities the same restaurants would obtain in a world without disclosure, in response to a change in their characteristics. To do this, we estimate how much the call probability of the focal advertisers would change if their characteristics changed. As we do not randomize these characteristics, this is not a causal effect. Nevertheless, we regress an indicator of whether any of the advertisers are called by users in group A, on the within-zone deciles of the attributes we reported on in the previous section (average rating, number of ratings received, cost for two and days since it was added to the database). Supplementary Appendix G presents the regression results. We find the disclosure effect is comparable to a two decile increase in the number of ratings received (statistically significantly estimated). So, roughly speaking, an average advertiser in our dataset obtains the same benefit in calls by disclosing sponsorship as a two decile increase in the number of ratings it has on the platform. Looking at the same table, we see that while ad disclosure drives more people to the advertiser’s page during the session, an increase is not statistically significant at the 95% confidence level.31 Overall, these patterns are consistent with signalling, as discussed in Section 5. What could explain the fact that calls increase more significantly than page-visits? Our interpretation of what drives this result is clarified by the model: a consumer visits the advertiser’s page in both treatments. In treatment A he believes the advertiser to be better along a search attribute, whereas in treatment B he believes the advertiser to be better along an experience attribute. The value of the positive belief diminishes in treatment A because visiting the product information page clearly reveals the search attributes. On the other hand, a consumer in treatment B continues to believe the advertiser to be better on the experience attribute after visiting the advertiser’s page, so he proceeds to call the restaurant at a higher rate relative to a consumer in treatment A. Supplementary Appendix B further elaborates on this aspect. Additionally, since the advertisers in our setting tend to have high ratings (e.g.Table 1 or Figure 4a in Supplementary Appendix A), and ratings are displayed on the search-results page, a consumer may decide to visit the advertiser’s page on seeing its listing regardless of disclosure. To summarize our main findings, ad disclosure has a measurable positive effect on demand that is consistent with the signalling predictions. Additional statistical robustness on these results, including exact p-values computed via simulation, and controls for search and advertiser characteristics implemented in a regression setting (via fixed effects for search-query and for advertisers) is reported in Section 7, and Supplementary Appendix I. 6.3.1. Exploring further: conditional call rate and continued search According to the model, a consumer in treatment A is less likely to be satisfied on visiting the advertiser’s page, and more likely to continue searching, relative to a consumer in treatment B. Can we find direct evidence for this? We report two additional comparisons to examine. The left panel of Table 6 compares the call probability conditional on a page-visit between A and B. This comparison focuses on the subset of individuals who chose to visit an advertising restaurant’s info page in each of the groups, and then compares the proportion of the subset that called the restaurant. We caution that since each subset within a group is a selected subsample, this comparison does not necessarily hold fixed user profiles across groups and does not estimate a clean causal effect. Nevertheless, it helps describe the effects found above. Table 6 shows that the conversion from a page-visit to a call is highest for the subset of users who visit the restaurant’s info page while seeing that the listing is a paid ad, and higher compared to showing the listing in the same position but without disclosure (a 76% improvement for B compared to A). Table 6. Call probability conditional on visit and probability of continuing search after a page-visit, split by experimental group Pr(Call | Visit) Pr(Continue Search | Visit) Group Description Mean SE p-value* Mean SE p-value* A Ad with no disclosure 2.33% 0.50% 0.005 78.50% 1.4% 0.18 B Ad with disclosure 4.11% 0.30% 76.36% 0.7% Pr(Call | Visit) Pr(Continue Search | Visit) Group Description Mean SE p-value* Mean SE p-value* A Ad with no disclosure 2.33% 0.50% 0.005 78.50% 1.4% 0.18 B Ad with disclosure 4.11% 0.30% 76.36% 0.7% Notes: *H|$_{0}$|⁠: Equal means. The table reports on 4,214 users who visited the page of an advertising restaurant across the groups. There are 814 observations in A; 3,452 observations in B; corresponding to 808 unique such users in A; and 3,406 unique such users in B. Means represent the estimated probability of calling the advertiser conditional on visit (left panel), and the estimated probability of visiting another restaurant’s info page after visiting the advertiser’s page (right panel). p-values computed by running a regression of the call/page-visit indicators for users in A and B on an indicator for group B, with standard-errors clustered at the user-level. The conditional call probability is 76% higher when the ad is placed with disclosure relative to the no-disclosure group. Disclosure that a listing is a paid ad thus seems to make the restaurant’s appeal stronger to exposed users. The probability of a user continuing search after visiting an advertiser’s page goes down moving from group A to B, though the difference is not statistically significant. Open in new tab Table 6. Call probability conditional on visit and probability of continuing search after a page-visit, split by experimental group Pr(Call | Visit) Pr(Continue Search | Visit) Group Description Mean SE p-value* Mean SE p-value* A Ad with no disclosure 2.33% 0.50% 0.005 78.50% 1.4% 0.18 B Ad with disclosure 4.11% 0.30% 76.36% 0.7% Pr(Call | Visit) Pr(Continue Search | Visit) Group Description Mean SE p-value* Mean SE p-value* A Ad with no disclosure 2.33% 0.50% 0.005 78.50% 1.4% 0.18 B Ad with disclosure 4.11% 0.30% 76.36% 0.7% Notes: *H|$_{0}$|⁠: Equal means. The table reports on 4,214 users who visited the page of an advertising restaurant across the groups. There are 814 observations in A; 3,452 observations in B; corresponding to 808 unique such users in A; and 3,406 unique such users in B. Means represent the estimated probability of calling the advertiser conditional on visit (left panel), and the estimated probability of visiting another restaurant’s info page after visiting the advertiser’s page (right panel). p-values computed by running a regression of the call/page-visit indicators for users in A and B on an indicator for group B, with standard-errors clustered at the user-level. The conditional call probability is 76% higher when the ad is placed with disclosure relative to the no-disclosure group. Disclosure that a listing is a paid ad thus seems to make the restaurant’s appeal stronger to exposed users. The probability of a user continuing search after visiting an advertiser’s page goes down moving from group A to B, though the difference is not statistically significant. Open in new tab The right panel of Table 6 shows the probability of a user continuing to search during the session, i.e., visiting another restaurant’s info page after visiting the advertiser’s info page, across experimental groups. As a first observation, note that the probability of a user continuing to search after visiting a restaurant’s page is high: 78.5% in group B. This shows this is a competitive setting with significant search, and a large majority of the individuals who consider a restaurant continue to explore more options. Comparing the estimates across groups, we see that users in B continue to search at similar (or lower) rate after seeing the advertised restaurants’ page. 6.3.2. Evidence for signalling Effect We infer the estimated effect of ad-disclosure—the consumer seeing that a restaurant is advertised—as evidence supporting the signalling effect, as described in Section 5. Ad disclosure causes the consumer to change his beliefs about the restaurant’s appeal, which is shown by his changed propensity to call the restaurant. Since we are holding ratings and other characteristics fixed in our comparisons, the effect we measure represents the informational value of advertising over and above the information provided to consumers by ratings and other characteristics. We believe that a mere indication disclosing the ad is unlikely to directly affect intrinsic restaurant preferences. While we cannot rule out this possibility emphatically, note that preference changing effects like persuasive or complementary effects of advertising are typically understood as operating through the content of the ad (for example, through the positive feelings evoked by the ad-copy or through positive associations kindled in memory through reminders of existence or enjoyment embedded in the ad). Such effects may well coexist with the signalling mechanism, but may operate through other aspects of the ad. To the extent that we have controlled for all aspects of ad-content, awareness, positions and existence, we believe what we have isolated in the response to disclosure is a cleaner signalling effect relative to the past literature. In Section 7, we consider other mechanisms not related to signalling, and show that they do not explain our data. 6.4. Investigating heterogeneity in more detail This section augments the main results reported previously by examining systematic heterogeneity in effects, and comparing them with the predictions of the model discussed in Section 5. The main comparative static is that the signal is likely to be more useful when uncertainty about the unobserved quality of the good is higher (see Section 5.5). As operationalization of this idea, we test whether the effect of the disclosure is higher for subpopulations of restaurants about which users are ex ante more likely to be uncertain. We also test whether the effect is higher for subpopulations of users that are likely to have more uncertainty about the appeal of the restaurants they are considering. 6.4.1. Behaviour of users searching in an outside city To identify a subpopulation of users who have more uncertainty about quality, we look for users in each experimental group who search for restaurants in a city that is different from the city where they predominantly searched in the past. Presumably, these users are more uncertain about the appeal of the restaurants they are browsing, compared to users searching in their home cities. Therefore, we expect the out-of-city (i.e. those who travel) subpopulation to be more responsive to ad disclosure, compared to the remaining “usual-city” subpopulation, as predicted by the signalling theory. With this motivation we test whether the difference between outcomes in A and B is larger for the out-of-city subpopulation compared to the rest. Note that the “usual-city” and out-of-city users may also differ on other aspects, apart from their uncertainty about restaurant quality. It is possible that these other differences cause the differential impact of disclosure. To the extent possible, we use our observed data to control for other observed factors in the regression analysis below. We maintain the assumption that the reason these users travel is not influenced by their experimental assignment. So, the ex ante distribution of usual-city and out-of-city users is the same in treatment groups A and B. A t-test is unable to reject that the proportion of out-of-city searchers in groups A and B is equal (p-value = 0.89). We use the historical data prior to the first session for each individual to identify the city in which the individual conducts most of his searches.32 We find that 5,031 users search for restaurants in a city that is different from the one they usually conduct their searches. We test whether the ad-disclosure has a differential effect for these user-sessions, compared to the rest of the users. Unfortunately, we do not have enough power to explore a similar comparison for the call variable.33 Table 7 compares the means for this analysis. Looking across the rows in the left column, we see that users searching in a different city from their usual one behave differently and visit the advertised restaurant’s page at a significantly higher rate if they are advertised with disclosure. This behaviour contrasts with the rest of the sample, whose behaviour looks similar to that reported for the overall group in Table 5: a statistically insignificant difference in the visit propensity with disclosure relative to no disclosure. Looking at the numbers across columns, within treatment A we see that probability of visiting the advertiser’s page is smaller among out of city searchers, relative to the rest of the population. However, the statistical significance of this difference is weak. In treatment B, the direction of this comparison reverses; out of city searchers have a significantly larger probability of visiting the advertiser’s page, relative to the rest of the population (p-value < 0.01). This indicates that the difference in behaviour between usual-city and out-of-city users increases due to disclosure. On the whole, these findings are consistent with a signalling role for advertising. The effect on page-visits, in particular, can be rationalized by the extension to the theory model (presented in Supplementary Appendix C), in which the search engine displays some search attribute information on the listings page, and consumers are allowed to have idiosyncratic responses to it.34 Table 7. Visit probability split by experimental group and by user uncertainty User sessions in a different city from hometown Remaining user sessions p-value of test* Group Description Mean SE Mean SE A Ad with no disclosure 0.71% 0.22% 1.11% 0.04% 0.077 B Ad with disclosure 1.73% 0.18% 1.17% 0.05% |$<$|0.01 p-value of test|$^{\dagger}$| |$<$|0.01 0.194 User sessions in a different city from hometown Remaining user sessions p-value of test* Group Description Mean SE Mean SE A Ad with no disclosure 0.71% 0.22% 1.11% 0.04% 0.077 B Ad with disclosure 1.73% 0.18% 1.17% 0.05% |$<$|0.01 p-value of test|$^{\dagger}$| |$<$|0.01 0.194 Notes: *H|$_{0}$|: Equal means across columns. |$^{\dagger}$| H|$_{0}$|: Equal means across rows. The table reports on two subsets of the sample of observations. We observe 1,409 user |$\times$|advertiser observations in group A, and 5,489 in B. The right panel reports on the rest of the users. The table reports the differences in visit probability, separately by these two groups of users, across all relevant sessions in each group. p-values in the last row are computed by regressing the info page-visit indicators for users in A and B on an indicator for group B, separately for two sets of users, with standard-errors clustered at the user-level. p-values in the last column are estimated similarly with the explanatory variable an indicator of whether the user is in out-of-city set or not. Users searching in a different city from their usual one are seen to behave differently: disclosure has a strong effect for users searching in a different city, but there is no statistically distinguishable effect for the rest. Open in new tab Table 7. Visit probability split by experimental group and by user uncertainty User sessions in a different city from hometown Remaining user sessions p-value of test* Group Description Mean SE Mean SE A Ad with no disclosure 0.71% 0.22% 1.11% 0.04% 0.077 B Ad with disclosure 1.73% 0.18% 1.17% 0.05% |$<$|0.01 p-value of test|$^{\dagger}$| |$<$|0.01 0.194 User sessions in a different city from hometown Remaining user sessions p-value of test* Group Description Mean SE Mean SE A Ad with no disclosure 0.71% 0.22% 1.11% 0.04% 0.077 B Ad with disclosure 1.73% 0.18% 1.17% 0.05% |$<$|0.01 p-value of test|$^{\dagger}$| |$<$|0.01 0.194 Notes: *H|$_{0}$|: Equal means across columns. |$^{\dagger}$| H|$_{0}$|: Equal means across rows. The table reports on two subsets of the sample of observations. We observe 1,409 user |$\times$|advertiser observations in group A, and 5,489 in B. The right panel reports on the rest of the users. The table reports the differences in visit probability, separately by these two groups of users, across all relevant sessions in each group. p-values in the last row are computed by regressing the info page-visit indicators for users in A and B on an indicator for group B, separately for two sets of users, with standard-errors clustered at the user-level. p-values in the last column are estimated similarly with the explanatory variable an indicator of whether the user is in out-of-city set or not. Users searching in a different city from their usual one are seen to behave differently: disclosure has a strong effect for users searching in a different city, but there is no statistically distinguishable effect for the rest. Open in new tab Can this difference in responsiveness to ad disclosure be explained by observable differences between out of city searchers and the remaining population? To investigate this, we conduct the above analysis in a regression framework and control for observable characteristics of the individual’s circumstances, and the systematically different effects they may cause. Specifically, we operationalize this regression as $$\begin{equation} Y_{ir}=\alpha+\beta I_{i,B}+\gamma_{1}I_{i,B}\times\text{diff_city}_{i}+\lambda_{1}\text{diff_city}_{i}+\gamma_{2}I_{i,B}\times\text{X}_{ir}+\lambda_{2}\text{X}_{ir}+\epsilon_{ir}, \end{equation}$$ (6.3) where |$Y_{ir}$| is an indicator of whether the individual |$i$| visited advertising restaurant |$r$|’s page; |$I_{i,B}$| is an indicator that |$i$| is in treatment group B; diff_city|$_{i}$| is an indicator that |$i$| is searching in an outside city; X|$_{ir}$| is a vector of observed characteristics that includes an 13 fixed effects for cities, and advertiser |$r$|’s characteristics described in Section 3.2; |$\lambda_{1}$| is the deviation in average page-visits for individuals who are in a different city; |$\gamma_{1}$| is the increase in sensitivity to Treatment B for individuals who are in a different city; |$\lambda_{2}$| captures the covariation in page-visits with observed characteristics, and |$\gamma_{2}$| captures how this covariation changes with disclosure. If the difference in sensitivity to disclosure between out of city searchers and the rest of the population observed in Table 7 can be explained by any of the factors in the vector X, we expect |$\gamma_{1}$| to be insignificant. Table 8 shows the results of this regression with and without adding the control variables. Column 1 without the controls replicates the results in the Table 7. Columns 2 and 3 show that adding the controls does not change the focal coefficient significantly. This indicates that the behaviour documented is not likely being driven by higher effects in cities to which travelers go, or types of restaurants whose ads they see. Table 8. Visit probability split by experimental treatment and by user uncertainty: controlling for observable characteristics Dependent variable (1) Page-visit (2) Page-visit (3) Page-visit Coeff: SE t-stat Coeff: SE t-stat Coeff: SE t-stat |$I_{i,B}$| 0.00057 0.00044 1.30 0.00072 0.00156 0.46 0.00466 0.0034 1.36 |$I_{i,B}{\times}\text{diff_city}_{i}$| 0.00964 0.00294 3.28 0.00965 0.00295 3.26 0.01135 0.00336 3.37 |$\text{diff_city}_{i}$| –0.00402 0.00226 –1.78 –0.00406 0.00228 –1.78 –0.00526 0.00252 –2.09 Intercept 0.01111 0.00039 28.40 0.01209 0.00138 8.74 –0.0070 0.0030 –2.32 13 fixed effects for city? No Yes Yes |$I_{i,B}\times$|fixed effects for city No Yes Yes Adding advertiser features? No No Yes |$I_{i,B}\times$|advertiser features No No Yes Num. of observations 366,330 366,330 286,651 Num. individuals (clusters) 221,742 221,742 202,913 Dependent variable (1) Page-visit (2) Page-visit (3) Page-visit Coeff: SE t-stat Coeff: SE t-stat Coeff: SE t-stat |$I_{i,B}$| 0.00057 0.00044 1.30 0.00072 0.00156 0.46 0.00466 0.0034 1.36 |$I_{i,B}{\times}\text{diff_city}_{i}$| 0.00964 0.00294 3.28 0.00965 0.00295 3.26 0.01135 0.00336 3.37 |$\text{diff_city}_{i}$| –0.00402 0.00226 –1.78 –0.00406 0.00228 –1.78 –0.00526 0.00252 –2.09 Intercept 0.01111 0.00039 28.40 0.01209 0.00138 8.74 –0.0070 0.0030 –2.32 13 fixed effects for city? No Yes Yes |$I_{i,B}\times$|fixed effects for city No Yes Yes Adding advertiser features? No No Yes |$I_{i,B}\times$|advertiser features No No Yes Num. of observations 366,330 366,330 286,651 Num. individuals (clusters) 221,742 221,742 202,913 Notes: The table reports the regression estimates of the model in equation (6.3), with standard-errors clustered at the user-level. It pools observations of individuals in experiment groups A and B. Column (2) controls for fixed effects for each of the thirteen cities, and interaction of these fixed effects with an indicator of the individual being in treatment group B (the base city is city-id 5 which has the median sensitivity to Treatment B). Column (3) adds advertiser-related features we describe in Section 3.2, and their interaction with indicator for treatment B as controls. This column has fewer observations because observed characteristics are missing for some restaurants. Column (1) reports results without these controls. Adding the fixed effects and other variables does not change the coefficient of |$I_{i,B}\times\text{diff_city}_{i}$|⁠, indicating that the users searching in a different city from their usual one increase their page-visits to the advertising restaurant because of ad disclosure. This increase is not due to the nature of the city they are searching in, or advertisers they get exposed to. Open in new tab Table 8. Visit probability split by experimental treatment and by user uncertainty: controlling for observable characteristics Dependent variable (1) Page-visit (2) Page-visit (3) Page-visit Coeff: SE t-stat Coeff: SE t-stat Coeff: SE t-stat |$I_{i,B}$| 0.00057 0.00044 1.30 0.00072 0.00156 0.46 0.00466 0.0034 1.36 |$I_{i,B}{\times}\text{diff_city}_{i}$| 0.00964 0.00294 3.28 0.00965 0.00295 3.26 0.01135 0.00336 3.37 |$\text{diff_city}_{i}$| –0.00402 0.00226 –1.78 –0.00406 0.00228 –1.78 –0.00526 0.00252 –2.09 Intercept 0.01111 0.00039 28.40 0.01209 0.00138 8.74 –0.0070 0.0030 –2.32 13 fixed effects for city? No Yes Yes |$I_{i,B}\times$|fixed effects for city No Yes Yes Adding advertiser features? No No Yes |$I_{i,B}\times$|advertiser features No No Yes Num. of observations 366,330 366,330 286,651 Num. individuals (clusters) 221,742 221,742 202,913 Dependent variable (1) Page-visit (2) Page-visit (3) Page-visit Coeff: SE t-stat Coeff: SE t-stat Coeff: SE t-stat |$I_{i,B}$| 0.00057 0.00044 1.30 0.00072 0.00156 0.46 0.00466 0.0034 1.36 |$I_{i,B}{\times}\text{diff_city}_{i}$| 0.00964 0.00294 3.28 0.00965 0.00295 3.26 0.01135 0.00336 3.37 |$\text{diff_city}_{i}$| –0.00402 0.00226 –1.78 –0.00406 0.00228 –1.78 –0.00526 0.00252 –2.09 Intercept 0.01111 0.00039 28.40 0.01209 0.00138 8.74 –0.0070 0.0030 –2.32 13 fixed effects for city? No Yes Yes |$I_{i,B}\times$|fixed effects for city No Yes Yes Adding advertiser features? No No Yes |$I_{i,B}\times$|advertiser features No No Yes Num. of observations 366,330 366,330 286,651 Num. individuals (clusters) 221,742 221,742 202,913 Notes: The table reports the regression estimates of the model in equation (6.3), with standard-errors clustered at the user-level. It pools observations of individuals in experiment groups A and B. Column (2) controls for fixed effects for each of the thirteen cities, and interaction of these fixed effects with an indicator of the individual being in treatment group B (the base city is city-id 5 which has the median sensitivity to Treatment B). Column (3) adds advertiser-related features we describe in Section 3.2, and their interaction with indicator for treatment B as controls. This column has fewer observations because observed characteristics are missing for some restaurants. Column (1) reports results without these controls. Adding the fixed effects and other variables does not change the coefficient of |$I_{i,B}\times\text{diff_city}_{i}$|⁠, indicating that the users searching in a different city from their usual one increase their page-visits to the advertising restaurant because of ad disclosure. This increase is not due to the nature of the city they are searching in, or advertisers they get exposed to. Open in new tab 6.4.2. Consumer behaviour with respect to advertiser characteristics We now examine the characteristics of restaurants that users visit/call under ad-disclosure. Following the discussion in Section 5.3, we expect users to place more weight on the signal-value incorporated into the ad-disclosure when they have higher prior uncertainty about the advertising restaurant’s appeal. Thus, if we compare across restaurants, we should see that those restaurants about which consumers a priori have more uncertainty benefit more from the ad-disclosure, all other things held equal. As an operationalization of this idea, we check whether newer restaurants and those rated by fewer users on Zomato benefit more from ad-disclosure controlling for other observable restaurant attributes. Implicitly, we are treating fewer ratings and newer entry as proxies for higher uncertainty, though these variables could be proxying for other things like popularity or quality, so the test is only suggestive and is weaker than the ones presented in Section 6.3. We also observe other characteristics about restaurants including the average rating and price-index presented on the Zomato listings. These characteristics shift the baseline chance of the individual choosing a restaurant. Theory does not have clear predictions about how the effect of the signal will change with such characteristics. Pick for instance, the average rating of a restaurant. If the rating is high enough, a user may not need an additional signal to buy from it. On the other hand, if the rating is low enough, a strong signal may not be enough for an individual to buy. Similar arguments hold for other characteristics that shift the baseline. To hold observed characteristics fixed, we analyze the results in a regression set-up. Let |$I_{i,B}$| be an indicator of whether user |$i$| is in group B, and let |$d_{r,\textrm{Num-rating}}$|⁠, |$d_{r,\textrm{Ave-rating}}$|⁠, |$d_{r,\textrm{Price-Index}}$| and |$d_{r,\textrm{Date-Added}}$| denote the within-zone deciles of restaurant |$r$| on the respective variables. Represent by |$Y_{ir}$| an indicator for whether user |$i$| calls restaurant |$r$|⁠. Stacking across all user, advertiser combinations in groups A and B in the analysis dataset, we run the following regression, $$Y_{ir}=\alpha+\beta I_{i,B}+\lambda_{1}d_{r,\textrm{Num-rating}}+\lambda_{2}d_{r,\textrm{Ave-rating}}+\lambda_{3}d_{r,\textrm{Price-Index}}+\lambda_{4}d_{r,\textrm{Date-Added}}\\ \quad\quad\quad\quad\quad +I_{i,B}\times\left[\gamma_{1}d_{r,\textrm{Num-rating}}+\gamma_{2}d_{r,\textrm{Ave-rating}}+\gamma_{3}d_{r,\textrm{Price-Index}}+\gamma_{4}d_{r,\textrm{Date-Added}}\right]+\epsilon_{ir.}$$ We also report the same regression using an indicator for page visit as the dependent variable. Recall that in the earlier analysis, we did not find a significant effect of ad-disclosure on page-visits. Therefore, a priori, we expect to find systematic heterogeneity in the effect of ad-disclosure on calls but not visits. In both, the main interest is in the interactions of the number of ratings and age variables with the group B dummy (⁠|$\gamma_{1}$| and |$\gamma_{4}$|⁠). These pick up the comparative static on prior uncertainty and prior appeal and are expected to be negative under the signalling hypothesis. We include interactions with the average rating and the price index so as to describe the heterogeneity in treatment response. Table 9 presents the results. Looking at the table, we see that restaurants of higher rating benefit more from ad disclosure in terms of both page visits and calls. The interaction of the group B dummy with the age of the restaurant is not statistically significant. We see that restaurants that have fewer ratings benefit more from the ad-disclosure in terms of their call probability (coefficient on |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| in the call regression). These interactions are consistent with a signalling mechanism. The effects are also quantitatively significant. Holding everything else the same, a restaurant that is rated fewer—by one decile at baseline, is likely to experience an incremental effect of ad-disclosure of +0.013%. This increase is equivalent to 40% of the average baseline call rate of 0.03%. Table 9. Heterogeneity in responsiveness to disclosure across restaurant types Dependent variable: Page visit Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| 00.04% –1.19 |$-$|0.01% |$-$|2.22 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.01% –0.56 –0.004% –1.25 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.06% 2.06 0.02% 3.43 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.01% –0.20 0.006% –1.36 |$d_{r,\textrm{Num-rating}}$| 0.03% 0.88 0.01% 2.26 |$d_{r,\textrm{Date-Added}}$| 0.004% 0.28 0.003% 1.00 |$d_{r,\textrm{Ave-rating}}$| 0.08% 3.35 –0.01% –1.50 |$d_{r,\textrm{Price-Index}}$| 0.10% 4.39 0.00% 0.14 |$\mathcal{I}_{i,B}$| 0.07% 0.24 0.05% 1.33 Intercept |$-$|0.51% |$-$|2.04 –0.03% –0.86 Num. observations 286,651 286,651 Num. individuals (clusters) 202,913 202,913 |$R^{2}$| 0.0012 0.0001 Dependent variable: Page visit Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| 00.04% –1.19 |$-$|0.01% |$-$|2.22 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.01% –0.56 –0.004% –1.25 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.06% 2.06 0.02% 3.43 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.01% –0.20 0.006% –1.36 |$d_{r,\textrm{Num-rating}}$| 0.03% 0.88 0.01% 2.26 |$d_{r,\textrm{Date-Added}}$| 0.004% 0.28 0.003% 1.00 |$d_{r,\textrm{Ave-rating}}$| 0.08% 3.35 –0.01% –1.50 |$d_{r,\textrm{Price-Index}}$| 0.10% 4.39 0.00% 0.14 |$\mathcal{I}_{i,B}$| 0.07% 0.24 0.05% 1.33 Intercept |$-$|0.51% |$-$|2.04 –0.03% –0.86 Num. observations 286,651 286,651 Num. individuals (clusters) 202,913 202,913 |$R^{2}$| 0.0012 0.0001 Notes: The table reports on 202,913 users in groups A and B in the analysis dataset who are exposed to ads for restaurants on which we have restaurant-attribute information. The table reports on the heterogeneity in responsiveness to ad-disclosure by running a regression of visit/call indicators across all relevant users in groups A and B in the analysis dataset, on a dummy for whether the session belonged to group B (⁠|$\mathcal{I}_{i,B}$|⁠), and on interactions of this dummy with |$d_{r,\textrm{Num-rating}}$|⁠, |$d_{r,\textrm{Ave-rating}}$|⁠, |$d_{r,\textrm{Price-Index}}$| and |$d_{r,\textrm{Date-Added}}$|⁠, which represent the within-zone deciles of a restaurant |$r$| on the respective restaurant characteristics. Standard errors are clustered at the user level. All coefficients multiplied by 100 to express as %. Restaurants that are newer and with fewer ratings (about which users have more uncertainty) are expected to benefit more from the signal. The table suggests these patterns hold. Open in new tab Table 9. Heterogeneity in responsiveness to disclosure across restaurant types Dependent variable: Page visit Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| 00.04% –1.19 |$-$|0.01% |$-$|2.22 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.01% –0.56 –0.004% –1.25 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.06% 2.06 0.02% 3.43 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.01% –0.20 0.006% –1.36 |$d_{r,\textrm{Num-rating}}$| 0.03% 0.88 0.01% 2.26 |$d_{r,\textrm{Date-Added}}$| 0.004% 0.28 0.003% 1.00 |$d_{r,\textrm{Ave-rating}}$| 0.08% 3.35 –0.01% –1.50 |$d_{r,\textrm{Price-Index}}$| 0.10% 4.39 0.00% 0.14 |$\mathcal{I}_{i,B}$| 0.07% 0.24 0.05% 1.33 Intercept |$-$|0.51% |$-$|2.04 –0.03% –0.86 Num. observations 286,651 286,651 Num. individuals (clusters) 202,913 202,913 |$R^{2}$| 0.0012 0.0001 Dependent variable: Page visit Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| 00.04% –1.19 |$-$|0.01% |$-$|2.22 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.01% –0.56 –0.004% –1.25 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.06% 2.06 0.02% 3.43 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.01% –0.20 0.006% –1.36 |$d_{r,\textrm{Num-rating}}$| 0.03% 0.88 0.01% 2.26 |$d_{r,\textrm{Date-Added}}$| 0.004% 0.28 0.003% 1.00 |$d_{r,\textrm{Ave-rating}}$| 0.08% 3.35 –0.01% –1.50 |$d_{r,\textrm{Price-Index}}$| 0.10% 4.39 0.00% 0.14 |$\mathcal{I}_{i,B}$| 0.07% 0.24 0.05% 1.33 Intercept |$-$|0.51% |$-$|2.04 –0.03% –0.86 Num. observations 286,651 286,651 Num. individuals (clusters) 202,913 202,913 |$R^{2}$| 0.0012 0.0001 Notes: The table reports on 202,913 users in groups A and B in the analysis dataset who are exposed to ads for restaurants on which we have restaurant-attribute information. The table reports on the heterogeneity in responsiveness to ad-disclosure by running a regression of visit/call indicators across all relevant users in groups A and B in the analysis dataset, on a dummy for whether the session belonged to group B (⁠|$\mathcal{I}_{i,B}$|⁠), and on interactions of this dummy with |$d_{r,\textrm{Num-rating}}$|⁠, |$d_{r,\textrm{Ave-rating}}$|⁠, |$d_{r,\textrm{Price-Index}}$| and |$d_{r,\textrm{Date-Added}}$|⁠, which represent the within-zone deciles of a restaurant |$r$| on the respective restaurant characteristics. Standard errors are clustered at the user level. All coefficients multiplied by 100 to express as %. Restaurants that are newer and with fewer ratings (about which users have more uncertainty) are expected to benefit more from the signal. The table suggests these patterns hold. Open in new tab 6.4.3. Overall changes in consumer choice under disclosure We now assess whether consumers are made better or worse off under disclosure. A formal analysis of consumer welfare would require taking a stance on a particular representation of utility. Instead, we ask whether consumers are more likely to call better or worse restaurants in the disclosure group, where we judge “better” or “worse” on the basis of the average baseline rating of the called restaurants. To be clear, this assessment is not picked up in the analysis above. While we documented there that users call advertised restaurants at a higher rate in the disclosure group, it could be that these calls are coming at the expense of calls to better, non-advertised restaurants. If so, that suggests users may be worse off under disclosure. To assess these, we compare calls to all restaurants (including those that did not advertise) between individuals in group A and B and examine how ad-disclosure changes users’ call choices. As an added test, if ad-disclosure works as a signal and makes the consumers better informed, we again expect the users in group B to pick options that might appear “risky”, a priori. As before, we use a regression setup for this analysis. Let |$I_{i,B}$| be an indicator of whether user |$i$| is in group B. Let |$d_{r,\textrm{Num-rating}}$|⁠, |$d_{r,\textrm{Ave-rating}}$|⁠, |$d_{r,\textrm{Price-Index}}$| and |$d_{r,\textrm{Date-Added}}$| denote the within-zone deciles of restaurant |$r$| on the respective variables. Represent by |$Y_{ir}$| an indicator for whether user |$i$| calls restaurant |$r$|⁠. Stacking across all user and restaurant combinations in groups A and B, we run the following regression, $$\begin{multline*} Y_{ir}=\alpha_{r}+\beta I_{i,B}+I_{i,B}\times\left[\gamma_{1}d_{r,\textrm{Num-rating}}+\gamma_{2}d_{r,\textrm{Ave-rating}}+\gamma_{3}d_{r,\textrm{Price-Index}}+\gamma_{4}d_{r,\textrm{Date-Added}}\right]+\epsilon_{ir}\quad\end{multline*}$$ Note that we include a fixed effect for each restaurant, therefore, the main effects of the restaurant characteristics are not included. Each row in the regression is a user-restaurant combination, which, with roughly 221K users and 140K restaurants would amount to roughly about 31B observations. To make the regression manageable, we consider 10,834 restaurants which received at least one call from any user in groups A or B. Therefore, we have about 2.4 billion observations (221K users in groups A or B |$\times$| 11K restaurants), and we cluster the standard errors at the user-level, which is the unit of randomization. The estimates from this regression are presented in Table 10. For ease of interpretation, the coefficients are scaled by the average probability of a call in the 2.4B observations (i.e. the coefficients are divided by 9.13|$\times10^{-6}$|⁠). We run separate regressions with and without including |$d_{r,\textrm{Date-Added}}$| because this measure is missing for about a quarter of the observations, and is consequential for the precision of the estimates. Column 1 shows that the coefficient corresponding to |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| is negative and |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-Rating}}$| is positive. The coefficient corresponding to |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| is negative but statistically insignificant.35 When we drop the observations with missing data (column 2), the estimates becomes imprecise, but remain of the same sign. Looking at Table 10, the effect of ad-disclosure decreases by 0.0191 of the average call probability, (a change of 2% to the baseline) if a restaurant moves up by one decile in the number of ratings it receives. Put another way, moving a restaurant from the lowest to the highest decile would produce an increase in call rate that is about 20% of the baseline effect. The effect of average ratings is comparable. Finally, as robustness, Supplementary Appendix H repeats the same regression considering only the user-restaurant pairs in which the restaurant’s city is the same as the user’s city of search, and shows that the qualitative nature of the results remain unchanged. Table 10. Heterogeneity in user choice across all restaurants to ad-disclosure Dependent variable: (1) Call (2) Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| |$-$|0.0191 |$-$|2.05 0.0110 –0.92 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.0187 2.22 0.0141 1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.0139 –1.84 –0.0125 –1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.0008 –0.11 |$\mathcal{I}_{i,B}$| 0.1117 1.86 0.0931 1.08 Intercept 1.00 51.10 1.0011 46.96 Fixed effect for each restaurant Yes Yes Num. Individuals (clusters) 221,742 221,742 Num. Restaurants 10,843 8,266 Num. Observations 2.4B 1.8B |$R^{2}$| 0.00 0.00 Dependent variable: (1) Call (2) Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| |$-$|0.0191 |$-$|2.05 0.0110 –0.92 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.0187 2.22 0.0141 1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.0139 –1.84 –0.0125 –1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.0008 –0.11 |$\mathcal{I}_{i,B}$| 0.1117 1.86 0.0931 1.08 Intercept 1.00 51.10 1.0011 46.96 Fixed effect for each restaurant Yes Yes Num. Individuals (clusters) 221,742 221,742 Num. Restaurants 10,843 8,266 Num. Observations 2.4B 1.8B |$R^{2}$| 0.00 0.00 Notes: Table reports on 221,742 users in groups A and B in the experiment. The table presents how the characteristics of restaurants called by individuals changes due to ad-disclosure by running a regression of call indicators across all combinations of users and restaurants (that got at least one call from any individual in groups A or B), on a dummy for whether the user belonged to group B (⁠|$\mathcal{I}_{i,B}$|⁠), and on interactions of this dummy with |$d_{r,\textrm{Num-rating}}$|⁠, |$d_{r,\textrm{Ave-rating}}$|⁠, |$d_{r,\textrm{Price-Index}}$| and |$d_{r,\textrm{Date-Added}}$|⁠, which represent the within-zone deciles of a restaurant |$r$| on the respective restaurant characteristics. Standard errors are clusters at the user level. Open in new tab Table 10. Heterogeneity in user choice across all restaurants to ad-disclosure Dependent variable: (1) Call (2) Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| |$-$|0.0191 |$-$|2.05 0.0110 –0.92 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.0187 2.22 0.0141 1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.0139 –1.84 –0.0125 –1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.0008 –0.11 |$\mathcal{I}_{i,B}$| 0.1117 1.86 0.0931 1.08 Intercept 1.00 51.10 1.0011 46.96 Fixed effect for each restaurant Yes Yes Num. Individuals (clusters) 221,742 221,742 Num. Restaurants 10,843 8,266 Num. Observations 2.4B 1.8B |$R^{2}$| 0.00 0.00 Dependent variable: (1) Call (2) Call Variable Coeff: |$t$|-stat Coeff: |$t$|-stat |$\mathcal{I}_{i,B}\times d_{r,\textrm{Num-rating}}$| |$-$|0.0191 |$-$|2.05 0.0110 –0.92 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Ave-rating}}$| 0.0187 2.22 0.0141 1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Price-Index}}$| –0.0139 –1.84 –0.0125 –1.48 |$\mathcal{I}_{i,B}\times d_{r,\textrm{Date-Added}}$| –0.0008 –0.11 |$\mathcal{I}_{i,B}$| 0.1117 1.86 0.0931 1.08 Intercept 1.00 51.10 1.0011 46.96 Fixed effect for each restaurant Yes Yes Num. Individuals (clusters) 221,742 221,742 Num. Restaurants 10,843 8,266 Num. Observations 2.4B 1.8B |$R^{2}$| 0.00 0.00 Notes: Table reports on 221,742 users in groups A and B in the experiment. The table presents how the characteristics of restaurants called by individuals changes due to ad-disclosure by running a regression of call indicators across all combinations of users and restaurants (that got at least one call from any individual in groups A or B), on a dummy for whether the user belonged to group B (⁠|$\mathcal{I}_{i,B}$|⁠), and on interactions of this dummy with |$d_{r,\textrm{Num-rating}}$|⁠, |$d_{r,\textrm{Ave-rating}}$|⁠, |$d_{r,\textrm{Price-Index}}$| and |$d_{r,\textrm{Date-Added}}$|⁠, which represent the within-zone deciles of a restaurant |$r$| on the respective restaurant characteristics. Standard errors are clusters at the user level. Open in new tab These estimates suggest that consumers are more likely to call restaurants that have higher ratings and have received fewer ratings in the past, when they are in experimental group B compared to when they are in group A. This suggests that users’ choices are shifting systematically toward options that are better rated but, presumably, perceived as risky without the information conveyed by advertising. 7. Robustness To close the paper, we discuss some statistical considerations pertaining to the main results; discuss some robustness to alternative explanations for the findings; and report on sensitivity analysis. We highlight some of the main analyses here, relegating full details to Supplementary Appendix I. 7.1. Statistical considerations 7.1.1. Robustness to session definition The results presented so far examine the effect of ad disclosure on outcomes in the session in which the ad exposure occurs. A session is defined as a period of continuous activity on the app not separated by more than 3 hours (see Supplementary Appendix F). We discuss robustness to the session definition. For each user-restaurant pair in the analysis dataset, we create outcome variables indicating whether the user visited the restaurant’s page or called the restaurant within 1 day (24 hours) of the exposure; within 2 days; 7 days, and for the maximum duration the data permits. Then, we repeat the main analysis with these new outcome variables, comparing outcome means of users in Treatment A with outcome means for users in Treatment B for varying session-duration definitions. One impact of using longer duration is there are more observed calls/page-visits for each user. Table 11 shows the results. The analysis shows that the main results—differences in mean page-visit and call behaviour between groups—do not vanish over longer horizons. If anything, we see that the difference in page-visits increases and becomes statistically significant.36 Table 11. Checking for sensitivity of main results to expanding the session definition Outcome No disclosure (Treatment A) Disclosure (Treatment B) Difference: B - A Mean SE Mean SE Mean SE |$p$|-value (0/1) Page-visit within the session 0.0110 0.0003 0.0118 0.0002 0.00075 0.00043 0.08 (0/1) Page-visit within 1 day 0.0122 0.0004 0.0130 0.0002 0.00083 0.00046 0.07 (0/1) Page-visit within 2 days 0.0129 0.0004 0.0140 0.0002 0.00101 0.00047 0.03 (0/1) Page-visit within 7 days 0.0154 0.0005 0.0169 0 .0002 0.00146 0.00051 0.005 (0/1) Page-visit until the end of data 0.0268 0.0006 0.0287 0.0003 0.00196 0.00068 0.004 (0/1) Call within the session 0.00031 0.00006 0.00055 0.00004 0.00024 0.000078 0.002 (0/1) Call within 1 day 0.00034 0.00007 0.00066 0.00005 0.00032 0.000083 <0.001 (0/1) Call within 2 days 0.00037 0.00007 0.00071 0.00005 0.00034 0.000086 <0.001 (0/1) Call within 7 days 0.00060 0.00009 0.00092 0.00006 0.00032 0.000106 0.002 (0/1) Call until the end of data 0.00214 0.00017 0.00252 0.00009 0.00038 0.000194 0.053 Outcome No disclosure (Treatment A) Disclosure (Treatment B) Difference: B - A Mean SE Mean SE Mean SE |$p$|-value (0/1) Page-visit within the session 0.0110 0.0003 0.0118 0.0002 0.00075 0.00043 0.08 (0/1) Page-visit within 1 day 0.0122 0.0004 0.0130 0.0002 0.00083 0.00046 0.07 (0/1) Page-visit within 2 days 0.0129 0.0004 0.0140 0.0002 0.00101 0.00047 0.03 (0/1) Page-visit within 7 days 0.0154 0.0005 0.0169 0 .0002 0.00146 0.00051 0.005 (0/1) Page-visit until the end of data 0.0268 0.0006 0.0287 0.0003 0.00196 0.00068 0.004 (0/1) Call within the session 0.00031 0.00006 0.00055 0.00004 0.00024 0.000078 0.002 (0/1) Call within 1 day 0.00034 0.00007 0.00066 0.00005 0.00032 0.000083 <0.001 (0/1) Call within 2 days 0.00037 0.00007 0.00071 0.00005 0.00034 0.000086 <0.001 (0/1) Call within 7 days 0.00060 0.00009 0.00092 0.00006 0.00032 0.000106 0.002 (0/1) Call until the end of data 0.00214 0.00017 0.00252 0.00009 0.00038 0.000194 0.053 Open in new tab Table 11. Checking for sensitivity of main results to expanding the session definition Outcome No disclosure (Treatment A) Disclosure (Treatment B) Difference: B - A Mean SE Mean SE Mean SE |$p$|-value (0/1) Page-visit within the session 0.0110 0.0003 0.0118 0.0002 0.00075 0.00043 0.08 (0/1) Page-visit within 1 day 0.0122 0.0004 0.0130 0.0002 0.00083 0.00046 0.07 (0/1) Page-visit within 2 days 0.0129 0.0004 0.0140 0.0002 0.00101 0.00047 0.03 (0/1) Page-visit within 7 days 0.0154 0.0005 0.0169 0 .0002 0.00146 0.00051 0.005 (0/1) Page-visit until the end of data 0.0268 0.0006 0.0287 0.0003 0.00196 0.00068 0.004 (0/1) Call within the session 0.00031 0.00006 0.00055 0.00004 0.00024 0.000078 0.002 (0/1) Call within 1 day 0.00034 0.00007 0.00066 0.00005 0.00032 0.000083 <0.001 (0/1) Call within 2 days 0.00037 0.00007 0.00071 0.00005 0.00034 0.000086 <0.001 (0/1) Call within 7 days 0.00060 0.00009 0.00092 0.00006 0.00032 0.000106 0.002 (0/1) Call until the end of data 0.00214 0.00017 0.00252 0.00009 0.00038 0.000194 0.053 Outcome No disclosure (Treatment A) Disclosure (Treatment B) Difference: B - A Mean SE Mean SE Mean SE |$p$|-value (0/1) Page-visit within the session 0.0110 0.0003 0.0118 0.0002 0.00075 0.00043 0.08 (0/1) Page-visit within 1 day 0.0122 0.0004 0.0130 0.0002 0.00083 0.00046 0.07 (0/1) Page-visit within 2 days 0.0129 0.0004 0.0140 0.0002 0.00101 0.00047 0.03 (0/1) Page-visit within 7 days 0.0154 0.0005 0.0169 0 .0002 0.00146 0.00051 0.005 (0/1) Page-visit until the end of data 0.0268 0.0006 0.0287 0.0003 0.00196 0.00068 0.004 (0/1) Call within the session 0.00031 0.00006 0.00055 0.00004 0.00024 0.000078 0.002 (0/1) Call within 1 day 0.00034 0.00007 0.00066 0.00005 0.00032 0.000083 <0.001 (0/1) Call within 2 days 0.00037 0.00007 0.00071 0.00005 0.00034 0.000086 <0.001 (0/1) Call within 7 days 0.00060 0.00009 0.00092 0.00006 0.00032 0.000106 0.002 (0/1) Call until the end of data 0.00214 0.00017 0.00252 0.00009 0.00038 0.000194 0.053 Open in new tab 7.1.2. Bootstrapped test statistics We use as the main test statistic the difference in mean call rates between the two groups.37 Though we have a large number of observations in each group and standard results based on the Central Limit Theorem apply, one may worry that the normal approximation to the distribution of the test statistic under the null may be poor given the small number of realized calls in both groups.38 To assess this, we report on a simulation that computes an exact p-value that does not rely on the normal approximation. We simulate the sampling distribution of the test statistic assuming the null that disclosure has no effect is true. To do this, we block-sample at the user-level with replacement two datasets from the no-disclosure group, one with |$n=44,637$| users and the other with |$n=177,105$| users, mirroring the setup. We then take the difference in means of the call indicators between the two datasets, and repeat this procedure 10,000 times to obtain 10,000 such mean differences. We plot the empirical CDF of these in Figure 3, representing the empirical distribution of the test statistic under the null. Denote this empirical CDF as |$\hat{\mathcal{F}}_{d}(d)$|⁠. The observed value of the test statistic in the data is 0.024% (0.055–0.031, see Table 5). This is represented as vertical red lines on the plot. The chance of seeing a value more extreme than 0.024% under the null, |$\hat{\mathcal{F}}_{d}(-0.024)+1-\hat{\mathcal{F}}_{d}(0.024)$|⁠, is |$\tfrac{11}{10000}=0.0011$|⁠. This can be interpreted as an exact p-value based on the empirical CDF. This is similar to the 0.002 value reported in Table 5. Visually inspecting Figure 3 also shows that an observed difference of 0.024% would be highly unusual under the null. Overall, we conclude that there is a robust statistically significant difference in the call probability between the two groups.39 Figure 3 Open in new tabDownload slide Assessing p-values by simulation Notes: The figure shows the sampling distribution of the test statistic assuming the null that disclosure has no effect is true. The observed value of the test statistic in the data is 0.024% (0.055–0.031, see Table 5). This is represented as vertical red lines on the plot. The chance of seeing a value more extreme than 0.024% under the null is |$\frac{11}{10000}=0.0011$|⁠, which can be interpreted as an exact p-value based on the empirical CDF, and not the normal approximation. Figure 3 Open in new tabDownload slide Assessing p-values by simulation Notes: The figure shows the sampling distribution of the test statistic assuming the null that disclosure has no effect is true. The observed value of the test statistic in the data is 0.024% (0.055–0.031, see Table 5). This is represented as vertical red lines on the plot. The chance of seeing a value more extreme than 0.024% under the null is |$\frac{11}{10000}=0.0011$|⁠, which can be interpreted as an exact p-value based on the empirical CDF, and not the normal approximation. Supplementary Appendix I discusses additional statistical considerations. SupplementarySection I.1.1 assess considerations raised in the statistical literature (e.g.Shaffer 2006; Gelman and Carlin 2014) about the risk of obtaining a statistically significant estimate of the wrong sign, as well as the risk of obtaining an overstated statistically significant effect, when a study is possibly under-powered. The simulations there show such risks are low in this study. Further, simulations in Supplementary Section I.1.2 show that our data have statistical power to conclude that the effect of disclosure on page-visits is smaller than the effect on calls. Supplementary Section I.1.3 shows the main results are stable when we cluster standard errors by advertiser restaurant and by search query, rather than by individual. Supplementary Section I.1.4 reports the main results in regression form, including fixed effects for advertiser restaurant and search query. The estimates are seen to stable across the various controls, documenting that the results are not driven by differences in search queries and advertisers across treatment groups. 7.2. Other considerations Supplementary Appendix I also discusses the following sensitivity analyses and alternatives explanations. 7.2.1. Is ad-disclosure simply catching user’s attention? If disclosure was merely catching user attention, we would expect it to have a larger impact on page-visits during the session compared to calls, which is the opposite of what we find. We investigate this matter further in Supplementary Section I.2. We report the effect of experimental variation in the salience of the listing, by comparing the control group and group A—which involves an increase in salience—as well as the comparisons between treatment sub-conditions incorporated into the experimental design to assess this directly. A contrast between the salience effect and the effect of ad disclosure suggests that salience is not the channel by which disclosure operates in this setting. 7.2.2. Do calls change without a change in demand? Supplementary Section I.3 reports on robustness to using calls as a proxy for demand by showing the effect of ad disclosure exists in the subset of cases where the search category chosen by the consumer is “home-delivery”. In this case, we know for sure the user is looking to order food for delivery, wherein a call is likely to match closely with demand. The main patterns continue to hold. 7.2.3. Are the effects driven by adverse user reactions to no-disclosure? Supplementary Section I.4 reports whether there is support for alternative phenomena that explain the difference in outcomes between groups A and B by positing negative user reactions to group A. For instance, one possibility is that users experience some “cognitive dissonance” (Festinger 1957) in group A because the listing shown in the ad-slot (i.e. the advertiser) is not a great match to their search and tastes. This may reduce their trust in the search-engine, and cause them to respond unfavourably to listings in group A. The evidence suggests this is not the main phenomenon driving the results. 7.2.4. Are results affected by cases in which the advertiser is also shown as an organic listing? In Treatment A in the experiment, the advertiser could appear twice in the organic listings (once as an non-disclosed ad, and the other as a regular organic listing). To assess this, first, in Supplementary Section I.5, we document that this aspect is not “unusual”—popular platforms repeat domains in organic listings for many reasons, seemingly unrelated to the quality of the product. Further, to assess whether our results are sensitive to this, in Supplementary Section I.5 we report the main results after dropping observations in which the advertiser is presented in the search listing page as an organic listing (5.2% overall), and find they remain robust. 7.3. Boundary conditions and generalizability Our model in Section 5 specifies conditions under which our effects are likely to occur. Other things held equal, the signalling value of advertising is likely to be high when consumer uncertainty about product quality is high; when repeat-purchase is more likely.40 Accordingly, these factors are likely to moderate the size of the signalling role of advertising. In the paid-search context, consumer uncertainty about the appeal of the product is dependent on the quality of the search-engine’s recommendations and what the consumer can learn from searching. In a world where the search-engine is able to communicate all relevant information to the consumer without advertising, there will be no remaining experience component of quality, and the signalling role of advertising will be mitigated. Finally, the credibility of the ad-signal is higher when the cost of delivering an ad-impression is high. Obtaining an ad-slot in response to a localized directed search on a popular search engine like Zomato is costly. This cost is ultimately driven by the nature of competition in the ad-market and the contractual mechanism by which ads are sold on the platform. Accordingly, as these factors change, the magnitude of the measured signalling effect will also vary. 8. Conclusions A field experiment to assess the signalling role of advertising is presented. The experiment randomizes users into treatment and control groups that enable us to measure the causal effect of disclosure to a consumer that the firm has advertised, separately from the content of the search ads served by the platform to the user, thereby pinning down signalling using a “demand-side” strategy. The effect of disclosure on conversion to the advertised restaurants is found to be economically meaningful. Separately from testing for signalling effects, this finding also holds importance for platform design and monetization for search platforms because it implies that disclosure of the sponsored nature of advertising is beneficial. Recent reports from online publishers that clearly disclosing the sponsorship status of non-annoying ads improves outcomes, is consistent with this finding (e.g.Moses 2016).41 This draft: Sept 19, 2019* The editor in charge of this paper was Aureo de Paula. Acknowledgments The views discussed here represent that of the authors and not of Stanford University or Zomato. We thank Ron Berman, Daisy Dai, Dafna Goor, Wes Hartmann, Randall Lewis, Puneet Manchanda, Sridhar Moorthy, Thomas Otter, John Roberts, Imran Rasul, Devesh Raval, Catherine Tucker, Raluca Ursu, Kosuke Uteke, Joel Waldfogel, Birger Wernerfelt; and three anonymous reviewers, Dan Ackerberg, the editor Aureo de Paula, Matt Gentzkow, and Sanjog Misra in particular for helpful comments and suggestions. Specific thanks to Pedro Gardete, Kinshuk Jerath, Mike Ostrovsky and Jiwoong Shin for comments on the theory model in the paper. Comments from participants at the 2015 Marketing Science, 2017 NBER IO, 2017 IIOC, 2016 SICS, QME conferences; seminars at FCC, FTC, Goethe-Univ. Frankfurt, HBS, HEC-Paris, MIT-Sloan, NYU, Santa Clara Univ, Stanford, Univ. of Colorado, Univ. of Mannheim, and Univ. of Michigan are also appreciated. We thank Shuo Xie and Charles Zhang for their excellent research assistance. The usual disclaimer applies. Please contact Sahni (navdeep.sahni@stanford.edu) or Nair (harikesh.nair@stanford.edu) for correspondence. The authors declare they have no commercial relationship with Zomato, and the research is conducted with no restrictions on the publishability of the findings from the field experiment. Supplementary Data Supplementary data are available at Review of Economic Studies online. Footnotes * |$^{*}$| Previous drafts: Feb 1, June 3, Dec 29, 2016; March 6, 2018; Nov 23, 2018; July 5, Aug 3 2019. 1. Ackerberg ’s (2001) analysis outlines how one may distinguish between informative effects of advertising (i.e. affecting consumer beliefs) versus persuasive effects (i.e. affecting consumer tastes) using such consumer-level data. He proposes examining whether experienced consumers respond to advertising (which is consistent with persuasive effects). However, reflecting the issues outlined above, Ackerberg does not distinguish between the direct channels (e.g. information contained in ads about existence, location, function, or price) versus the indirect channels (i.e. the information contained in the fact that the firm is willing to spend on advertising) of informative advertising. Related consumer-level studies include Anand and Shachar (2011), Goeree (2008), and Terui et al. (2011) who estimate models where advertising has an informative role by affecting consumer’s information sets; Shum (2004) and Erdem et al. (2008) who study the impact of advertising on consumer price sensitivity; and Homer (1995) and Kirmani (1990, 1997) who report on relationships between consumer perceptions of advertising and quality in a lab setting. Please see Bagwell (2007) for more discussion. 2. The call action we observe occurs after a consumer visits the restaurant’s page. As detailed later in the article, calls represent a consumer’s intent to order food from a restaurant. 3. We do this to avoid bias from within-user feedback effects that may be problematic in longitudinal analysis (described in more detail later in the article). 4. Perfectly matching consumers to restaurants is unlikely in our setting because, as we note in Section 4, the experiment serves ads for “broad searches” (e.g. “restaurants in New Delhi”), in which consumers do not specify narrow search criteria in their queries. 5. From TechCrunch (2015): “Zomato started in 2008 as a supercharged portal for restaurant search that went beyond basic names and addresses. Zomato staff would visit venues, collecting menus, and photos that would be scanned and input into Zomato’s larger database (think Google Maps’ roving cars but for restaurants) which in turn would be used to power searches not only for certain restaurants but places where consumers could go for very specific dishes, for example. This filled a niche: smaller and independent venues are not always up to date with their online presence (many don’t even have websites today) and this provided a way to find them on the web. It also helped differentiate Zomato from the likes of Yelp and others that looped in crowdsourced information, which can be hard to verify as not being biased and more generally keep up to date.” 6. We obtain access to a snapshot of the restaurant profile database at baseline. Out of 210,302 restaurants in the database, we retained 142,934 restaurants that had at least one rating provided. Out of these, 15,976 were missing the date added, 31 missing the number of total ratings, and 3,522 missing the price index. 7. Restaurants are added in Zomato’s database by a field team that regularly surveys the market and updates the database. Therefore, within a market, days since a restaurant is added is likely to be correlated with when the restaurant started, albeit noisily. 8. In the data, we observe a chain id corresponding to each restaurant id. We say a restaurant is a part of a chain if it has at least one other restaurant with the same chain id. 9. Note that column (1) uses as a covariate “days since the restaurant was added,” which is the opposite of the covariate “decile of the date when the restaurant is added” used in column (2), so we see opposite signed coefficients, implying the same directional effect. 10. To be clear, here’s an example. Suppose a restaurant |$r$| has “Zone decile:Avg rating” of 10. This implies that if we compare zones based on average restaurant ratings, |$r$|’s zone is within the top 10% in its city. We chose to create “within-city” deciles that pick a zone’s rank relative to other zones within its city. This helps differentiate say, costly areas within a city from less costly ones in that city. This is not a consequential choice; inferences from the regression below remain the same if we create deciles across all zones. 11. As Milgrom and Roberts (1986) note, whether or not high quality firms use high prices and high advertising to signal quality depend on the relative marginal costs between high and low quality firms. When marginal costs increase with quality, equilibria exist where the high quality firms set high prices and uses high advertising outlays to signal its quality. To see this, assume that there are two firms of high and low quality, such that the high quality firm has higher marginal costs. Assume that advertising is dissipative, and that the higher quality firm obtains more repeat business once consumers visit the firm and experience the good. From a position of equal initial prices, assume that the high quality firm increases its prices a bit. Then, the number of current consumers it loses should be the same as that of the low quality firm, but the “pain” of this current loss should be lower for the high quality firm as its margin per lost consumer is lower. However, if these lost consumers had actually visited the firm, they would have revisited and repurchased at a higher rate for the high quality firm, because their visit reveals its quality is higher. So, the “pain” associated with lost repeats is likely higher for the high quality firm. Since the price increase effects on current and repeats go in opposite directions, its possible that the high quality firm cannot use high-prices alone to signal quality, and will also need to use advertising to signal quality. In the possible equilibria in this scenario, the high quality firm sets its prices above its full-information level and chooses a high level of advertising to separate from the low quality firm. If on the other hand, marginal costs decrease with quality, low prices will signal high quality in this model, and there is no need to use advertising as a signal. 12. Thanks to Matt Gentzkow for suggesting this test. 13. Casual empiricism suggests that the best restaurants do not advertise heavily. There is also some evidence in the data of non-monotonicity in the probability of advertising at the very high end of the ratings. See Supplementary Appendix A. How do we reconcile that with signalling? What is relevant to signalling is the advertising behaviour of those restaurants when they were new to the market and consumers did not know they were good. Our data suggest that better restaurants indeed tend to advertise more on the platform when they are new; if we include an interaction between rating and age in the regressions reported in Table 1, the interaction term’s coefficient is positive, and the rating’s coefficient turns negative, indicating new and higher rated restaurants are more likely to advertise. Other theory has pointed out that “counter-signalling” equilibria that obtains with multidimensional signals and large heterogeneity across firms can also explain this phenomena (Orzach et al. 2002). In these equilibria, even noisy signals are sufficient to separate the “best” firms from the “low” quality firms. The “medium” quality firms then use advertising to separate themselves from the “low” quality firms, while the “best” firms avoid advertising to separate from those with “medium” quality. Note that even in these models, advertising works as a signalling device, serving to separate firms of differing quality, which is what we are testing here. 14. These comprise large cities in what Zomato refers to as “Full-stack” regions. The firm defines these as “(a) large markets (b) growing very fast (c) [where] Zomato is the strongest player in its space.” See http://blog.zomato.com/post/131277554406/shifting-focus-to-what-matters-and-what-works. 15. For an announcement of the introduction of the mobile ads, see “Zomato Introduces Ads On The Mobile App; Doubles Consumer Reach For Merchants,” 26 November 2014, http://www.thestartupjournal.com/zomato-mobile-app-ads. 16. The yellow label with the white text denoting “Ad” is chosen to match the disclosure practice of popular search platforms like Google and Yelp. Within group B, we also sub-randomize users into alternative ways of ad-disclosure used in the industry, so as to explore robustness and alternative mechanisms. The experimental group and the sub-conditions remain the same for a user across the time period of the experiment (there is no re-randomization over time). This article focuses on the contrast in outcomes between disclosure versus no-disclosure and aggregates these sub-conditions. A companion paper (Sahni and Nair 2018) explores these alternatives in more detail, and reports no statistically significant difference in outcomes across these sub-conditions. 17. Advertisers also cannot track if calls to the restaurant originate from the Zomato website or from the app. 18. Search listings that appear when a user clicks the “Nearby” tab on the Zomato app are also excluded from the experiment. We did not place any ads for such searches in our experiment. The rationale follows. In general, when a user clicks on the “Nearby” tab, Zomato shows him restaurant listings sorted by their distance from the user’s actual location (obtained via GPS). Placing an experimental ad in such a list may seem odd, especially in treatment A (no ad disclosure), if the advertiser is not located closer to the user compared to the restaurant listed below the ad. By excluding these searches, our experiment avoids situations that can be potentially confusing for the consumer. 19. Advertisers rarely appear in the organic results on the same search results page as their ads. An advertiser and its organic listing appears on the first page together in 5.2% of all searches in our data. Due to randomization, this number is on average the same across the three groups (p = 0.50). The findings in the paper are not sensitive to omitting searches in which an advertiser appears twice in groups A and B. See Section 7. 20. The likelihood that the need is satisfied is |$\mathbb{P}(s_{j}+e_{j}>0)$| = |$\mathbb{P}(\bar{s}_{j}+v_{j}+e_{j}>0)$| = |$\mathbb{P}(v_{j}>-\bar{s}_{j}-e_{j})$| =1- |$\mathcal{F}\left(-e_{j}-\bar{s}_{j}\right)$|= |$\mathcal{F}\left(e_{j}+\bar{s}_{j}\right)$|⁠. The last step follows because of the symmetry of the cdf |$\mathcal{F}$|⁠. 21. One could interpret this as following. The consumer’s taste for the search attribute is drawn from a distribution. For instance, even if |$j$| is the preferred product on average, i.e., has a higher |$\bar{s}_{j}+e_{j}$|⁠, there is a chance the consumer will not be satisfied by it if he gets a bad enough draw from the distribution. This “heterogeneity” causes him to potentially visit multiple product pages on the search engine. 22. We assume |$1>t>\frac{1}{2}$| to simplify the workings of the model below. Relaxing this assumption does not change the main implications (see Supplementary Appendix B). Other parameter restrictions are also explained in the Supplementary Appendix. 23. Firms are unaware of treatment A, and advertise believing the ad will be shown as in treatment B. 24. This parallels our experimental scenario. Consumers randomized into treatment A believe there is no advertising on the app. 25. |$\mathbb{E}\left[e\right]=\frac{\left(H_{E}+L_{E}\right)}{2}$|⁠, and |$\textrm{var}\left[e\right]=\frac{1}{2}\times\left(H_{E}-\frac{\left(H_{E}+L_{E}\right)}{2}\right)^{2}+\frac{1}{2}\times\left(L_{E}-\frac{\left(H_{E}+L_{E}\right)}{2}\right)^{2}$|= |$\frac{1}{4}(H_{E}-L_{E})^{2}.$| 26. Signalling theory does not have a clear prediction for other information acquisition or search-related outcome measures, such as browsing through reviews. Such outcomes may decrease or increase because of ad disclosure. For a more elaborate discussion, see Supplementary Appendix D. 27. The fact that orders comprise a large proportion of the calls is not surprising given that online ordering was not available on Zomato until 2015. Also, there is no well established technology-enabled facilitator of reservations (like OpenTable in the US) in these markets. Quoting an official announcement by Zomato, “A few months ago, we made an important move in India – launching our online ordering service. And we’ve really been kicking our competition’s ass in this business with less than 0.1% of their marketing budgets. Why we were able to win? Because we have millions of users who already use us for ordering food over the phone. Now, they have started placing the same orders online using our app. And there is a lot of growth still left; 92% of our users who use Zomato to search for restaurants that deliver haven’t even started ordering online on Zomato as yet. Our ticket sizes are more than double our competitors’—because our users are not using us for the discounts. They are using us for the convenience, and a product they already love Zomato for. Our users hold tremendous potential for transaction-based businesses. Getting into transactions was always the natural next step for our business. Online ordering is a natural and logical alternative for our users who, up until now, used to call restaurants to place their orders for delivery. Table reservations fit into Zomato as easily as online ordering did. The time has come for us to focus deeply on transactions in countries where it matters.” See, http://blog.zomato.com/post/131277554406/shifting-focus-to-what-matters-and-what-works, dated 16 October 2015. 28. For any search, depending on the day when the search happens and the filters applied, we know whether any ads would be served, and which restaurants would be advertised. 29. The design intended to assigned individuals to Control, Treatment A, Treatment B with ratio 1:1:4. To be exact, the randomization code assigned probabilities across the 6 groups as 16.67:16.67:16.67:16.67:16.67:16.67:16.65(=100-5|$\times$|16.67). The last group got a slightly lower weightage. On checking the actual assignments to the different treatment groups we can see that these ratios realize in the sample. We are unable to reject that proportion of individuals in Control = 1/6 (p-value = 0.62); proportion of individuals in Treatment A = 1/6 (p-value = 0.11); proportion of individuals in Treatment B = 4/6 (p-value = 0.38). 30. In absolute terms, we observe 23 calls in treatment A, and 161 in treatment B. Sensitivity analysis reported in Section 7 repeats these analyses including calls that occur after the first session (and not just during that session) in the dependent variable. The results are found to be robust. If we use as the dependent variable all calls a user made after the first ad-exposure over the entire duration of the data, the corresponding effect of disclosure is a 17.7% increase in B relative to A (from 0.214% to 0.252%: see last row, Table 11). 31. We can reject the null-hypothesis that disclosure decreases page visits using a single tailed-test (p-value = 0.04). 32. Specifically, we use the data from before the individual conducts a search for which experimental ads are shown. Because of limited availability of past data, we are able to do this for 83,245 (of the 265,975) users in the experiment. 33. Out of the 5,031 users, 860 are in the control group, 843 in A and 3,328 in B. We observe 118 visits across groups that facilitates a test of differences in rates on this variable. We observe only 4 calls totally, all in treatment B. 34. In other extensions not reported (available on request), we can also allow in the theory model for a proportion |$p$| of the population that has more uncertainty about the experience attribute signaled by advertising. This model shows that the effect of disclosure of ads (treatment B |$-$| treatment A) increases as |$p$| increases. The subpopulation of out-of-city users is expected to have a higher |$p$| than the remaining sample. Therefore, the effect of disclosure is also expected to be larger for this sample: more people change their beliefs about the advertiser’s likelihood of satisfying their need when advertising is disclosed. 35. Note that the numbers in the table are very small in magnitude compared to the numbers in the previous tables because of the regression setup. This regression considers all combinations of users and restaurants (not just advertisers), including restaurants that (1) may be less popular than the advertisers as noted in Section 3.2, and (2) less likely to be relevant to the user compared to the advertiser whose ad is shown to the user. Both these reasons decrease the baseline propensity of a user calling a restaurant. 36. The following behaviour could explain this. In the session with exposure, the consumer recognizes the signal but does not visit the advertiser’s page due to idiosyncratic reasons, for example, if he is satiated with the advertiser’s cuisine at that time. At a subsequent purchase occasion he remembers the inference from the signal, and acts on it. “Carry-over” effects of advertising of this nature are documented in Sahni (2015) in a similar setting. 37. With binary outcomes and independent samples, the |$t$| and |$\chi^{2}$| tests are equivalent, and yield the same p-value. 38. The binomial distribution with parameters |$\left(N,p\right)$| is skewed when the success probability, |$p$| is close to 0 or 1, and the normal approximation may be poor unless the number of trials, |$N$|⁠, is very large (Blyth and Still 1983; Samuels and Lu 1992). 39. The two-tailed p-value of the test is also low enough to alleviate a concern related to multiple hypothesis testing. For example, above, we conducted two tests comparing the disclosure and no-disclosure group. Adjusting the p-value for multiple hypothesis testing using a conservative Bonferroni correction yields p = 0.004 (0.002|$\times$|2). 40. As an empirical test of this, we tried to see whether we could isolate a set of restaurants in the data that were situated closer to tourist spots and less likely to have repeat customers. Unfortunately, we were unable to pin down enough restaurants to reliably run this test. 41. The article reports that Slate “found that on its more explicitly labelled ads, the click-through rates were three times higher than the previous units (though Slate wouldn’t disclose the CTR). The publisher also contends that average time spent on the new units doubled, to 4 minutes, 15 seconds.” REFERENCES ACKERBERG, D. A. ( 2001 ), “ Empirically Distinguishing Informative and Prestige Effects of Advertising ”, RAND Journal of Economics 32 , 316 – 333 . Google Scholar Crossref Search ADS WorldCat ANAND, B. and SHACHAR, R. ( 2011 ), “ Advertising, the Matchmaker, ” RAND Journal of Economics , 42 , 205 – 245 . Google Scholar Crossref Search ADS WorldCat ARCHIBALD, R. B. , HAULMAN, C. H. and MOODY, Jr. C. E. ( 1983 ), “ Quality, Price, Advertising and Published Quality Ratings, ” Journal of Consumer Research , 9 , 347 – 356 . Google Scholar Crossref Search ADS WorldCat ATHEY, S. and ELLISON, G. ( 2011 ), “ Position Auctions with Consumer Search .” Quarterly Journal of Economics , 126 , 1213 – 1270 . Google Scholar Crossref Search ADS WorldCat BACKUS, M. , BLAKE, T. and TADELIS, S. ( 2019 ), “ On the Empirical Content of Cheap-Talk Signaling: An Application to Bargaining, ” Journal of Political Economy , 127 , 1599 – 1628 . Google Scholar Crossref Search ADS WorldCat BAGWELL, K. ( 2007 ), “ The Economic Analysis of Advertising ” in Armstrong, M. and Porter, R. (eds) Handbook of Industrial Organization , Vol. 3 ( North Holland : Elsevier ). Google Preview WorldCat COPAC BLAKE, T. , NOSKO, C. and TADELIS, S. ( 2015 ), “ Consumer Heterogeneity and Paid Search Effectiveness: A Large Scale Field Experiment, ” Econometrica , 83 , 155 – 174 . Google Scholar Crossref Search ADS WorldCat BLYTH, C. R. and STILL, H. A. ( 1983 ), “ Binomial Confidence Intervals, ” Journal of the American Statistical Association , 78 , 108 – 16 . Google Scholar Crossref Search ADS WorldCat CAVES, R. E. and GREENE, D. P. ( 1996 ), “ Brands’ Quality Levels, Prices and Advertising Outlays: Empirical Evidence on Signals and Information Costs, ” International Journal of Industrial Organization , 14 , 29 – 52 . Google Scholar Crossref Search ADS WorldCat CHAN, T. Y. , WU, C. and XIE, Y. ( 2011 ), “ Measuring the Lifetime Value of Customers Acquired from Google Search Advertising, ” Marketing Science , 30 , 837 – 850 . Google Scholar Crossref Search ADS WorldCat ERDEM, T. , KEANE, M. and SUN, B. ( 2008 ), “ Advertising and Consumer Price Sensitivity in Experience Goods Markets, ” Quantitative Marketing and Economics , 6 , 139 – 176 . Google Scholar Crossref Search ADS WorldCat FESTINGER, L. ( 1957 ), A Theory of Cognitive Dissonance ( California : Stanford University Press ). Google Preview WorldCat COPAC GELMAN, A. and CARLIN, J. ( 2014 ), “ Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors, ” Perspectives on Psychological Science , 9 , 641 – 651 . Google Scholar Crossref Search ADS PubMed WorldCat GOLDFARB, A. and TUCKER, C. ( 2011 ), “ Search Engine Advertising: Channel Substitution when Pricing Ads to Context, ” Management Science , 57 , 458 – 470 . Google Scholar Crossref Search ADS WorldCat GOEREE, M. S. ( 2008 ), “ Limited Information and Advertising in the US Personal Computer Industry, ” Econometrica , 76 , 1017 – 1074 . Google Scholar Crossref Search ADS WorldCat HERTZENDORF, M. ( 1993 ), “ I’m Not a High-Quality Firm - But I Play One on TV, ” Rand Journal of Economics , 24 , 236 – 47 . Google Scholar Crossref Search ADS WorldCat HOLLAND, P. W. ( 1986 ), “ Statistics and Causal Inference, ” Journal of the American Statistical Association , 81 , 945 – 960 . Google Scholar Crossref Search ADS WorldCat HOMER, P. M. ( 1995 ), “ Ad Size as an Indicator of Perceived Advertising Costs and Effort: The Effects on Memory and Perceptions, ” Journal of Advertising , 24 , 1 – 12 . Google Scholar Crossref Search ADS WorldCat HORSTMANN, I. J. and MACDONALD, G. M. ( 1994 ), “ When Is Advertising a Signal of Quality?, ” Journal of Economics and Management Strategy , 3 , 561 – 584 . Google Scholar Crossref Search ADS WorldCat HORSTMANN, I. J. and MACDONALD, G. M. ( 2003 ), “ Is Advertising A Signal of Product Quality? Evidence from the Compact Disc Player Market, 1983–92, ” International Journal of Industrial Organization , 21 , 317 – 345 . Google Scholar Crossref Search ADS WorldCat HORSTMANN, I. J. and MOORTHY, S. ( 2003 ), “ Advertising Spending and Quality for Services: The Role of Capacity, ” Quantitative Marketing and Economics , 1 , 337 – 365 . Google Scholar Crossref Search ADS WorldCat JOHNSON, G. ( 2013 ), “ The Impact of Privacy Policy on the Auction Market for Online Display Advertising ” ( Working Paper, Boston University ). WorldCat KAWAI, K. , ONISHI, K. and UTEKE, K. ( 2014 ), “ Signaling in Online Credit Markets ” ( Working Paper, Yale University ). WorldCat KIHLSTROM, R. E. and RIORDAN, M. H. ( 1984 ), “ Advertising as a Signal, ” Journal of Political Economy , 92 , 427 – 450 . Google Scholar Crossref Search ADS WorldCat KIRMANI, A. ( 1990 ), “ The Effect of Perceived Advertising Costs on Brand Perceptions, ” Journal of Consumer Research , 17 , 160 – 71 . Google Scholar Crossref Search ADS WorldCat KIRMANI, A. ( 1997 ), “ Advertising Repitition as a Signal of Quality: If It’s Advertised So Often, Something Must Be Wrong, ” Journal of Advertising , 26 , 77 – 86 . Google Scholar Crossref Search ADS WorldCat KIRMANI, A. and RAO, A. R. ( 2000 ), “ No Pain, No Gain: A Critical Review of the Literature on Signaling Unobservable Product Quality, ” Journal of Marketing , 64 , 66 – 79 . Google Scholar Crossref Search ADS WorldCat KWOKA, J. E. Jr. ( 1984 ), “ Advertising and the Price and Quality of Optometric Services, ” American Economic Review , 74 , 211 – 216 . WorldCat LEWIS, R. and REILEY, D. ( 2014 ), “ Online Ads and Offline Sales: Measuring the Effects of Retail Advertising via a Controlled Experiment on Yahoo!, ” Quantitative Marketing and Economics , 12 , 235 – 266 . Google Scholar Crossref Search ADS WorldCat MOSES, L. ( 2016 ), “ Slate Now Relies on Native Ads for Nearly 50 Percent of its Revenue ” http://digiday.com/publishers/slate-now-relies-native-ads-nearly-50-percent-revenue/, accessed 14 December 2016 . WorldCat MANCHANDA, P. , DUBE, J. P. , GOH, K. Y. et al. ( 2006 ), “ The Effect of Banner Advertising on Internet Purchasing, ” Journal of Marketing Research , XLIII , 98 – 108 . Google Scholar Crossref Search ADS WorldCat MILGROM, P. and ROBERTS, J. ( 1986 ), “ Price and Advertising Signals of Product Quality, ” Journal of Political Economy , 94 , 796 – 821 . Google Scholar Crossref Search ADS WorldCat NARAYANAN, S. and KALYANAM, K. ( 2015 ), “ Position Effects in Search Advertising and their Moderators: A Regression Discontinuity Approach, ” Marketing Science , 34 , 388 – 407 . Google Scholar Crossref Search ADS WorldCat NELSON, P. ( 1970 ), “ Information and Consumer Behavior, ” Journal of Political Economy , 78 , 311 – 29 . Google Scholar Crossref Search ADS WorldCat NELSON, P. ( 1974 ), “ Advertising as Information, ” Journal of Political Economy , 82 , 729 – 54 . Google Scholar Crossref Search ADS WorldCat ORZACH, R. , OVERGAARD, P. B. and TAUMAN, Y. ( 2002 ), “ Modest Advertising Signals Strength, ” Rand Journal of Economics , 33 , 340 – 358 . Google Scholar Crossref Search ADS WorldCat RUTZ, O. J and BUCKLIN, R. E. ( 2011 ), “ From Generic to Branded: A Model of Spillover in Paid Search Advertising, ” Journal of Marketing Research , 48 , 87 – 102 . Google Scholar Crossref Search ADS WorldCat SAHNI, N. ( 2015 ), “ Effect of Temporal Spacing between Advertising Exposures: Evidence from Online Field Experiments, ” Quantitative Marketing and Economics , 13 , 203 – 247 . Google Scholar Crossref Search ADS WorldCat SAHNI, N. and NAIR, H. ( 2018 ), “ Native Advertising, Sponsorship Disclosure and Consumer Deception: Evidence from Mobile Search-Ad Experiments , ” Marketing Science , doi: . WorldCat Crossref SAMUELS, N. L. and LU, T.-F. ( 1992 ), “ Sample Size Requirements for the Back-of-the-Envelope Binomial Confidence Interval, ” The American Statistician , 46 , 228 – 231 . WorldCat SHAFFER, J. P. ( 2006 ), “ Recent Developments Towards Optimality in Multiple Hypothesis Testing, ” IMS Lecture Notes Monograph Series , 49 , 16 – 32 . Google Scholar Crossref Search ADS WorldCat SHUM, M. ( 2004 ), “ Does Advertising Overcome Brand Loyalty? Evidence from the Breakfast-Cereals Markets, ” Journal of Economics and Management Strategy , 13 , 241 – 71 . Google Scholar Crossref Search ADS WorldCat TAYLOR, G. ( 2013 ), “ Search Quality and Revenue Cannibalization by Competing Search Engines, ” Journal of Economics & Management Strategy , 22 , 445 – 467 . Google Scholar Crossref Search ADS WorldCat TECHCRUNCH ( 2015 ), “ Restaurant Discovery Site Zomato Eats Up $60M To Fuel Expansion Into New Verticals ” http://techcrunch.com/2015/09/07/restaurant-discovery-site-zomato-eats-up-60m-to-fuel-expansion-into-new-verticals/, accessed 8 December 2015 . TELLIS, G. and FORNELL, C. ( 1988 ), “ Advertising and Quality Over the Product Life Cycle: A Contingency Theory, ” Journal of Marketing Research , 15 , 64 – 71 . Google Scholar Crossref Search ADS WorldCat TERUI, N. , BAN, M. and ALLENBY, G. M. ( 2011 ), “ The Effect of Media Advertising on Brand Consideration and Choice, ” Marketing Science , 30 , pp. 74 – 91 . Google Scholar Crossref Search ADS WorldCat THOMAS, L. , SHANE, S. and WEIGELT K. ( 1998 ), “ An Empirical Examination of Advertising as a Signal of Product Quality, ” Journal of Economic Behavior & Organization , 37 , 415 – 430 . Google Scholar Crossref Search ADS WorldCat TYLER, J. H. , MURNANE, R. J. and WILLETT, J. B. ( 2000 ), “ Estimating the Labor Market Signaling Value of the GED, ” Quarterly Journal of Economics , 115 , 431 – 468 . Google Scholar Crossref Search ADS WorldCat URSU, R. ( 2018 ), “ The Power of Rankings: Quantifying the Effect of Rankings on Online Consumer Search and Purchase Decisions, ” Marketing Science , 37 , 530 – 552 . Google Scholar Crossref Search ADS WorldCat WEISS, A. ( 1995 ), “ Human Capital versus Signaling Explanations of Wages, ” Journal of Economic Perspectives , 9 , 133 – 54 . Google Scholar Crossref Search ADS WorldCat YAO, S. and MELA, C. F. ( 2011 ), “ A Dynamic Model of Sponsored Search Advertising, ” Marketing Science , 30 , 447 – 468 . Google Scholar Crossref Search ADS WorldCat YANG, S. and GHOSE, A. ( 2010 ), “ Analyzing the Relationship Between Organic and Sponsored Search Advertising: Positive, Negative, or Zero Interdependence? ” Marketing Science , 29 , 602 – 623 . Google Scholar Crossref Search ADS WorldCat YELP.com ( 2015 ), “ Why Yelp? Yelp for Business Owners ” https://biz.yelp.com/support/what_is_yelp, accessed 8 December 2015 . YELP FACT SHEET ( 2015 ), “ An Introduction to Yelp Metrics as of September 30, 2015 ” http://www.yelp.com/factsheet, accessed 8 December 2015 . WorldCat ZHANG, J. and LIU, P. ( 2012 ), “ Rational Herding in Microloan Markets, ” Management Science , 58 , 892 – 912 . Google Scholar Crossref Search ADS WorldCat © The Author(s) 2019. Published by Oxford University Press on behalf of The Review of Economic Studies Limited. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - Does Advertising Serve as a Signal? Evidence from a Field Experiment in Mobile Search
JF - The Review of Economic Studies
DO - 10.1093/restud/rdz053
DA - 2020-05-01
UR - https://www.deepdyve.com/lp/oxford-university-press/does-advertising-serve-as-a-signal-evidence-from-a-field-experiment-in-IVU5Wxw5U7
SP - 1
VL - Advance Article
IS - 
DP - DeepDyve
ER -