TY - JOUR
AU - Komakech,, Polycarp
AB - Abstract Governments around the world are investing in technologies that allow citizens to participate in the coproduction of public services by providing monitoring and feedback, but there is little evidence about how these initiatives affect the quality of public services. We implemented a large-scale field experiment that involved organizing 50 citizen reporters in each of 100 neighborhoods across Kampala, Uganda, to provide weekly reports to the municipal government about the delivery of solid waste services via an SMS-messaging platform, resulting in 23,856 reports during the 9-month study period. Citizen reporting did not reduce informal waste accumulation as targeted, which would indicate improvements to formal services. Using our observations as participants in the development and deployment of the reporting platform and interviews with staff at the government agency receiving the citizen reports, we show how the public generated inconsistent information that did not fit existing decision-making processes. We generalize lessons from this field experiment by explaining how coproduction involving information sharing through information and communication technologies is likely to affect public services based on the alignment of citizen-produced data with the information problems managers face; the search costs of detecting public services failures; the quality of citizen-produced data; and the operating costs of citizen-reporting platforms. Introduction Governments around the world are building or adopting platforms to collect and process information from citizens about the delivery of public services. Relying on citizens who directly experience public services to report on service delivery promises to lower the costs and increase the coverage of information available for the management of services (Clark, Zingale, and Logan 2017; Grossman, Humphreys, and Sacramone-Lutz 2014; Linders 2012; Noveck 2015, 2017). By gaining better information on the delivery of public services, agencies can hold frontline providers accountable, optimize delivery efforts, or plan for improvements to services. Furthermore, by opening channels of communication and information sharing with citizens, governments can increase responsiveness and generate goodwill, cooperation, and trust by citizens (Bertot, Jaeger, and Grimes 2010; Buntaine, Nielson, and Skaggs 2019; Jo and Nabatchi 2019; O’Brien 2018). Despite substantial optimism, evidence about the effects of citizen reporting on service outcomes is limited. More generally, evidence about the empirical relationship between coproduction—the voluntary contribution by members of the public to the process of producing services with public agents (see Nabatchi, Sancino, and Sicilia 2017)—and effective service delivery is limited. Loeffler and Bovaird (2016, 1013) conclude that “the actual and potential impact of coproduction on citizen outcomes is as yet only sketchily researched.” Nabatchi, Sancino, and Sicilia (2017, 766) conclude in their recent review of the coproduction literature that “the evidence base for coproduction is relatively weak.” On the ability of information and communication technologies (ICTs) to foster the successful coproduction of public services, Lember, Brandsen, and Tonurist (2019, 2) conclude that “empirical evidence on the effects of new technologies in this area is still scarce, at best...Systematic empirical evidence is still very hard to come by.” The lack of evidence limits the development of theory about when different types of coproduction are likely to improve the delivery of public services. Moving from general optimism about citizen reporting to evidence-based practice is important because although data from citizens can be collected more broadly and in a more timely manner than many top-down approaches to monitoring public services, the adoption and operation of reporting platforms can be difficult (Gil-García and Pardo 2005; Heintze and Bretschneider 2000). Processing new flows of data and turning them into information that can be used for decision making require new skills and capacities, potentially implying significant costs for public agencies. Being responsive to new information often requires a realignment of work effort, which can be administratively challenging, politically contentious, or limited by existing procedure (Laffin and Ormston 2013; Liu and Yuan 2015). Using citizen reporting to produce public services can also be difficult because citizen-sourced data are often unstructured, noisy, off-topic, or inconsistent, making it difficult for public officials to act upon. If citizen-sourced data are not consistent enough to reduce the uncertainty managers face about decision making, then its potential to improve the delivery of public services is eroded (Grossman, Platas, and Rodden 2018). Moreover, certain types of citizens might be more likely to provide information, rendering the resulting data unrepresentative of actual conditions or opinions (Parrado et al. 2013). We provide a large-scale, field experimental test of the impact of citizen reporting on the delivery of public services and first-hand qualitative analysis about the challenges of adopting and operating a citizen-reporting platform as a coproduction strategy. Our main contribution is to explain the conditions under which governments would benefit from relying on citizens to produce information for use in managing public services. Based on causal evidence about the impact of a citizen-reporting program, we offer broader lessons about promising directions. Based on a partnership with the Kampala Capital City Authority (KCCA) in Uganda, we study the adoption of a new text-messaging platform to collect, process, and aggregate citizen reports about waste collection. We randomly assigned 100 neighborhoods to a citizen-reporting program, with 50 citizens recruited to provide weekly reports to the municipal government about various aspects of solid waste management. Volunteer citizens sent 23,862 reports on the delivery of solid waste services over the approximately 9-month study period. We measured and photographed 679 informal waste piles—indicators of a lack of access to formal disposal options—at baseline and twice after the reporting platform had operated for several months, both in the neighborhoods assigned to the citizen-reporting program and in another 100 control neighborhoods not assigned to the citizen-reporting program. We also recorded our observations about the operation of the platform systematically based on a unique opportunity to embed part of our research team at the KCCA. We conducted interviews with every staff member at the KCCA who interacted with the platform. We are thus in a position not only to report on a large-scale experiment testing how citizen reporting affected service delivery, but also to offer first-hand lessons about the challenges of using citizen reporting to provide public services. Neighborhoods assigned to the citizen-reporting program did not experience substantial reductions in informal dumping, when compared with neighborhoods not assigned to the citizen-reporting program. We observe some promising results in terms of waste burning, pile organization, and pile containment at the first post-treatment audit of informal waste piles 5 months after the baseline, but these results do not persist to the second post-treatment audit 4 months later. This primary result does not vary based on the amount of reporting or the political affiliation of the division councilor, nor is the result associated with differences in the content of reporting by neighborhoods or the accessibility of neighborhoods to KCCA workers. During the implementation of the reporting program, we observed that staff at the KCCA questioned the effectiveness of citizen reporting for coproduction. Eventually, managers at the KCCA chose to abandon the program because of concerns about cost and the reliability of citizen reports. Reliance on citizen reporting, even though it can be massive, timely, and localized, does not provide an easy solution for managers who struggle to collect the information needed to effectively produce public services. Although citizen reporting can reduce the cost of monitoring services and allow for a greater proportion of public resources to be spent on services, it is also likely to produce unreliable data in many settings, requiring significant effort for processing and interpretation. Digital Coproduction, Information, and Public Services Public agencies confront two fundamental management challenges when trying to improve public services. First, public agencies face resource constraints and must decide how to allocate a limited pool of resources to maximize service delivery. Second, managers at public agencies lack perfect information about how services are actually delivered by their frontline agents. When public managers lack information about the delivery of services, they can neither oversee frontline providers effectively nor identify which changes in the allocation of resources would yield the largest improvements to services. The severities of these challenges are related because managers need to allocate resources to access and process information, leaving fewer resources for delivering services. If managers spend fewer resources to acquire and process information, they can deliver more services, but they may do so less effectively because they do not have the information needed to make the best decisions. This means that as managers relax one constraint they tighten the other. One way managers can navigate this trade-off is to rely on citizens to provide information about the extent and quality of service delivery. By relying on citizens to provide information, managers might simultaneously decrease their own costs of monitoring public services and improve the quality and breadth of information that they have available to make decisions (McCubbins and Schwartz 1984). For this reason, many public agencies have engaged citizens in a specific form of coproduction called “coassessment,” where citizens provide voluntary contributions of information regarding public services and managers actively seek to use that information to deliver public services (Nabatchi, Sancino, and Sicilia 2017). Citizens can often rely on their daily experiences to obtain this information. Traditionally, this type of coproduction has occurred in the context of group meetings between citizens and managers (Hock, Anderson, and Potoski 2013). Yet, the provision of information by citizens in face-to-face forums is often infrequent, and participation is often limited to a small subset of citizens (Clark, Brudney, and Jang 2013; Pestoff 2006). Organizing in-person forums for information sharing can be slow and costly (Irvin and Stansbury 2004), particularly given that effective strategies to increase participation are elusive (Arceneaux and Butler 2016). Advances in public management increasingly involve addressing dynamic problems with real-time data (Mergel, Rethemeyer, and Isett 2016). ICTs might significantly expand coproduction opportunities across a variety of services by making it easier for citizens to provide information managers can use to improve service delivery. Indeed, there is broad optimism that ICTs might improve the volume, timeliness, and coverage of information useful for decision making about public services, while also encouraging broader participation (Bovaird 2007; Charalabidis et al. 2012; Grossman, Humphreys, and Sacramone-Lutz 2014; Linders 2012; Meijer 2012; Oates 2003; Rotberg and Aker 2013; Zurovac, Talisuna, and Snow 2012). If ICTs help managers collect citizen-sourced information broadly, accurately, in a timely manner, and at low costs, then they might simultaneously relax managers’ information and budget constraints and therefore improve service delivery. Despite the promise of ICTs for coproduction, limited evidence is available about the overall effects of ICT-enabled citizen reporting on public services (Lember, Brandsen, and Tonurist 2019). The conditions, technologies, scales, and services where citizen reporting is most likely to improve public services remain underspecified. For instance, although ICT-based citizen reporting might increase the information available to managers, deploying these platforms at scale can impose large costs on governments (Gil-García and Pardo 2005; Heintze and Bretschneider 2000). Alternatively, the inherent inconsistency of information from citizens might be tolerated for some tasks like long-term planning, but not others like deciding where to deploy crews for the maintenance of infrastructure. The underspecification of when and why ICT-enabled coproduction has the greatest potential to improve service delivery is particularly concerning given the proliferation of citizen-reporting programs around the world (Linders 2012). Conditions for Successful ICT-Enabled Citizen Reporting We investigate several factors that are likely to determine whether ICT-enabled citizen reporting will improve public services: the alignment of citizen-produced data with the information problems that managers face; the search costs of detecting public services failures for managers; the quality of citizen-produced data; and the operating costs managers incur from deploying and operating ICT-based reporting platforms. We use qualitative evidence to assess how these factors contributed to the overall effect of citizen reporting on public services for the KCCA, which we evaluate using a large-scale field experiment. Alignment of Information and Uncertainty Public managers face various information problems related to the provision of public services, including uncertainty about the performance of frontline providers, the most effective allocation of resources, public preferences for different service outcomes, and where the maintenance of public infrastructure is necessary. Resolving different types of uncertainty requires different types of information, which citizens may or may not be in a good position to provide. For example, matching long-term service improvements to citizen preferences requires representative information about public demands, while identifying maintenance tasks requires a single citizen to accurately report a specific need for agency action. Citizens are most likely to have advantages in contributing two kinds of information through ICT-enabled reporting platforms: their own observations and their own preferences. If the main information problem that managers face is about where to direct efforts to discrete problems like potholes in public roads (Sjoberg, Mellon, and Peixoto 2017), then reporting platforms that collect precise spatial information based on citizens’ observations can help target agency actions. Alternatively, if managers need to decide what types of public investments to prioritize, new tools might enable a broader, more representative set of citizens to contribute information about their preferences (Grossman, Humphreys, and Sacramone-Lutz 2014; Robbins, Simonsen, and Feldman 2008; Stipak 1980). However, incorporating citizen feedback into technical decisions requiring high levels of expertise could detract from decision-making effectiveness (Irvin and Stansbury 2004). Citizen reporting must generate information that reduces the kind of uncertainty that prevents effective decision making. Search Costs for Public Service Failures Governments are frequently responsible for managing dynamic problems that are difficult to detect in real time, such as crime or the failure of public infrastructure. Efficiently addressing these problems requires timely and specific knowledge about where to direct agency effort. When citizens have a comparative advantage in generating this kind of information, citizen-reporting platforms might contribute positively to service delivery. For example, increasing citizen reporting to the police about crime should improve the effectiveness of policing, since officers cannot patrol with enough frequency to have the same level of knowledge as residents (Bennett, Holloway, and Farrington 2006). However, when governments can effectively monitor service shortfalls themselves at low costs—for example, tracking electricity blackouts using real-time voltage data or tracking the performance of public transit using automated GPS systems—citizen reporting is unlikely to lead to more effective service provision. In these cases, citizen reporting does not have advantages in detecting service failures. Related research suggests that sourcing information from a crowd is best applied in settings where organizations can turn problems that involve “distant searching” for information into problems that involve less-costly “local searching” (Afuah and Tucci 2012). Citizen reporting is most likely to be effective when citizens themselves are the best “sensors” of the problems that managers would like to detect. Quality of Citizen-Produced Information The quality of information provided through ICT-enabled citizen reporting must be reliable and consistent. Public managers perceive citizen-produced information to be unhelpful for many of the decisions that they face (Liao and Schachter 2017). ICT-enabled citizen reporting often falls flat because it represents a broader but shallower form of engagement with citizens. ICTs deployed at scale greatly expand the flow of information from citizens but potentially limit the specificity of data. Citizens also may drop out of ICT-enabled reporting more frequently than traditional forms (Yetano and Royo 2017), potentially eroding the coverage of data over time. The average quality and content of information sourced from ICTs is often low (Grossman, Platas, and Rodden 2018), relative to more personal information exchanges between citizens and governments. If citizen reporting generates inconsistent or indecipherable information about where public managers should direct effort, then it may not overcome the uncertainty that limits service delivery. Operating Costs of Citizen-Reporting Platforms The promise of ICT-enabled coproduction for managers is that it can generate an unprecedented amount of useful information at a low cost, relative to the other options that managers have at their disposal. An assumption underlying this promise is that the main cost of operating an ICT-platform comes from maintenance, such as fixing technical glitches. However, ICT-enabled coproduction might strain the existing capacity of public agencies to process information, given the massive flow of data that large citizen-reporting platforms generate. Indeed, evidence suggests that many public officials face significant challenges in processing even simple flows of data (Masaki et al. 2017). Managers also might have to establish additional channels of communication to follow up on citizen reports or significantly reorient work effort to be responsive (Laffin and Ormston 2013; Liu and Yuan 2015). If managers have other options for collecting information that are more cost-effective, they may choose to abandon ICT-enabled citizen reporting altogether. Therefore, ICT-enabled coproduction should lead to improved service provision when adopting ICT platforms does not place significant stress on an agency’s capacity to process and act on information. Theoretical Proposition Strong empirical designs are needed to test theory about the conditions under which ICT-enabled citizen reporting can enhance the provision of public services (Charalabidis et al. 2012; Linders 2012; Saxton, Oh, and Kishore 2013; Seltzer and Mahmoudi 2013). Although several prominent platforms generate citizen monitoring of public services in developed countries (e.g., SeeClickFix, FixMyStreet, NoiseTube), these platforms are not designed to facilitate research on citizen-sourced data provision, quality, and impact. Our field experiment and qualitative research design provide a strong test of the following theoretical proposition: Theoretical Proposition: Citizen reporting that provides information to governments about deficiencies in public services will result in an improvement to public services (when information is aligned with uncertainty, addresses search costs for public service failures, is high-quality, and when platforms to collect it are not onerous to operate). Our study is distinct from recent research about the motivations of citizens to participate in coproduction, such as requesting nonemergency services (O’Brien et al. 2017; Sjoberg, Mellon and Peixoto 2017), providing information on the delivery of public services (Buntaine, Nielson, and Skaggs 2019; Grossman, Michelitch, and Santamaria 2017), and reporting on corruption (Blair, Littman, and Paluck 2019). Instead, we study the impacts of citizen reporting on public service outcomes. Only a few studies address how coproduction can improve the quality of public services. There is some evidence that coproduction improves educational outcomes for children enrolled in public schools (Jakobsen 2012; Jakobsen and Andersen 2013; Thomsen and Jakobsen 2015). Closer to the focus of this study, Grossman, Platas, and Rodden (2018) find no evidence that citizen reporting to politicians substantially improves access to health, water, and education in rural Uganda. We contribute to this body of research by providing a quantitative test of the impact of citizen reporting to the bureaucrats responsible for service delivery, and qualitative evidence about when citizen reporting is helpful for management. Research Design and Analytical Methods Overview We study the impact of citizen reporting on the delivery of solid waste services in Kampala based on a large-scale, randomized field experiment and primary qualitative evidence about how the conditions theorized above contributed to the overall impact of the coproduction program. Studying coproduction in the context of waste management in Kampala is appropriate because the agency responsible for waste services faces significant resource and information constraints and has actively sought the voluntary contributions of information from citizens. Providing information is presently the most relevant of all the coproduction activities available to citizens for waste management. Setting A majority of solid waste in Kampala, a city of approximately 1.5 million residents, is dumped informally or openly burned in streets and alleys. These practices have created major public health challenges in terms of both air and water pollution. A large majority of residents are personally concerned with solid waste services (Buntaine, Nielson, and Skaggs 2019). The KCCA faces similar problems in monitoring and delivering solid waste management as other agencies around the world, as the rapid rate of urbanization in Kampala has outstripped its capacity to provide and oversee services (Bhuiyan 2010; Okot-Okumu and Nyenje 2011; Vermeiren et al. 2012). In the last few years, the KCCA adopted a public–private partnership (PPP) approach to deliver waste services. Under this approach, the KCCA contracts out the management of solid waste services to private concessionaires, which are responsible for collecting, transporting, and disposing solid waste from particular areas of the city. Under the PPP, the private concessionaires are allowed to charge residents in return for collecting their solid waste on a door-to-door basis. At the same time, they are contractually required to provide common collection points available to all residents regardless of ability to pay. The incentive to maximize revenue from citizens through door-to-door collection is at odds with requirements to make collection widely accessible, so contractors have mostly failed to establish and service common collection points. As a result, solid waste conditions have deteriorated in recent years for residents who are not able to afford door-to-door collection or who do not live in an area where it is offered. The KCCA needs information about where services are not being provided to allocate oversight and supplementary clean-up efforts. Yet with a small office of professional staff, the waste management unit has not been in a position to widely monitor the performance of private contractors with existing resources. Problems are so widespread and pressing that it has chosen to spend most resources organizing haphazard clean-ups, rather than oversight. Because of resource and information constraints, the KCCA was enthusiastic that citizen reporting would expand its ability to exercise oversight at a scale that matched the growing lack of access to formal services among residents. With better information from citizens about the locations and types of service failures, the KCCA anticipated that it could make more informed decisions about the oversight of pick-up times, service routes, schedules, and pricing options—all of which are essential parts of the waste management process. Experimenting with a new approach to collect information was also viewed as an opportunity to learn about the effectiveness of this kind of coproduction program generally, given a broader interest at the highest levels of management in rolling out citizen reporting across directorates for water supply, sanitation, and transportation. Prior to the citizen-reporting program initiated as part of this study, the KCCA collected unstructured information about waste management through toll-free phone lines, a general SMS shortcode, and social media websites. KCCA frontline staff, or Client Care Officers, were responsible for processing information received through these channels. Once processed, input from citizens was relayed to the appropriate supervisor within the KCCA, who then decided how to address any problems. Staff estimated that 15–20 complaints relevant to solid waste management were received per working day. The KCCA also gained information from local leaders, such as parish councilors and zone chairpersons. Solid Waste Officers often employed a small number of informal scouts to obtain information. Yet these sources of information were not systematic enough for public managers to plan, allocate resources, and exercise oversight. From the perspective of citizens, the adoption of a service system in which they were required to pay for the waste collection services provided a greater incentive to report service deficiencies. Additionally, citizens in Kampala are uniquely situated to provide much of the information the KCCA sought, since citizens directly observe informal waste piles, missed pick-ups, and patterns of collection efforts that do not match their needs for waste services. Citizen Reporting and the Coproduction of Services The rapid proliferation of mobile phones in Kampala created opportunities for coproduction by citizen reporting. The latest statistics about Kampala indicate that more than 90% of adults own a mobile phone (Uganda Bureau of Statistics 2017), which might enable citizens to share valuable information broadly and at low costs. In 2014, our research team approached the KCCA to investigate whether they would be interested in adopting and testing a platform that would enable citizens to send information about the quality of waste collection services in real time and at the scale of neighborhoods. The idea was met with enthusiasm from leadership, overcoming a key challenge for these kinds of efforts (Hansen and Norup 2017). The KCCA codeveloped a toll-free SMS-messaging platform with our research team over time, which it used to collect information from volunteer citizen reporters. Because we recruited these citizen reporters in the field, all of the reports can be tagged to individual “zones” or neighborhoods throughout Kampala, which are the lowest-level administrative unit (LC1) in both the city and throughout Uganda. The program built on existing initiatives at the KCCA to be more responsive to citizens’ complaints. Between November 2015 and August 2017, we prompted citizens to provide reports about various aspects of solid waste management to a single, toll-free SMS shortcode established for the project. To process citizen reports, we employed a customized application of SMSOne procured by the KCCA. This platform offers a tested and convenient way to manage a large number of incoming and outgoing messages from mobile phones, and the KCCA was expanding its use to manage all types of communication with citizens. Our research team and the KCCA waste management unit codesigned the prompts sent to citizen reporters. For example, we used the following prompt at various points throughout the study period: When did the rubbish truck last collect your rubbish? A) never B) more than two weeks ago C) last week D) this week Treatment Treatment in our study is the assignment of a zone to the citizen-reporting program, which involved recruiting 50 volunteer reporters and then sending them prompts for information each week. Under this clustered-assigned design, treated neighborhoods (“zones”) had reporters that were prompted to provide information to the KCCA over several months, whereas control neighborhoods did not have any reporters recruited or prompted to send information.1 Each week, the platform sent an identical prompt to recruited reporters in treated zones from among a list of prompts that the KCCA waste management unit identified as important for management. Additionally, all reporters were sent messages confirming that the KCCA was actively seeking to use the information they provided, since responsiveness is key to sustaining citizen reporting (Buntaine, Nielson, and Skaggs 2019). We recruited the number of reporters necessary to ensure that information was available from reporters each week in almost all zones assigned to treatment. We observed the rate of on-topic and usable reporting from citizens that we expected, averaging around a 10% response rate throughout the study period. To our knowledge, this is the best response rate for this kind of platform in a low-income country (see, e.g., Buntaine, Nielson, and Skaggs 2019; Grossman and Michelitch 2018). With this response rate, there was an average of three to seven reports per week per treated zone, from among 50 recruited reporters. This means that the treatment succeeded in providing the KCCA with consistent information about treated zones that it did not have about control zones. In total, the KCCA received 17,538 verified and usable reports prior to the final waste audit, drawn from 23,862 raw reports. Although the amount of reporting varies by zone and across weeks under this design, the zone-based randomization of the reporting program means that the amount of reporting delivered from treated zones is equivalent to the expected amount of reporting that would have been delivered in control zones had they been assigned to treatment. We do not seek to identify the effect of individual reports, but rather the effect of a zone-level citizen-reporting program on zone-level waste outcomes. Our research team transmitted all relevant reports to the KCCA on a weekly basis in a spreadsheet format, with the responses aggregated to the zone level, as requested by the KCCA (see Supplementary Appendix L). Our research team was not involved in planning or delivering responses to the reports. Sample and Random Assignment We randomly selected 200 of Kampala’s 755 zones to form our sample of zones that could be assigned to treatment or control. We randomly selected an additional 50 zones to use as replacements for zones that were inaccessible to our enumerators, demolished at the time of enumeration, or for which at least two informal waste piles could not be identified by residents of the zone at baseline. We randomly assigned half of the zones to the citizen-reporting program using complete randomization, with the result displayed in Figure 2. Figure 1. Open in new tabDownload slide Reporting rates for all prompts during the study period. Figure 1. Open in new tabDownload slide Reporting rates for all prompts during the study period. Figure 2. Open in new tabDownload slide Experimental sample, including continuing reporting from previous phases. Figure 2. Open in new tabDownload slide Experimental sample, including continuing reporting from previous phases. We intended to select a sample that included entirely new zones without any previous reporting. Due to an indexing error, we selected a sample that overlapped with the samples from earlier phases of the project where low levels of reporting had been organized (Buntaine, Nielson, and Skaggs 2019). This error was not caught until after baseline data had been collected. The resulting treatment still adds 50 new reporters to each of these zones, on average boosting the number of reports considerably. Our baseline measure of the size of waste piles accounts for any differences in waste conditions that emerged from earlier citizen reporting. To account for variation that might be driven by this indexing error, we add an indicator of whether any citizen reporting was organized previously in all analyses. A challenging aspect of this setting was that the KCCA is a single organization that could not be blinded to treatment assignment, since the delivery of reports would indicate that the zone had been treated. The threat to inference is that that KCCA might redirect attention to some zones due to their assignment to treatment, rather than in response to the information in reports. Our approach to this challenge was multifaceted. First, we operated at a scale that would make a general reallocation of effort to treated zones very difficult and costly. The prompts addressed different aspects of waste management that required precise actions such as follow-up with contractors, the organization of clean-ups, or the sensitization of communities regarding disposal practices at the zone level. Responses varied by zone and could not be feasibly rolled out indiscriminately. Second, we tracked that responses were at the zone level and specific to information in reports. In particular, the KCCA shared their weekly action plans created in response to reports. An example action plan is displayed in Supplementary Appendix M, which shows that responses to reports were largely specific to zones. Finally, we continued to collect and pass along reports from hundreds of zones in previous phases of the project, which would not enable the KCCA to precisely identify zones being measured for the present study. Supplementary Figure J1 displays balance and descriptive statistics for pretreatment covariates, none of which are inconsistent with random assignment. Figure 3 tracks the study design. Figure 3. Open in new tabDownload slide CONSORT diagram tracking study design. Figure 3. Open in new tabDownload slide CONSORT diagram tracking study design. Empirical Prediction Both our research team and the KCCA predicted that information from the reporting platform could be used to improve services. Prior to the platform’s launch, the KCCA lacked a method to collect data from a broad base of citizens and relied on information from informally employed “scouts” and administrative records on waste collection (see Supplementary Appendix G for details). Information on waste conditions, therefore, was limited to a subset of easily accessible areas. The management team that we worked with to codevelop the reporting platform was confident that many of the factors we expected to support a positive impact of ICT-enabled coproduction were present. In terms of the alignment of information and decision making, the key gap identified by managers was the lack of spatial data on service failures, which could be used for oversight. In terms of search costs, existing methods of gaining information could not cover the growing scale of solid waste management challenges.2 In terms of the quality of information produced by citizens, there was broad optimism that prompts for structured responses based on factual observations would lead to high-quality information. In terms of operating costs, the KCCA was already making large-scale investments in programs to respond to citizen complaints and anticipated that gaining systematic data might decrease the burden of responding in haphazard ways to individual complaints. Our field experiment thus tests the following hypothesis: Empirical Prediction Zones assigned to citizen reporting will experience a larger decrease in solid waste accumulation than zones assigned to control. Measurement of Outcomes To assess whether zones assigned to the citizen-reporting program experienced improvements in waste services, we completed field-based audits of informal waste piles (see Supplementary Appendix A for details). We focus on informal waste piles because they represent a direct outcome of low quality or inaccessible formal methods of waste disposal. Improvements in the delivery of KCCA waste services should offset citizens’ use of informal waste piles. At baseline, we visited each zone in the experimental sample and asked residents to show us four informal waste piles that were of greatest concern. We measured the size of these waste piles, photographed them, recorded their locations by GPS, and mapped the easiest way to return to them. In both post-treatment audits, we revisited each sampled waste pile and remeasured its size. The core outcome of our field experiment is whether informal waste piles in treated zones have larger reductions in size than those in control zones, comparing baseline pile sizes to remeasurements at 5 and 9 months post-treatment. Figures 4–6 are representative examples of small, medium, and large waste piles, respectively. Figure 4. Open in new tabDownload slide Small pile. Figure 4. Open in new tabDownload slide Small pile. Figure 5. Open in new tabDownload slide Medium pile. Figure 5. Open in new tabDownload slide Medium pile. Figure 6. Open in new tabDownload slide Large pile. Figure 6. Open in new tabDownload slide Large pile. We additionally recorded whether piles displayed evidence of rubbish burning, the composition of waste in each pile, and the organization of waste at each pile at each post-treatment audit. These measures allow us to quantify any secondary improvements to waste management short of full clean-ups. For instance, KCCA staff often would organize rubbish and place it into containers for future transport if their collection trucks ran out of space. Measuring how waste is organized and contained at each pile allows us detect this action by the KCCA to improve the quality of waste services. Collecting evidence of waste burning at each informal waste pile allows us to investigate whether citizens experienced marginally better access to formal waste services, since citizens commonly burn their waste when they cannot utilize formal services. Preregistered Measures (From Photographs and Field Measurements) • Area of total waste accumulation (primary outcome) • Area of unmanaged waste accumulation • Amount of burning • Amount of nonorganic waste Estimation Our estimation strategy investigates the extent to which waste piles in treated zones differ from waste piles in control zones. We test the main empirical predictions using a series of ordinary least squares regressions, where the outcome measurements about the waste piles are the dependent variables and the treatment status of the zone is the main independent variable. While not necessary for unbiased estimates of the effect of treatment on waste outcomes, we also add a number of covariates to the regression to increase the precision of the estimated effects: zone-level treatment status in previous phases, baseline pile sizes, and zone-level measures of population, density of improved roads, and luminosity. We obtain our estimates of uncertainty from randomization inference, which computes the uncertainty from the experimental design and different possible randomization draws (Gerber and Green 2012). In particular, we compute sharp null standard errors by assuming no effect and recording the treatment effect that would have been observed under each of 10,000 permissible randomization draws. The estimating equation used for this process for measures with both baseline and endline values is Equation 1. yij,t=b+n=α+τMj++γyij,t=b+βXj+νh+∈j(1) where y is the relevant size measure for pile i in zone j at time b baseline plus some follow-up period n, τ is the treatment effect of interest, Mj+ is a binary indicator of treatment assigned at the zone-level j, r is the parameter estimating the relationship of baseline size measure yij,t=b to the follow-up outcome measure, βXj is the estimated adjustment for pretreatment, zone-level covariates including the treatment status of zones during previous phases, νh is a fixed effect for division, and ∈j is an error term clustered at zone, often irrelevant in our case because we report sharp null standard errors for analyses conducted by randomization inference. This estimation deviates from our preregistered strategy in that it takes the pile, rather than the zone as the unit of analysis, which increases precision. Justifications of deviations from our preregistered analytical strategy are available in Supplementary Appendix E. Summary statistics for the effective sample used for analysis are available in Supplementary Appendix, Table J10. Qualitative Analysis Our field experiment allows us to estimate the average effect of the citizen-reporting program on levels of informal waste disposal. This estimate is informative about the overall effect of ICT-enabled citizen reporting on public service quality. However, the field experimental estimate of program effects does not provide direct evidence about how the features of the specific coproduction program conditioned its success. We rely on qualitative data to address theoretical propositions about the conditions under which coproduction will improve service provision. Our team was embedded in the KCCA waste management unit for close to 1 year. During this time, we interacted with a variety of KCCA staff members, from managers to frontline staff providing waste services. We had access to and reviewed KCCA documents, participated in KCCA meetings, and regularly observed interactions between the KCCA and its stakeholders. We recorded ongoing observations systematically. Following the last field audit, we conducted in-depth interviews with all individuals who interacted with the citizen-reporting platform. We use these qualitative data to shed light on the key factors that contributed to the overall effects estimated in the field experiment and to offer guidance about the conditions that are necessary for ICT-enabled citizen reporting to enhance public service provision generally. We present the qualitative data in the theoretical categories outlined above—type of agency uncertainty, costs of detecting service failures, quality of citizen-produced information, and the cost of processing information. Taken together with the field experimental results, the qualitative analysis significantly expands our ability to explain when and why ICT-enabled citizen reporting might succeed. Results Field Experimental Results Pile Sizes and Pile Clean-up Speaking directly to the impact of citizen reporting on the delivery of waste services, Table 1 shows that we cannot rule out a zero effect of treatment on waste accumulation in informal piles. We also test the degree to which treatment increases the probability that KCCA staff or contractors clear waste piles fully. We again cannot rule out a zero effect of treatment on waste pile clean-ups. The proportion of cleaned piles in treated and control zones at both post-treatment audits are very similar (see also Figures 7 and 8). We also cannot rule out a zero effect for the more statistically powerful rank test, which we added to deal with a few unexpectedly large outliers. We do not observe a different result when using a difference-in-difference estimation strategy (Supplementary Table J11). Even when we relax our coding of a cleaned pile to include sites where all waste was collected into a single, transportable container, we find no difference in the proportion of cleaned sites among treatment and control groups inconsistent with a zero effect of treatment (Figures 7 and Figures 8, “Pile Cleaned, Adjusted” plots). Table 1. Treatment Effect of Citizen Reporting on Cleaned Pile Sizes . Pile Size . Pile Size . Pile Present . Pile Present . Pile Rank . Pile Rank . Audit M1 M2 M1 M2 M1 M2 Treatment effect −4.234 −7.781 −0.032 −0.011 −11.109 −6.544 Standard error 3.467 12.746 0.033 0.039 16.673 16.264 p-value .112 .303 .176 .389 .262 .342 N 679 679 679 679 679 679 . Pile Size . Pile Size . Pile Present . Pile Present . Pile Rank . Pile Rank . Audit M1 M2 M1 M2 M1 M2 Treatment effect −4.234 −7.781 −0.032 −0.011 −11.109 −6.544 Standard error 3.467 12.746 0.033 0.039 16.673 16.264 p-value .112 .303 .176 .389 .262 .342 N 679 679 679 679 679 679 Note: Results calculated using cleaned waste pile size measurements. One-sided p-values in hypothesized direction. Description of Dependent Variables 1. Pile size: waste pile area (m2), measured at the specified midline audit. Enumerators recorded waste pile dimensions at each audit. These dimensions were used to estimate waste pile area for each site in the sample. 2. Pile present: binary indicator variable for whether a waste pile was cleaned or not at the specified midline audit. Recorded values of 0 indicate that no pile was present at the given midline audit (e.g., the pile had been cleaned); recorded values of 1 indicate the opposite. 3. Pile rank: waste pile size rank, calculated at the specified midline audit. Due to high variance in the recorded waste pile sizes, we perform a rank test comparing the ranked change in pile size between the baseline audit and each midline audit. Waste piles were ranked at each audit based on their size relative to other waste piles. Open in new tab Table 1. Treatment Effect of Citizen Reporting on Cleaned Pile Sizes . Pile Size . Pile Size . Pile Present . Pile Present . Pile Rank . Pile Rank . Audit M1 M2 M1 M2 M1 M2 Treatment effect −4.234 −7.781 −0.032 −0.011 −11.109 −6.544 Standard error 3.467 12.746 0.033 0.039 16.673 16.264 p-value .112 .303 .176 .389 .262 .342 N 679 679 679 679 679 679 . Pile Size . Pile Size . Pile Present . Pile Present . Pile Rank . Pile Rank . Audit M1 M2 M1 M2 M1 M2 Treatment effect −4.234 −7.781 −0.032 −0.011 −11.109 −6.544 Standard error 3.467 12.746 0.033 0.039 16.673 16.264 p-value .112 .303 .176 .389 .262 .342 N 679 679 679 679 679 679 Note: Results calculated using cleaned waste pile size measurements. One-sided p-values in hypothesized direction. Description of Dependent Variables 1. Pile size: waste pile area (m2), measured at the specified midline audit. Enumerators recorded waste pile dimensions at each audit. These dimensions were used to estimate waste pile area for each site in the sample. 2. Pile present: binary indicator variable for whether a waste pile was cleaned or not at the specified midline audit. Recorded values of 0 indicate that no pile was present at the given midline audit (e.g., the pile had been cleaned); recorded values of 1 indicate the opposite. 3. Pile rank: waste pile size rank, calculated at the specified midline audit. Due to high variance in the recorded waste pile sizes, we perform a rank test comparing the ranked change in pile size between the baseline audit and each midline audit. Waste piles were ranked at each audit based on their size relative to other waste piles. Open in new tab Figure 7. Open in new tabDownload slide Dependent variables, Midline 1. Note: Plotted bars report simple bootstrapped standard errors. Figure 7. Open in new tabDownload slide Dependent variables, Midline 1. Note: Plotted bars report simple bootstrapped standard errors. Figure 8. Open in new tabDownload slide Dependent variables, Midline 2. Note: Plotted bars report simple bootstrapped standard errors. Figure 8. Open in new tabDownload slide Dependent variables, Midline 2. Note: Plotted bars report simple bootstrapped standard errors. Neither a lack of statistical power nor spillover is driving our results. In Supplementary Appendix K, we show that we have power to detect standardized effect sizes that are all <0.2 in the main estimation, implying that the experiment had sufficient power. Spillover between zones may incorrectly suggest a null effect of treatment if zones assigned to the control condition were improved alongside treated zones, for example by supplementary clean-ups based on proximity. Supplementary Appendix I demonstrates that our results hold when using specifications that account for possible spillover in citizen reporting between contiguous zones in our sample. Pile Characteristics Although we find no evidence suggesting that treatment significantly reduced total waste accumulation or significantly increased waste pile clearance, we observe some promising effects of treatment on the amount of uncontained, disorganized, and burnt waste at the first post-treatment audit. Table 2a shows that treatment reduced the estimated proportion of burnt area among waste piles in treated zones. The effects of treatment on the area of uncontained and dispersed waste pile area are also indicative of a positive effect of treatment at the first post-treatment audit (Table 2a). The results at the second post-treatment audit are less conclusive. Although the point estimates of treatment on each outcome appear similar between audits, variation in measured pile sizes inflates our standard errors and thus prevents us from confidently ruling out a zero effect of treatment at the second post-treatment audit. Table 2. Treatment Effect of Citizen Reporting on Secondary Dependent Variables for Area (Cleaned) . Uncontained . Uncontained . Disorganized . Disorganized . Burnt . Burnt . (a)Midline 1  Variable specification A B A B A B  Treatment effect −4.444 −4.521 −4.536 −4.546 −2.298 −2.495  Standard error 3.338 3.368 3.428 3.415 0.768 0.859  p-value 0.094 0.092 0.096 0.094 0.001 0.001  BH p-value 0.096 0.096 0.096 0.096 0.003 0.003  N 679 679 679 679 679 679 (b)Midline 2  Variable specification A B A B A B  Treatment effect −4.521 −7.662 −7.337 −7.216 −2.002 −3.097  Standard error 3.354 10.499 12.753 12.798 1.931 2.418  p-value 0.094 0.284 0.316 0.319 0.174 0.123  BH p-value 0.319 0.319 0.319 0.319 0.319 0.319  N 679 679 679 679 679 679 . Uncontained . Uncontained . Disorganized . Disorganized . Burnt . Burnt . (a)Midline 1  Variable specification A B A B A B  Treatment effect −4.444 −4.521 −4.536 −4.546 −2.298 −2.495  Standard error 3.338 3.368 3.428 3.415 0.768 0.859  p-value 0.094 0.092 0.096 0.094 0.001 0.001  BH p-value 0.096 0.096 0.096 0.096 0.003 0.003  N 679 679 679 679 679 679 (b)Midline 2  Variable specification A B A B A B  Treatment effect −4.521 −7.662 −7.337 −7.216 −2.002 −3.097  Standard error 3.354 10.499 12.753 12.798 1.931 2.418  p-value 0.094 0.284 0.316 0.319 0.174 0.123  BH p-value 0.319 0.319 0.319 0.319 0.319 0.319  N 679 679 679 679 679 679 Note: Results calculated using cleaned waste pile size measurements. Description of Dependent Variables1. Uncontained: estimate of uncontained waste pile area (m2). At each midline audit, enumerators recorded how rubbish was stored in each waste pile. Responses ranged from “all of the rubbish is neatly contained” to “no rubbish is contained in sacks or containers.” Each response was assigned a scalar between 0.0 and 1.0, which was multiplied against the recorded waste pile area to estimate the uncontained waste pile area.2. Disorganized: estimate of unorganized waste pile area (m2). At each midline audit, enumerators recorded the dispersion of rubbish in each waste pile. Responses ranged from “all of the rubbish is collected in a single pile” to “rubbish is spread all around [with] no evidence of the rubbish being organized.” Each response was assigned a scalar between 0.0 and 1.0, which was multiplied against the recorded waste pile area to estimate the unorganized waste pile area.3. Burnt: estimate of burnt waste pile area (m2). At each midline audit, enumerators recorded any evidence of burning they observed at each waste pile. Responses ranged from “no evidence of burning” to “more than half of the area of the rubbish pile contains evidence of burning.” Each response was assigned a scalar between 0.0 and 1.0, which was multiplied against the recorded waste pile area to estimate the burnt waste pile area.Variable specification A is less conservative than specification B. Specification A assigns a smaller scalar to enumerator responses indicating less organization/storage and more burning. See Supplementary Appendix F for description of scalars. Benjamini–Hochberg (BH) p-value is minimum family-wise false discovery rate under which the null hypothesis would be rejected for each test using the BH procedure. Open in new tab Table 2. Treatment Effect of Citizen Reporting on Secondary Dependent Variables for Area (Cleaned) . Uncontained . Uncontained . Disorganized . Disorganized . Burnt . Burnt . (a)Midline 1  Variable specification A B A B A B  Treatment effect −4.444 −4.521 −4.536 −4.546 −2.298 −2.495  Standard error 3.338 3.368 3.428 3.415 0.768 0.859  p-value 0.094 0.092 0.096 0.094 0.001 0.001  BH p-value 0.096 0.096 0.096 0.096 0.003 0.003  N 679 679 679 679 679 679 (b)Midline 2  Variable specification A B A B A B  Treatment effect −4.521 −7.662 −7.337 −7.216 −2.002 −3.097  Standard error 3.354 10.499 12.753 12.798 1.931 2.418  p-value 0.094 0.284 0.316 0.319 0.174 0.123  BH p-value 0.319 0.319 0.319 0.319 0.319 0.319  N 679 679 679 679 679 679 . Uncontained . Uncontained . Disorganized . Disorganized . Burnt . Burnt . (a)Midline 1  Variable specification A B A B A B  Treatment effect −4.444 −4.521 −4.536 −4.546 −2.298 −2.495  Standard error 3.338 3.368 3.428 3.415 0.768 0.859  p-value 0.094 0.092 0.096 0.094 0.001 0.001  BH p-value 0.096 0.096 0.096 0.096 0.003 0.003  N 679 679 679 679 679 679 (b)Midline 2  Variable specification A B A B A B  Treatment effect −4.521 −7.662 −7.337 −7.216 −2.002 −3.097  Standard error 3.354 10.499 12.753 12.798 1.931 2.418  p-value 0.094 0.284 0.316 0.319 0.174 0.123  BH p-value 0.319 0.319 0.319 0.319 0.319 0.319  N 679 679 679 679 679 679 Note: Results calculated using cleaned waste pile size measurements. Description of Dependent Variables1. Uncontained: estimate of uncontained waste pile area (m2). At each midline audit, enumerators recorded how rubbish was stored in each waste pile. Responses ranged from “all of the rubbish is neatly contained” to “no rubbish is contained in sacks or containers.” Each response was assigned a scalar between 0.0 and 1.0, which was multiplied against the recorded waste pile area to estimate the uncontained waste pile area.2. Disorganized: estimate of unorganized waste pile area (m2). At each midline audit, enumerators recorded the dispersion of rubbish in each waste pile. Responses ranged from “all of the rubbish is collected in a single pile” to “rubbish is spread all around [with] no evidence of the rubbish being organized.” Each response was assigned a scalar between 0.0 and 1.0, which was multiplied against the recorded waste pile area to estimate the unorganized waste pile area.3. Burnt: estimate of burnt waste pile area (m2). At each midline audit, enumerators recorded any evidence of burning they observed at each waste pile. Responses ranged from “no evidence of burning” to “more than half of the area of the rubbish pile contains evidence of burning.” Each response was assigned a scalar between 0.0 and 1.0, which was multiplied against the recorded waste pile area to estimate the burnt waste pile area.Variable specification A is less conservative than specification B. Specification A assigns a smaller scalar to enumerator responses indicating less organization/storage and more burning. See Supplementary Appendix F for description of scalars. Benjamini–Hochberg (BH) p-value is minimum family-wise false discovery rate under which the null hypothesis would be rejected for each test using the BH procedure. Open in new tab Figures 7 and 8 similarly display the weak persistence of improvements from the first to second post-treatment audit. At the first post-treatment audit, we observe a larger proportion of waste piles in treated zones that show no evidence of waste burning compared with piles in control zones (te = 0.07, p = .04). We also see that a smaller proportion of piles in treated zones contain more than 10 pieces of inorganic waste than do piles in control zones at the first midline audit (te = −0.13, p = .0005). However, both of these effects attenuate at the second post-treatment audit, where the proportion of piles in treated zones displaying significant evidence of burning and the proportion of piles in treated zones with more than ten pieces of inorganic waste are statistically indistinguishable from control zones. Only with respect to waste storage do we find evidence that the treatment effect we observe at the first post-treatment audit persists to the second post-treatment audit. Although the proportion of fully contained piles in treated zones exceeds the proportion of fully contained piles in control zones (Figure 8), this effect is not very inconsistent with the null hypothesis (te = 0.03, p = .08). When estimating the uncontained area of waste piles, we find a similarly suggestive effect (Table 2b, p = .10). However, there is more uncertainty about this effect when we use a more conservative estimate of uncontained waste pile area (Table 2b, p = .27). Robustness It is possible that the results reported above mask treatment effects where they are most likely to occur by averaging across heterogeneity at the zone level. We therefore investigate the heterogeneous effects of treatment across a number of zone-level characteristics that might condition the impact of citizen reporting on waste services. These analyses are robustness checks that give us greater confidence that the null effect of treatment in the main analysis is not a consequence of averaging over heterogeneous effects. Political Targeting Politicians often use public goods and services to reward supporters in elections (Baldwin 2013; Briggs 2012; Drazen and Eslava 2010; Jablonski 2014). In the setting of our study, the National Resistance Movement (NRM) is the ruling party nationally, but has low levels of political support within Kampala. In 2011, aiming to reverse entrenched opposition within the capital city, the municipal government was nationalized and responsibility for services transferred from the elected city council to the KCCA. Thus, the KCCA may have used their discretion to reward areas of the city that vote for NRM candidates, when compared with opposition or independent candidates, with disproportionate improvements to waste services. As displayed in Supplementary Table J3, we fail to find evidence that the treatment effect on waste pile clearance or waste pile sizes is conditional on whether the Division councilor was a member of the NRM ruling party. Distance to KCCA Division Headquarters Part of the appeal of citizen-reporting platforms is the ability to gather information on the status of services in areas that are otherwise costly to monitor. The KCCA might mostly lack information on service delivery in zones that are distant from any of the organization’s five division headquarter offices because residents and zone leaders have less contact with officials or because it is more costly and time-consuming to send scouts to assess conditions in these zones. If monitoring is more difficult in outlying areas, citizen reporting might mostly improve waste services in zones that are distant from KCCA division headquarter offices. However, we find no evidence that the treatment effect of citizen reporting is conditional on either linear distance (Supplementary Table J8) or travel time to the nearest KCCA division headquarters (Supplementary Table J9). Reporting Rates and Message Content Vocal stakeholders often receive the most attention from public agencies. Under public pressure, KCCA officials might have responded disproportionately to treated zones that frequently or consistently report significant shortfalls in waste service provision. We calculate the average response rate across the study period for each treated zone to examine whether frequent reporting is associated with improved waste services among treated zones. We use the content of reports to create zone-level measures of reporting consistency, dissatisfaction with waste services, and waste problem severity, and assess whether improvements are concentrated in treatment zones with reports indicating these characteristics. We find mixed results, though none of the resulting associations are inconsistent with the null hypothesis and signed in the hypothesized direction. For instance, we see that poorer service provision at baseline is associated with a pile not being cleaned by the second post-treatment audit (Supplementary Table J5). Higher levels of baseline dissatisfaction with KCCA services too are associated with a pile not being cleaned by the first post-treatment audit (Supplementary Table J6). Otherwise, we do not find that the amount or content of reports is associated with waste pile size in ways that are inconsistent with the null hypothesis at each post-treatment audit (Supplementary Tables J4–J7).3 Qualitative Results The opportunity to embed part of our research team at the KCCA and interview the staff who interacted with the citizen-reporting platform allows us to assess how several features of this coproduction effort lead to the overall disappointing results: the quality of citizen-produced information; the operating costs of citizen-reporting platforms; the search costs of public service failures; and the alignment of managerial uncertainty and the information citizen-reporting produces. Challenges in each of these areas posed significant barriers to a greater impact of the reporting platform. Reporting Quality and Operating Costs Managers at the KCCA hoped that citizen reporting would produce reliable and consistent information on the locations of waste service shortfalls, which could be used to improve service delivery. In practice, though, information from citizen reporters proved to be both inconsistent and unverifiable. Strategies to process the incoming information increased the cost to operate the platform for the KCCA, undermining the perceived effectiveness of ICT-based citizen reporting among KCCA staff. Within zones, the consistency of citizen reports to the same prompts varied substantially. Figure 9 displays the average consistency of reporters indicating poor or good service quality over the entire study period.4 A consistency of 0.5 means that managers receive an equal number of citizen reports indicating good and bad service quality within a given zone for each question, averaged over the study period. The more inconsistent the reports are within a zone, the more uninformative the reports are about waste conditions for managers. Figure 9. Open in new tabDownload slide Consistency of reports from treated zones in the experimental sample. Along a standardized measure of poor service provision, zones outlined in red indicated that KCCA service provision was poor on average. Zones with darker fills represent zones that inconsistently reported the quality of KCCA services, relative to the zone-level modal response, across all weeks and questions in the study period. Figure 9. Open in new tabDownload slide Consistency of reports from treated zones in the experimental sample. Along a standardized measure of poor service provision, zones outlined in red indicated that KCCA service provision was poor on average. Zones with darker fills represent zones that inconsistently reported the quality of KCCA services, relative to the zone-level modal response, across all weeks and questions in the study period. The mean rate of internal inconsistency for citizen reporting was 21% over the study period. On average, roughly one-fifth of responses disagreed with the modal response direction for each question in each zone. Moreover, 30 zones in the sample produced highly uncertain signals, with an average inconsistency in excess of 30%. This reporting inconsistency might be attributed to citizens accurately observing different waste conditions in different parts of zones, pointing to the need to increase the spatial precision of reports. Alternatively, some reporters might have reported inaccurately based on faulty observations. KCCA staff employed a number of strategies to cope with the inconsistency of incoming citizen reports, including (1) contacting individual reporters who sent consecutive contradictory reports; (2) following up with citizens where reporting inconsistency was high to get additional input on local waste conditions; (3) utilizing staff knowledge of reporting areas to interpret the information from the citizens; and (4) following up with other stakeholders (e.g., speaking with private contractors operating in a zone with inconsistent reporting). This verification process frustrated staff at the KCCA because it substantially increased the operating costs of the platform.5 KCCA staff reported that it took between one and two working days for a staff member to transform the data received in a spreadsheet into actionable information. Since the KCCA adopted the citizen-reporting platform to reduce the overall cost of acquiring information, KCCA management thought that the unexpectedly high processing costs of the reporting platform detracted from its effectiveness as a tool for improving service delivery. Facing these costs, one Solid Waste Officer commented that the KCCA had “hit a dead-end” in terms of verifying information from citizen reporters (Interview B). The verification process also undermined the trust of KCCA staff in citizen reporters. One manager involved with solid waste stated: “The data is not useful because its authenticity or accuracy cannot be verified... Those champions [i.e., the recruited citizen reporters] don’t report to me... they reply according to whatever they want; and to me, that is very wrong.” (interview J). The hostility toward the KCCA expressed in a small number of reports was especially damaging to trust in the quality of the information at the KCCA.6 Beliefs that the monitoring platform had low response rates furthered the perception at the KCCA that the citizen-reporting platform had unjustifiably high operating costs. On most occasions, the KCCA received reports from no more than 12% of citizen reporters enrolled in the program (see Figure 1). Given that the KCCA was billed for every SMS it sent to citizen reporters—even for reporters who failed to respond—several KCCA staff felt that a large portion of the program’s budget was being wasted. This sentiment resonated among high-ranking officials in the KCCA. During a presentation of the results of previous phases of the project by our research team, the high-level management of the KCCA expressly criticized the citizen-reporting program for producing limited information on waste conditions at a high cost. Comparing the monthly cost of citizen reporting to the monthly cost of employing a team of KCCA scouts—informal agency staff who monitored waste conditions on foot throughout Kampala—helps clarify staff concerns about operating costs. One engagement cycle of the SMS-reporting platform cost the KCCA UGX 915,000 ($254 USD), and on average yielded 750 responses from citizen reporters. Over the course of a month, KCCA would go through at least four engagement cycles. Without accounting for the cost of processing and verifying reports, the monthly cost of the citizen-reporting program was approximately 3,660,000 UGX (US$1,016). Complete funding for the 72-person team of KCCA Solid Waste scouts was approximately the same. However, incoming information from Scouts seldom required additional verification or processing, shielding the KCCA from the downstream costs it incurred planning responses to information. Thus, although citizen reporting gave the KCCA access to a broader base of information, the quality of data it generated increased the platform’s operating costs such that KCCA staff believed the platform was an ineffective strategy to improve the delivery of waste services. A member of the management team summarizes this sentiment bluntly: “For me, these messages are very expensive for nothing. That is why I was saying, ‘Why don’t we buy the scouts airtime and communicate on WhatsApp?’” (interview J). Search Costs and the Alignment of Information and Uncertainty At launch, KCCA staff believed that the ICT-based reporting platform would increase the agency’s ability to hold third-party contractors accountable and spread information about proper methods of waste disposal among citizens.7 Acquiring information about these service failures imposed high search costs on the KCCA prior to launching the program, since KCCA staff could not patrol neighborhoods frequently enough either to monitor contractor performance or to measure citizens’ knowledge of waste management.8 We codesigned the citizen-reporting platform with these types of uncertainties in mind. However, the KCCA underwent an unexpected staff restructuring process between May and July 2017. In total, around 120 people were transferred, fired, or newly hired across all branches of KCCA. The KCCA’s waste management unit was not exempt from this process. A number of staff were moved into and out of the unit, and those who remained in the unit typically were reassigned to different roles. The incoming management team played no role in launching the citizen-reporting program, did not trust information from citizen reporters, and did not share broad views about the value of engaging with citizens for non-instrumental reasons.9 The incoming management team also believed that waste conditions in Kampala stemmed from the illegal dumping of waste by citizens, not solely from poor waste contractor performance in delivering services.10 Speaking directly to our theory, the unexpected turnover in staff at the KCCA marked a shift in the type of information deemed essential for management of waste services and thus also the search costs for obtaining a new type of information. The reporting platform we codeployed with the KCCA was designed help the organization reduce the search costs of detecting poor contractor performance and citizen knowledge about waste disposal at scale. Conversely, identifying illegal dump sites is a relatively static information problem, with low search costs, that the incoming team thought could be resolved using scouts.11 The citizen-reporting platform became extraneous when information that could help with the enforcement of disposal standards became the focus of the unit. The type of information the reporting platform produced no longer aligned with the information problem of the new management team, since citizen reporters were not asked to identify illegal dump sites.12 Moreover, the reporting platform no longer meaningfully extended the KCCA’s capacity to collect information, since the management team could communicate directly with scouts over WhatsApp to monitor and sanction illegal dumping. Therefore, the new management team decided to discontinue the citizen-reporting program in favor of expanding the KCCA scout program. Bluntly, a member of the management team stated: “The phone project [i.e., the citizen reporting platform] is not in my needs” (Interview J). The scout program was doubled after the change in management—increasing the size of the program to 200 employees. In addition to investigating illegal dumping, scouts began monitoring general waste conditions and service delivery, effectively subsuming the role of citizen reporters. Discussion and Conclusions ICTs create new spaces for governments and citizens to collaborate on the production of public services. There has been significant optimism that citizens collectively have advantages in producing information in volume and scope that could be used to improve public service delivery (Bertot, Jaeger, and Grimes 2010; Noveck 2017). After all, citizens directly experience services, or the lack thereof, as part of their daily lives. Yet, it is increasingly recognized that the adoption and operation of technologies for coproduction raise a host of managerial, institutional, and political challenges (Gil-García and Pardo 2005; Heintze and Bretschneider 2000; Laffin and Ormston 2013; Liu and Yuan 2015). Despite the tension between these perspectives, there has been a surprising lack of empirical evidence about whether new forms of coproduction, and specifically coassessment, enabled by communication technologies improve public services (see Lember, Brandsen, and Tonurist 2019; Loeffler and Bovaird 2016; Nabatchi, Sancino, and Sicilia 2017). Our study contributes formative evidence about the challenges of using ICT-enabled citizen reporting to coproduce public services. We do not find evidence that zones assigned to citizen reporting experienced reductions in informal waste accumulation. In interviews with KCCA staff and our direct observations, we found that citizen reporters generated inconsistent information about service delivery, requiring the KCCA to adopt an costly, extensive verification process that undermined the perceived effectiveness of citizen reporting among KCCA staff. Moreover, the platform produced information that, after an unexpected turnover in staff, became less relevant to the way that the new management team understood its core information problems. In particular, the problem of waste management came to be viewed as a problem of enforcing rules against illegal dumping by residents, rather than oversight of service provision by contractors. Given the relatively low cost of identifying illegal dumps and the static nature of the information problem, it was little surprise that the KCCA chose to abandon the citizen-reporting platform altogether. Our study is one of the only field studies that systematically and independently tracks the impacts of ICT-enabled coproduction to the actual delivery of a targeted public service. We find similar results to Grossman, Platas, and Rodden (2018), who report that recruiting citizens to send text messages to local politicians in Uganda did not result in significant improvements to public services. It is possible that effects in that study did not emerge because text messages were sent to politicians who do not directly control services, rather than bureaucrats who are responsible for responding in operational ways to complaints. Our study confirms that the same types of problems with inconsistent reporting can lead to disappointing results even when information is sent to bureaucrats directly responsible for services, particularly in a context where managers do not view responding to public input as a main priority. In developing country contexts characterized by significant resource constraints, the immediate, functional goals of citizen reporting are likely to dominate managers’ perceptions of the value of expending resources to respond to citizens. The results of our study highlight the importance of accumulating evidence about different types of coproduction on services across a wider variety of settings. In one of the only studies that tracks coproduction to outcomes, Jakobsen and Andersen (2013) found that equipping parents with tools to support the education of their children in coordination with public schools increased educational attainment. This evidence about “codelivery” of services contrasts with the results of our study that focuses on “coassessment” and may be distinct because the benefits to the codelivery of educational outcomes are largely internal to the family. It is likely significantly more challenging to find ways to impact large-scale, collective, public outcomes like waste management through reporting (Grossman, Platas, and Rodden 2018). The results of our study also contrast with the tentative conclusions of observational studies about citizen reporting for coproduction. Both Sjoberg, Mellon, and Peixoto (2017) and Allen et al. (2019) describe how cities have made real efforts to respond to citizen reporting. Although it is clear that many governments are attempting such approaches, the aggregate results of our study point to the importance of counterfactual research designs. Clearly, more research on supporting conditions is needed. Reporting platforms that solve problems related to information consistency and specificity might overcome challenges brought to light in our study. In addition to theorizing about the conditions under which ICT-enabled reporting might improve services, our study contributes to broader theory about coproduction. First, the challenges governments will face in processing and acting on information gained by citizen reporting have not been fully appreciated. Our study highlights how, when deployed at scale, the massive data citizens generate can overwhelm the processing capacity of public agencies. The variable quality and consistency of data conveyed in citizen reports compounds this challenge, since it requires a greater commitment from agency staff to transform citizen-sourced data into actionable information. Indeed, surveys examining when officials use data to make decisions find that complexity is a key limiting factor (Ammons and Rivenbank 2008; Masaki et al. 2017). Given that managing public services is complex already (Moynihan et al. 2011), the cost of processing and verifying data from citizens might divert resources away from service delivery, making citizen reporting a nonstarter for many public agencies. Our study confirms that many practical considerations, such as perceptions of data quality and the capacity to deal with data complexity, can ultimately stymie models of coproduction that rely on citizen-sourced data (Gil-García and Pardo 2005). Second, our study highlights the importance of managerial and organizational continuity for the effectiveness of efforts to coproduce public services. Instead of treating the adoption of reporting platforms as technical exercises, more emphasis should be placed on gaining broad managerial and staff buy-in within agencies for the goals of coproduction (Meijer and Rodriguez Bolivar 2016). At launch, the platform we tested coincided with a broad reorientation of effort at the KCCA toward responsiveness to citizens. Following a turnover in staff, the new management team discarded the platform because they understood organizational goals more narrowly and focused on reducing illegal dumping. The response highlights how governments are likely to have more success at coproduction when they view citizen satisfaction and responsiveness to public concerns as core to their legitimacy. More evidence is needed on how continuity in government agencies affects the success of coproduction programs generally (see Boyne et al. 2011; Meier and Hicklin 2007). Our study suggests that reasonable continuity in managerial support is a necessary condition for effective coproduction strategies. Third, understanding how ICT-enabled coproduction maps onto the specific information problems affecting the provision of different public services requires further attention. Citizen reporting has the best chance to improve service provision when the search costs of detecting service failures are high. Indeed, the high cost of monitoring private concessionaires motivated the early management team to pursue citizen reporting, and the ability to internally monitor illegal dumping by citizens partially drove the new management team to discontinue the reporting program. When it is costly for public managers to locate the source of service failures, citizen reporting can provide advantages despite some inconsistency. The persistence of participation in platforms that solve these types of problems by both citizens and officials suggests that citizen reporting is best applied to settings where the allocation of effort by managers is very uncertain (see Sjoberg, Mellon, and Peixoto 2017), such as for tasks like detecting dispersed, infrequent service failures. As Afuah and Tucci (2012) argue more generally, crowdsourced information is best applied in settings where organizations can turn problems that involve “distant searching” for information into problems that involve less-costly “local searching” by engaging with the crowd. Our study suggests that it will be most fruitful to apply citizen reporting to detect infrequent and dispersed service failures. Our study also suggests how governments might deploy ICT-enabled platforms for coproduction more effectively in the future. First, our study highlights the need for governments to ensure that public agencies have sufficient capacity to process and act on incoming information. Limited processing capacity not only prevented KCCA staff from fully leveraging the information citizen reporters provided, but also contributed to perceptions that the platform had unjustifiably high operating costs. Working to build processing capacity within the KCCA prior to the platform’s launch, such as hiring additional employees or providing existing employees training on SMS-based reporting, might have reduced the strain that citizen reporting placed on the agency. Future research might consider how concurrent capacity-building programs condition the effect of citizen reporting on service provision. Second, our interviews with KCCA staff emphasize the need to build and maintain buy-in for citizen-reporting programs, both among top-level managers and other agency staff. Building this buy-in may be particularly important when public agencies partner with third-parties to provide services, since the incentives of private contractors might not align with the values underpinning coproduction (Rodriguez Müller and Steen 2019). Although our team codesigned the reporting platform with one management team, an unexpected reorganization of the KCCA brought in a new management team that did not share the collaborative vision of governance underpinning the platform.13 Even staff at the KCCA who were supportive of citizen reporting at baseline later expressed frustration with the platform given the variable quality of information citizens relayed and their inability to verify reports. Ultimately, the perceptions that citizens reported inaccurate, inconsistent, and offensive information to the KCCA contributed to the platform’s termination. Maintaining staff buy-in for citizen-reporting programs may be particularly difficult when the political nature of public service provision places organizations under pressure to improve service delivery quickly. The political mandate for the KCCA to quickly improve solid waste conditions in Kampala probably contributed to perceptions among the incoming management team that citizen reporting was costly and ineffective, relative to strategies based on enforcing rules about dumping. Future attempts to engage in coproduction through ICTs might work to strengthen relationships between citizen reporters and agency staff. This could be accomplished either prescriptively, by providing formal training and specific reporting standards to citizen reporters, or collaboratively, by holding events that increase interpersonal trust between each group. Future research might explore which engagement strategies are most usefully combined with efforts to use ICT-enabled citizen reporting. These lessons should apply broadly to the conduct of coproduction through ICT-enabled citizen reporting. However, there are limits to our study that should be kept in mind as the evidence about this kind of approach to coproduction grows. First, the outcomes that we measured as part of the field experiment focused on short-term service quality, but this may not be the only goal of citizen reporting and coproduction. Information from citizen reporting might be used for long-term planning, to improve relations and trust between governments and citizens, or to more effectively contract with providers of services. Second, our study focused on coproduction in a setting where the available resources to process information and respond to reports were relatively limited. Although this is precisely the kind of setting where ICT-enabled coproduction might decrease the trade-off between acting and gaining information, it is possible that the kind of platform we study is better suited to government agencies with greater capacity. Nonetheless, one positive aspect of the study is that citizen reporting was sustained even in the context of limited government capacity, indicating that opportunities for citizen contributions are plentiful. Finally, during the course of our study an unexpected administrative turnover occurred that could have short circuited the growing benefits of citizen reporting. Indeed, although we see evidence of limited positive effects at the first post-treatment audit, these results do not persist after turnover in the staff of the unit receiving the reports. Citizen reporting might require a sufficient period of stability that allows for “learning by doing” before benefits can be fully achieved. Future research should seek to accumulate evidence both across settings and over longer periods of time. Although engaging citizens to produce information for the management of public services is promising because of the potential to expand the scope and volume of information available to public managers, we find that this promise is likely overstated due to the effort and cost involved in processing and interpreting inconsistent, citizen-sourced data. Citizen reporting will be helpful at improving public services when the data it produces are easy to process, consistent, low cost relative to alternatives, and are brought to bear on well-defined decisions with high degrees of uncertainty about where to act. These conditions are unlikely to exist in many settings currently considered viable candidates for technology-enabled coproduction. Acknowledgments We are grateful to numerous staff and managers at the Kampala Capital City Authority for partnership on this project. Anonymous reviewers and the editor provided insightful comments and suggestions that improved the manuscript. This study was supported by AidData at the College of William and Mary and the US Agency for International Development (USAID) Global Development Lab through cooperative agreement AID-OAA-A-12-00096. The views expressed here do not necessarily reflect the views of AidData, USAID, or the US Government. Activities described in this article received approval from the University of California, Santa Barbara Human Subjects Committee (protocol 5-20-0110), the Uganda Mildmay Research Ethics Committee (protocol 0706-2015), the Uganda National Council for Science and Technology (protocol SS 3840), and the Uganda Offce of the President (ref. ADM/154/212/03). We pre-registered the hypotheses and our plans for testing them at the Evidence in Governance and Politics registry (20151103AA, addendum). Replication materials including data, metadata, analysis code, and data collection instruments are deposited at the following Dataverse: doi: 10.7910/DVN/M1IHH2. References Afuah , A. , and Tucci , C. L. 2012 . Crowdsourcing as a solution to distant search . Academy of Management Review 37 ( 3 ): 355 – 75 . Google Scholar Crossref Search ADS WorldCat Allen , B. , Tamindael , L. E., Bickerton , S. H., and Cho , W. 2019 . Does citizen coproduction lead to better urban services in smart cities projects? An empirical study on e-participation in a mobile big data platform . Government Information Quarterly 37 ( 1 ): 101412 . Google Scholar Crossref Search ADS WorldCat Ammons , D. N. , and Rivenbank , W. C. 2008 . Factors influencing the use of performance data to improve municipal services: Evidence from the North Carolina Benchmarking project . Public Administration Review 68 ( 2 ): 305 – 18 . Google Scholar Crossref Search ADS WorldCat Arceneaux , K. , and Butler , D. M. 2016 . How not to increase participation in local government: The advantages of experiments when testing policy interventions . Public Administration Review 76 ( 1 ): 131 – 9 . Google Scholar Crossref Search ADS WorldCat Baldwin , K . 2013 . Why vote with the chief? Political connections and public goods provision in Zambia . American Journal of Political Science 57 ( 4 ): 794 – 809 . OpenURL Placeholder Text WorldCat Bennett , T. , Holloway , K., and Farrington , D. 2006 . Does neighborhood watch reduce crime? A systematic review and meta-analysis . Journal of Experimental Criminology 2 ( 4 ): 437 – 58 . Google Scholar Crossref Search ADS WorldCat Bertot , J. C. , Jaeger , P. T., and Grimes , J. M. 2010 . Using ICTs to create a culture of transparency: E-government and social media as openness and anti-corruption tools for societies . Government Information Quarterly 27 ( 3 ): 264 – 71 . Google Scholar Crossref Search ADS WorldCat Bhuiyan , S. H . 2010 . A crisis in governance: Urban solid waste management in Bangladesh . Habitat International 34 ( 1 ): 125 – 33 . Google Scholar Crossref Search ADS WorldCat Blair , G. , Littman , R., and Paluck , E. L. 2019 . Motivating the adoption of new community-minded behaviors: An empirical test in Nigeria . Science Advances 5 ( 3 ): eaau5175 . Google Scholar Crossref Search ADS PubMed WorldCat Bovaird , T . 2007 . Beyond engagement and participation: User and community coproduction of public services . Public Administration Review 67 ( 5 ): 846 – 60 . Google Scholar Crossref Search ADS WorldCat Boyne , G. A. , James , O., John , P., and Petrovsky , N. 2011 . Top management turnover and organizational performance: A test of a contingency model . Public Administration Review 71 ( 4 ): 572 – 81 . Google Scholar Crossref Search ADS WorldCat Briggs , R. C . 2012 . Electrifying the base? Aid and incumbent advantage in Ghana . The Journal of Modern African Studies 50 ( 4 ): 603 – 24 . Google Scholar Crossref Search ADS WorldCat Buntaine , M. T. , Nielson , D. L., and Skaggs , J. T. 2019 . Escaping the disengagement dilemma: Two field experiments on motivating citizens to report on public services . British Journal of Political Science . doi:10.1017/S0007123419000322 OpenURL Placeholder Text WorldCat Charalabidis , Y. , Triantafillou , A., Karkaletsis , V., and Loukis , E. 2012 . Public policy formulation through non moderated crowdsourcing in social media . Electronic Participation 156 – 69 . doi:10.1007/978-3-642-33250-0_14. OpenURL Placeholder Text WorldCat Clark , B. , Brudney , J., and Jang , S.-G. 2013 . Coproduction of government services and the new information technology: Investigating the distributional biases . Public Administration Review 73 ( 5 ): 687 – 701 . Google Scholar Crossref Search ADS WorldCat Clark , B. Y. , Zingale , N., and Logan , J. 2017 . Intelligence and information gathering through deliberative crowdsourcing . Journal of Public and Nonprofit Affairs 3 ( 1 ): 55 – 78 . Google Scholar Crossref Search ADS WorldCat Drazen , A. , and Eslava , M. 2010 . Electoral manipulation via voter-friendly spending: Theory and evidence . Journal of Development Economics 92 ( 1 ): 39 – 52 . Google Scholar Crossref Search ADS WorldCat Gerber , A. S. , and Green , D. P. 2012 . Field experiments: Design, analysis, and interpretation . New York: WW Norton . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Gil-García , J. R. , and Pardo , T. A. 2005 . E-government success factors: Mapping practical tools to theoretical foundations . Government Information Quarterly 22 ( 2 ): 187 – 216 . Google Scholar Crossref Search ADS WorldCat Grossman , G. , Humphreys , M., and Sacramone-Lutz , G. 2014 . “I wld like u WMP to extend electricity 2 our village”: On information technology and interest articulation . American Political Science Review 108 ( 3 ): 688 – 705 . Google Scholar Crossref Search ADS WorldCat Grossman , G. , and Michelitch , K. 2018 . Information dissemination, competitive pressure, and politician performance between elections: A field experiment in Uganda . American Political Science Review 112 ( 2 ): 280 – 301 . Google Scholar Crossref Search ADS WorldCat Grossman , G. , Michelitch , K., and Santamaria , M. 2017 . Texting complaints to politicians: Name personalization and politicians’ encouragement in citizen mobilization . Comparative Political Studies 50 ( 10 ): 1325 – 57 . Google Scholar Crossref Search ADS WorldCat Grossman , G. , Platas , M. R., and Rodden , J. 2018 . Crowdsourcing accountability: ICT for service delivery . World Development 112 : 74 – 87 . Google Scholar Crossref Search ADS WorldCat Hansen , M. B. , and Norup , I. 2017 . Leading the implementation of ICT innovations . Public Administration Review 77 ( 6 ): 851 – 60 . Google Scholar Crossref Search ADS WorldCat Heintze , T. , and Bretschneider , S. 2000 . Information technology and restructuring in public organizations: Does adoption of information technology affect organizational structures, communications, and decision making? Journal of Public Administration Research and Theory 10 ( 4 ): 801 – 30 . Google Scholar Crossref Search ADS WorldCat Hock , S. , Anderson , S., and Potoski , M. 2013 . Invitation phone calls increase attendance at civic meetings: Evidence from a field experiment . Public Administration Review 73 ( 2 ): 221 – 8 . Google Scholar Crossref Search ADS WorldCat Irvin , R. A. , and Stansbury , J. 2004 . Citizen participation in decision making: Is it worth the effort? Public Administration Review 64 ( 1 ): 55 – 65 . Google Scholar Crossref Search ADS WorldCat Jablonski , R. S . 2014 . How aid targets votes: The impact of electoral incentives on foreign aid distribution . World Politics 66 ( 2 ): 293 – 330 . Google Scholar Crossref Search ADS WorldCat Jakobsen , M . 2012 . Can government initiatives increase citizen coproduction? Results of a randomized field experiment . Journal of Public Administration Research and Theory 23 ( 1 ): 27 – 54 . Google Scholar Crossref Search ADS WorldCat Jakobsen , M. , and Andersen , S. C. 2013 . Coproduction and equity in public service delivery . Public Administration Review 73 ( 5 ): 704 – 13 . Google Scholar Crossref Search ADS WorldCat Jo , S. , and Nabatchi , T. 2019 . Coproducing healthcare: Individual-level impacts of engaging citizens to develop recommendations for reducing diagnostic error . Public Management Review 21 ( 3 ): 354 – 75 . Google Scholar Crossref Search ADS WorldCat Laffin , M. , and Ormston , C. 2013 . Disconnected communities? ICT, policy learning and the lessons for centrallocal relations . Public Money & Management 33 ( 3 ): 185 – 91 . Google Scholar Crossref Search ADS WorldCat Lember , V. , Brandsen , T., and Tonurist , P. 2019 . The potential impacts of digital technologies on co-production and co-creation . Public Management Review 21 ( 11 ): 1665 – 86 . Google Scholar Crossref Search ADS WorldCat Liao , Y. , and Schachter , H. L. 2017 . Exploring the antecedents of municipal managers’ attitudes towards citizen participation . Public Management Review 20 ( 9 ): 1287 – 308 . Google Scholar Crossref Search ADS WorldCat Linders , D . 2012 . From e-government to we-government: Defining a typology for citizen coproduction in the age of social media . Government Information Quarterly 29 ( 4 ): 446 – 54 . Google Scholar Crossref Search ADS WorldCat Liu , S. M. , and Yuan , Q. 2015 . The evolution of information and communication technology in public administration . Public Administration & Development 35 ( 2 ): 140 – 51 . Google Scholar Crossref Search ADS WorldCat Loeffler , E. , and Bovaird , T. 2016 . User and community co-production of public services: What does the evidence tell us? International Journal of Public Administration 39 ( 13 ): 1006 – 19 . OpenURL Placeholder Text WorldCat Masaki , T. , Custer , S., Eskenazi , A., Stern , A., and Latourell , R. 2017 . Decoding data use: How do leaders source data and use it to accelerate development? Technical report, AidData at the College of William & Mary. http://aiddata.org/publications/decoding-data-use (accessed June 8, 2020). Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC McCubbins , M. D. , and Schwartz , T. 1984 . Congressional oversight overlooked: Police patrols versus fire alarms . American Journal of Political Science 28 ( 1 ): 165 – 79 . Google Scholar Crossref Search ADS WorldCat Meier , K. J. , and Hicklin , A. 2007 . Employee turnover and organizational performance: Testing a hypothesis from classical public administration . Journal of Public Administration Research and Theory 18 ( 4 ): 573 – 90 . Google Scholar Crossref Search ADS WorldCat Meijer , A . 2012 . Co-production in an information age: Individual and community engagement supported by new media . Voluntas 23 ( 4 ): 1156 – 72 . Google Scholar Crossref Search ADS WorldCat Meijer , A. , and Rodriguez Bolivar , M. P. 2016 . Governing the smart city: A review of the literature on smart urban governance . International Review of Administrative Sciences 82 ( 2 ): 392 – 408 . Google Scholar Crossref Search ADS WorldCat Mergel , I. , Rethemeyer , R. K., and Isett , K. 2016 . Big data in public affairs . Public Administration Review 76 ( 6 ): 928 – 37 . Google Scholar Crossref Search ADS WorldCat Moynihan , D. P. , Fernandez , S., Kim , S., LeRoux , K. M., Piotrowski , S. J., Wright , B. E., and Yang , K. 2011 . Performance regimes amidst governance complexity . Journal of Public Administration Research and Theory 21 ( Suppl 1 ): i141 – 55 . Google Scholar Crossref Search ADS WorldCat Nabatchi , T. , Sancino , A., and Sicilia , M. 2017 . Varieties of participation in public services: The who, when, and what of coproduction . Public Administration Review 77 ( 5 ): 766 – 76 . Google Scholar Crossref Search ADS WorldCat Noveck , B. S . 2015 . Smart citizens, smarter state: The technologies of expertise and the future of governing . Cambridge: Harvard University Press . Google Scholar Crossref Search ADS Google Scholar Google Preview WorldCat COPAC Noveck , B. S . 2017 . Five hacks for digital democracy . Nature 544 ( 7650 ): 287 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat O’Brien , D. T . 2018 . The urban commons: How data and technology can rebuild our communities . Cambridge: Harvard University Press . Google Scholar Crossref Search ADS Google Scholar Google Preview WorldCat COPAC O’Brien , D. T. , Offenhuber , D., Baldwin-Philippi , J., Sands , M., and Gordon , E. 2017 . Uncharted territoriality in coproduction: The motivations for 311 reporting . Journal of Public Administration Research and Theory 27 ( 2 ): 320 – 35 . OpenURL Placeholder Text WorldCat Oates , B. J . 2003 . The potential contribution of ICTs to the political process . Electronic Journal of e-Government 1 ( 1 ): 31 – 9 . OpenURL Placeholder Text WorldCat Okot-Okumu , J. , and Nyenje , R. 2011 . Municipal solid waste management under decentralization in Uganda . Habitat International 35 ( 4 ): 537 – 43 . Google Scholar Crossref Search ADS WorldCat Parrado , S. , Van Ryzin , G. G., Bovaird , T., and Löffler , E. 2013 . Correlates of co-production: Evidence from a five-nation survey of citizens . International Public Management Journal 16 ( 1 ): 85 – 112 . Google Scholar Crossref Search ADS WorldCat Pestoff , V . 2006 . Citizens and co-production of welfare services: Childcare in eight European countries . Public Management Review 8 ( 4 ): 503 – 19 . Google Scholar Crossref Search ADS WorldCat Robbins , M. D. , Simonsen , B., and Feldman , B. 2008 . Citizens and resource allocation: Improving decision making with interactive web-based citizen participation . Public Administration Review 68 ( 3 ): 564 – 75 . Google Scholar Crossref Search ADS WorldCat Rodriguez Müller , P. , and Steen , T. 2019 . Behind the scenes of coproduction of smart mobility: Evidence from a public values’ perspective . In Electronic government. EGOV 2019 . Google Scholar Crossref Search ADS Google Scholar Google Preview WorldCat COPAC Rotberg , R. I. , and Aker , J. C. 2013 . Mobile phones: Uplifting weak and failed states . The Washington Quarterly 36 ( 1 ): 111 – 25 . Google Scholar Crossref Search ADS WorldCat Saxton , G. D. , Oh , O., and Kishore , R. 2013 . Rules of crowdsourcing: Models, issues, and systems of control . Information Systems Management 30 ( 1 ): 2 – 20 . Google Scholar Crossref Search ADS WorldCat Seltzer , E. , and Mahmoudi , D. 2013 . Citizen participation, open innovation, and crowdsourcing: Challenges and opportunities for planning . Journal of Planning Literature 28 ( 1 ): 3 – 18 . Google Scholar Crossref Search ADS WorldCat Sjoberg , F. M. , Mellon , J., and Peixoto , T. 2017 . The effect of bureaucratic responsiveness on citizen participation . Public Administration Review 77 ( 3 ): 340 – 51 . Google Scholar Crossref Search ADS WorldCat Stipak , B . 1980 . Local governments’ use of citizen surveys . Public Administration Review 40 ( 5 ): 521 – 5 . Google Scholar Crossref Search ADS WorldCat Thomsen , M. K. , and Jakobsen , M. 2015 . Influencing citizen coproduction by sending encouragement and advice: A field experiment . International Public Management Journal 18 ( 2 ): 286 – 303 . Google Scholar Crossref Search ADS WorldCat Uganda Bureau of Statistics . 2017 . National population and housing census 2014 area specific profiles: Kampala Capital City authority . Technical report Uganda Bureau of Statistics. https://www.ubos.org/wp-content/uploads/publications/2014CensusProfiles/KAMPALA-KCCA.pdf (accessed June 8, 2020). Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Vermeiren , K. , Van Rompaey , A., Loopmans , M., Serwajja , E., and Mukwaya , P. 2012 . Urban growth of Kampala, Uganda: Pattern analysis and scenario development . Landscape and Urban Planning 106 ( 2 ): 199 – 206 . Google Scholar Crossref Search ADS WorldCat Yetano , A. , and Royo , S. 2017 . Keeping citizens engaged: A comparison between online and offline participants . Administration & Society 49 ( 3 ): 394 – 422 . Google Scholar Crossref Search ADS WorldCat Zurovac , D. , Talisuna , A. O., and Snow , R. W. 2012 . Mobile phone text messaging: Tool for malaria control in Africa . PLoS Medicine 9 ( 2 ): e1001176 . Google Scholar Crossref Search ADS PubMed WorldCat Footnotes 1 Supplementary Appendix B details our recruitment protocol for citizen reporters. 2 As one Waste Management Officer reported: “My area of supervision contains 23 parishes and over 200 zones. It is impossible for me to be in all those places at the same time. The citizen-monitors enable me to keep tabs in those areas by keeping me up-to-date with what is going on” (interview I). 3 We operationalize “severity of waste management problems” in Supplementary Table J5 from responses to a prompt asking citizen-monitors to report if a rubbish-collection truck visited their neighborhood. Possible responses include: yes, no, don’t know. The latter two responses were coded as indicative of severe waste management problems. We additionally operationalize waste management problem severity using citizen-monitor reports commenting on rubbish burning, litter and illegal piles, rubbish spilling from KCCA trucks, and mistreatment by KCCA waste collectors. 4 The categorical measure of poor service provision combined the following indicators: the frequency and accessibility of service provision, reported waste collector treatment of citizens outlined, and receipt of waste services, and satisfaction with waste services. 5 Even Solid Waste Officers at the KCCA who were generally supportive of the platform at baseline acknowledged that the poor quality of citizen reports undermined the cost-effectiveness of the citizen monitoring. One Solid Waste Officer commented “sometimes, they’re [the messages citizen reporters send] are not genuine” (Interview H); while another commented “sometimes it [the platform] does not give us very accurate information” (Interview B). 6 One member of the management team commented: “I find it difficult to act on such messages when some are even insulting” (interview J). 7 Commenting on the usefulness of the platform and the information it provided, one Solid Waste Officer stated: “When this information is given out to the people…[they] can suggest ways how we can really change the systems of waste collection. And then, it helps people to come on board that these private people are mandated to do their work efficiently. So, a person can tell what time are they supposed to be at my door to collect my garbage. It helps the person to know, and to be very streamlined, in terms of waste storage...There is actually a lot that people didn’t know, that this portal helps them to know.” 8 One Solid Waste Officer stated: “Now in [my area of supervision] you have about 22 parishes, you can’t move everywhere. But at least if you can get respondents from each, you know what is happening there, because sometimes you may not reach everywhere” (Interview H). 9 Commenting on the usefulness of the monitoring platform, one member of the management team stated: “I cannot tell whether the message which is sent is genuine. Where somebody is not being paid, even if they give you wrong information, how do you track?” (Interview J). 10 During our interview, a member of the management team stated a need to “find illegal dump sites” and “track the dumping” and detailed that the main uncertainty was about how to arrest suspects of illegal dumping: “After dumping, if I get the suspects, how do I pick them?” (Interview J). 11 On the use of scouts for tackling illegal dumping, one manager stated: “I have a problem of illegal dumping. And I have my scouts. When they find a suspect, they use WhatsApp to send a message, I send a car to pick the suspect and take them to court” (Interview J). 12 A member of the management team stated with discontent that they had “never seen a suspect being reprimanded” for illegal dumping using information from the citizen-reporting platform (Interview J). 13 In fact, a member of the new management team “believe[d] that there was a problem with the consultation [for the platform]” since it did not serve efforts to curb illegal dumping (Interview J). © The Author(s) 2020. Published by Oxford University Press on behalf of the Public Management Research Association. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com. This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)
TI - The Challenges of Using Citizen Reporting to Improve Public Services: A Field Experiment on Solid Waste Services in Uganda
JO - Journal of Public Administration Research and Theory
DO - 10.1093/jopart/muaa026
DA - 2020-04-14
UR - https://www.deepdyve.com/lp/oxford-university-press/the-challenges-of-using-citizen-reporting-to-improve-public-services-a-lhdumuM4ad
DP - DeepDyve
ER -