The potential and pitfalls of collaborating with development organizations and policy makers in Africa

The potential and pitfalls of collaborating with development organizations and policy makers in... Abstract A growing number of academics are engaging in collaborative research projects with development organizations and policy makers. Increasingly, this includes efforts to co-produce research, rather than simply share information. These new ways of doing research raise important ethical and practical issues that are rarely discussed but deserve attention. Sub-Saharan Africa is the region of the world in which these new approaches are particularly prevalent, and one where the challenges created by those approaches tend to manifest in distinct or acute ways. In this Research Note, we draw on a collaborative research project with the Westminster Foundation for Democracy to illuminate these difficulties. We also offer suggestions for how to manage the challenges that arise when academics conduct research with policy makers and development organizations. Ensuring that such collaborations are both effective and ethical is not easy, but it must be done if we are to develop better informed policy and scholarship. In the last decade, academic researchers and development organizations such as bilateral aid agencies, multilateral organizations, and international NGOs have found themselves working together more frequently than in the past. Sometimes this has been the product of genuine enthusiasm about the opportunities offered by collaboration. In other cases, it has been the product of outside pressures, particularly from funders and review bodies, an issue we return to later. Researchers' collaborations with development organizations and policy makers have also changed in nature. Previously, scholarly policy engagement tended to either take the form of academics being employed as private consultants outside of their university roles, or policy makers funding academic work in areas of specific interest. By contrast, the last decade has witnessed the proliferation of new collaborations that go beyond simply evaluating completed projects or answering specific questions. In an increasing number of cases, research is now being integrated into development projects from the outset, with researchers playing an active role in shaping interventions. In these cases, development organizations and policy makers, such as the United Kingdom’s Department for International Development (DFID), have worked with teams of researchers to design and implement the monitoring and evaluation of major interventions.1 In one such project, researchers from Columbia University examined the impact of scorecards in monitoring the performance of Members of Parliament and promoting accountability in Uganda.2 In programmes like this one, academics are involved from a very early stage and help to make decisions that shape policy and practice in profound ways. Thus, what is distinctive about this latest phase of collaboration is its scope and institutional embeddedness: many of the new partnerships run for years, involve teams of researchers rather than an individual, and are co-hosted within universities. This greater entanglement of the academic and policy worlds gives rise to the ethical and practical challenges we discuss in this Research Note. The trend towards more frequent and more extensive research collaboration between academics, development organizations and policy makers is not confined to sub-Saharan Africa, though as we discuss below, it raises some particularly thorny issues in that context. Nor is this trend confined to the social sciences; if anything, the social sciences are late to the party. Partnerships between academic researchers and development organizations have a long history in more technical fields such as global health, agriculture and sanitation, though those areas have also witnessed an increase in such collaborations in the last few years.3 We focus on political science and development because the literature is less well developed in this area, and it is where we have the most evidence and experience. The growing number of collaborations has been incentivised by the pressure that funders place on academics, particularly in the United Kingdom, to achieve research impact, and on practitioners to demonstrate the efficacy of their policies, a point we explore in greater detail below. Of course, satisfying the demands of funders is not the only reason to team up. Collaborative research offers many potential benefits. While those working within development organizations gain access to expert advice, analysis, and cutting edge methodological techniques, academics gain access to fresh sources of evidence, opportunities to test novel ideas, and new routes through which their research might influence practice and policy-making. Indeed, at first-glance, collaborations between academics and development practitioners look like a win–win proposition.4 However, such projects also generate risks and challenges. Academics and practitioners do not necessarily have the same objectives and priorities. Nor do they always work within the same time horizons. Out-dated stereotypes as to what ‘academic’ and ‘policy’ work entails can also hamper effective collaboration.5 Perhaps more importantly, collaborations with development organizations and policy makers raise serious structural and systemic risks, including those of structural bias, losing objectivity and deviating from ethical standards relating to consent. While some of these risks arise regardless of where research is carried out, others manifest in distinct or particularly acute ways when research is conducted in the African context because of existing inequalities in the production of knowledge about the continent. As a recent article in African Affairs demonstrated, despite democratization and the ‘Africa rising’ narrative, the proportion of journal articles published on the continent that are written by those living in it has fallen in recent years.6 Given this, there are difficult questions to be asked about who benefits when Africa is used as a laboratory for social science, how collaborative research can avoid perpetuating power imbalances entrenched by colonialism, and whether encouraging researchers to work more closely with development organizations will further marginalize African voices in the production of knowledge about the continent. In this Research Note, we use the Political Economy of Democracy Promotion Project7 as the starting point for a more detailed examination of these issues. This research programme is a collaboration between the International Development Department of the University of Birmingham and the Westminster Foundation for Democracy, the United Kingdom’s leading democracy assistance organization. It aims to identify the conditions under which democracy promotion activities are successful and to explain how such interventions can be improved. Using this project as an entry point, we explore the challenges that arise when academics collaborate with policy makers and development organizations. In order to structure the discussion, we distinguish between ethical and practical challenges, though we recognize that there is often overlap between these; many ethical problems have a practical dimension, while practical issues often raise ethical dilemmas. We do not claim to have ‘solved’ all of the issues that we identify in our project. Indeed, one of the main arguments of this Research Note is that many of these issues cannot simply be avoided or wished away; rather, they need to be constantly managed. Nor will the solutions that we have adopted be appropriate in all cases. Nevertheless, we believe that our project offers useful insights into the kinds of challenges that such collaborations generate, and how they can be tackled more effectively. The motivation behind collaboration The increase in both the frequency and the depth of collaboration between academics and development organizations has been driven on both sides by the demands of funders.8 On the academic side, scholars face increased pressure to explain how they will create ‘pathways to impact’ for their research. This trend has been most pronounced in the United Kingdom, where the provision of funding is now contingent on the provision of such an explanation,9 but similar policies have been adopted by funders in other countries such as Australia10 and Ireland.11 These kinds of requirement make it clear that funders expect proposals to set out convincing strategies and ring-fenced budgets for influencing policy makers and practitioners working in relevant fields. Such strategies are generally seen as more convincing when stakeholders close to the problem under scrutiny ‘are actively involved in the research … through the whole life cycle from initiation, design, progression, knowledge exchange and application of the research.’12 In the United Kingdom, the emphasis on research impact goes even further; academics are now formally evaluated on the policy impact of their work under the government’s Research Excellence Framework. This requires every academic department in the country to write an ‘Impact Environment Statement’ and provide ‘Impact Case Studies’ detailing projects with significant influence. In 2014, the vast majority of such case studies focused on ‘impacts’ in developed countries; researchers submitted 4,596 Impact Case Studies whose impact was located in Europe, while only 685 were based in Africa.13 However, this figure is relatively high when one takes into account the fact that only a small minority of research conducted within UK higher education institutions focuses on Africa. Moreover, we expect the number of Impact Case Studies on the continent to rise in the future given moves to channel a significant amount of UK Official Development Assistance to UK Research Councils via the Global Challenges Research Fund. On the other side, development organizations face mounting pressure from their funders and an increasingly critical media to demonstrate two things. First, that the interventions they deliver are based on evidence. Second, that those interventions have a demonstrable (and ideally, measurable) impact on development outcomes. Among funders, DFID has worked particularly hard to build a reputation as an organization that generates, and uses, high-quality and robust evidence about what works in international development, investing heavily in both external research and its internal evaluation systems. In some cases, projects come with specific guidelines regarding expenditure on monitoring and evaluation, for example that such activities should receive around 10 percent of the overall budget. This investment in research and evaluation has been particularly marked in relation to Africa. In 2014, an internal review conducted by DFID found that its programmes in Africa were more likely to be evaluated, either by DFID’s internal evaluation team or by independent experts commissioned by DFID, than programmes in any other region.14 While DFID’s investment in research and evaluation is distinctive, it is by no means unique. Over the last few decades, there has been a marked increase in the number of published impact evaluations of international development interventions, almost a third of which have taken place in Africa.15 In 2015, the US Agency for International Development (USAID) launched the Monitoring, Evaluation, Research and Learning Innovations Program. This programme is designed to allow USAID and its partners ‘to source, co-create, and co-design development solutions that innovate on traditional approaches to monitoring, evaluation, research and learning.’16 The fiscal constraints faced by many traditional donors, together with the election of leaders such as President Trump who exhibit clear scepticism about the efficacy and ideological justification for development aid, mean that the pressure on aid and educational budgets is unlikely to ease any time soon. As a result, academics, policy makers and practitioners will continue to face strong incentives to join forces for some time to come. Critical friends, not cheerleaders Our collaboration with the Westminster Foundation for Democracy (WFD) started in late 2015.17 WFD’s contribution to the project includes a funded post-doctoral position and the provision of access to three things: (i) internal WFD documents (including programme proposals, reports and evaluations); (ii) people (through formal interviews and informal exchanges, the latter facilitated by the presence of the post-doctoral researcher at WFD’s London office on a semi-regular basis); and (iii) events (WFD programme activities). In exchange, the academic research team comprised, at present, of the authors of this Research Note have produced a series of policy papers, some of which have formed the basis of academic articles. One of the explicit aims of the project is that the academic research will help to strengthen the approach of WFD. While the project has given rise to many gains, it also generates a number of ethical risks. One of the most obvious ethical risks that emerge in long-term collaborations like this is that academics will be co-opted, losing their objectivity. To do good collaborative research, academics need to find a way of being critical friends to practitioners, rather than cheerleaders. Evaluations commissioned by development organizations, and written by academics acting as consultants, are inevitably approached with a degree of scepticism because they tend to pull their punches. This is often true in sub-Saharan Africa, where the acknowledged difficulty of promoting development or democracy makes it tempting to tone down criticism of practitioners. Overly generous evaluations are especially likely when a researcher’s continued employment is contingent on their partner organization being happy with the research they produce, or when researchers are sympathetic to the goals of development interventions. The latter represents a particular challenge in collaborative work because researchers are unlikely to partner with development organizations unless they accept at least some basic assumptions made by those organizations, including the assumption that such organizations ought to intervene to promote development. This may make a degree of bias inevitable. There are no hard and fast rules that can ward off favouritism, especially where it is unconscious, but there are ways to reduce the problem. Our project with WFD provides one example of what this might look like in practice. While WFD funds a post-doctoral position linked to the project, the post-doctoral fellow is not employed directly by WFD. This ensures that the university, rather than WFD, is responsible for assessing their performance and making any decisions about their continued employment. In addition, while the post-doctoral fellow works in the WFD office on as semi-regular basis to facilitate data collection, the majority of their work is conducted at locations outside of WFD. This helps to ensure a degree of critical distance, as does the fact that the remainder of the project team (including the academic lead) is not embedded within WFD. More generally, it is important that the lead academic on such projects (often called the Principal Investigator) does not receive a direct financial benefit from their involvement in the form of a salary increase or other perks. This establishes a degree of formal independence that can prevent institutional capture. It is also advisable that the outputs of any such collaboration are reviewed by an advisory board of individuals who have no ties to the organization or projects being evaluated, providing an external check on the process. If these rules are followed, the risk of producing biased findings can be significantly reduced, though not eliminated. It is therefore important to also maintain transparency at all times, and clearly state that research outputs are the product of collaborative projects. Our experience also suggests that academics should avoid assuming that development practitioners want research to be biased in their favour. Indeed, some practitioners are just as worried as academics about the potential loss of objectivity inherent in this kind of research. In our case, a significant percentage of mid-level and senior staff at WFD have doctorates, which has fostered an organizational culture that casts objective research as a good thing. WFD’s staff tend to see research as a way of generating more robust evidence that will allow them to demonstrate the value of their work at a time when the United Kingdom’s aid budget is under increasing pressure. They know that evidence will only be convincing if it is produced in an objective and transparent way. In other words, most WFD staff believe they are better off having critical friends, rather than cheerleaders. This is not to say that our research collaboration has been perceived in a positive manner by every single person linked to WFD. Initially, there were some who clearly felt uncomfortable with too many details of their projects being made public. We found practical ways to manage these concerns. Sometimes this was as simple as explaining that we did not have to ‘name names’—anonymity was an option. In other cases, we circulated drafts of research outputs (specifically, policy papers) for comment in advance. This was done on the understanding that while we would not change our findings, we could clarify certain points if staff felt the factual details of their programmes had been misrepresented. Unsurprisingly, we have found that practitioners are far more tolerant of criticism when they have had a chance to discuss and respond to it prior to publication. Leading the way in methodological innovation, or turning Africa into a laboratory? A central question that faces research in the African context is who benefits: researchers or research participants? As Helen Tilley has argued, Western scholars have a long history of using Africa as a ‘living laboratory’ to produce research, some of which helped to maintain colonial empires.18 The San people of Southern Africa, a population that is frequently the subject of academic research, recently launched their own code of ethics for researchers, partly due to dissatisfaction with the benefits that research has provided to their community.19 How to conduct research in a fair and ethical way is a particularly pressing issue given the rising number of collaborative research projects that use Africa as a site for experimental methods, including randomized controlled trials (RCTs) and ‘laboratory’ techniques.20 Between 1995 and 2013 four African countries—Kenya, Malawi, Uganda and Ghana—accounted for around 25 percent of all RCTs conducted in non-OECD countries and published in the top 25 economics and political science journals, or listed on the websites of major organizations supporting RCTs in development economics.21 At the same time, African examples have become particularly prevalent in field experiments linked to democracy and governance programmes, which typically require academics to work collaboratively with practitioners. When Devra Moehler compiled a (non-exhaustive) list of such projects in 2010, she found that just over half of them took place on the continent.22 While it is good to see African studies at the forefront of methodological innovation, the growing popularity of these methods has proved controversial, triggering renewed debates about what constitutes ‘good research’, along with concern about ethical and moral issues. Though experimental methods such as RCTs have advantages, they raise their own methodological challenges23 and have important limitations that are often overlooked.24 For example, while RCTs are good for testing whether an intervention achieved its stated goal, they often fail to identify the unintended consequences of development programmes.25 Some also fear that the spread of experimental methods may marginalize qualitative research, which remains essential regardless of advances on the quantitative front,26 and is the most common form of research conducted within African universities.27 Collaborative research between academics and development organizations has the potential to exacerbate concerns that Western scholars have reduced Africa to a ‘testing ground’ for their theories. The majority of the policy/research collaborations established in the United States or United Kingdom are led by American and British academics and some lack African involvement at the top levels. There is also evidence that the limited resources available to many African universities discourages collaboration between development organizations and Africa-based researchers, as do (often unfounded) concerns among some policy makers about the ‘usability’ of the research that the latter produce.28 However, it is also true that bodies such as DFID have invested in forming three-legged partnerships with African institutions. We suspect that in many cases uneven representation results from the ease of working with researchers who are close by, the desire to partner with prestigious institutions, and in some cases the imperative of spending a proportion of budgets within the home country of the funder. The limited presence of Africa-based researchers in the management of large projects is problematic for a number of reasons. Most notably, while a lack of local knowledge may undermine research and policy outputs, African researchers miss out on access to privileged information and funding to collect new data. The latter is of particular concern given the extremely limited core funding that is available to research institutions on the continent. The exclusion of African scholars from these opportunities also means that collaborations between academics and development organizations might further marginalize African researchers in disciplines in which they are already underrepresented.29 An analysis of the impact evaluations of international development interventions published between 1981 and 2012 found that over time the share of authors with institutional affiliations in North America and Western and Northern Europe increased but the proportion from sub-Saharan Africa declined.30 While impact evaluations and collaborations are not synonymous, this trend suggests that collaborations between researchers and development organizations may have undesirable side-effects. One way to deal with this is to ensure that such collaborations include ring-fenced funds to support research partnerships with Africa-based organizations. However, such research partnerships will need to be carefully designed as they raise a number of ethical issues in themselves.31 In particular, the fact they are funded and designed in the West creates a significant risk that they will simply perpetuate imbalanced relationships between Western and African researchers.32 Recent survey and interview-based evidence indicates that in existing north–south research partnerships the role of southern researchers is often limited to collecting data for country case studies while northern researchers take the lead in identifying research questions, deciding on research methods, conducting cross-country analysis and—as a consequence—authoring publications.33 In light of this, collaborative research partnerships will need to consciously prioritize the co-development of research questions, the co-design of research methods, the equal division of research tasks, and the co-authorship of publications. The fact that the development agencies of Western governments fund a significant proportion of the RCTs and other experimental research projects also raise concerns that academics are becoming increasingly involved in promoting external interests on the continent. Even when research is intended to benefit African citizens, some question whether—by feeding into the policies of Western donors, rather than those of African states—it has become a new ‘tool for indirect rule’.34 This is a valid question to ask of many collaborations, including our project; WFD is engaged in the kind of democracy support programmes that are sometimes attacked by African governments as an infringement on sovereignty and an attempt to impose foreign values. How can such concerns be mitigated? One potential avenue through which to do this is to create more space for the voices of programme beneficiaries—and critics—to be heard. While the intended beneficiaries of development programmes are routinely surveyed to provide feedback, such checks often represent a token exercise that does little to speak to the concerns of African citizens. This has been a problem for WFD, not because WFD does not care about this kind of feedback, but because it is exceptional for beneficiaries to provide direct criticism of WFD activities. Workshop participants asked to identify which session they found ‘least useful’ will typically (sometimes, unanimously) report that there ‘was no least useful session’. This reluctance to give negative feedback is—we suspect—partly due to politeness, and partly due to a worry that doing so would harm the potential for future engagement. Although this challenge is hard to overcome, where research involves deeper collaboration in the form of co-production, academics can use different techniques such as longer interviews, focus groups, and anonymous questionnaires to elucidate more accurate responses, and thus improve the way in which programmes are designed, delivered and evaluated. Academics introduced as being independent from, but working with, development practitioners may also be better placed to uncover alternative perspectives. However, this potential ‘solution’ raises a further ethical challenge regarding the quality of consent. Protecting the quality of consent In many academic codes of conduct, receiving informed and genuine consent from research subjects is one of, if not the most, significant component in carrying out ethical research.35 It is essential to ensure that all participation in research activities is voluntary and that research participants never feel coerced, something that will often require care and forethought when research is conducted in collaboration with development organizations. This is because such bodies typically act as gatekeepers of resources and opportunities by determining who will be included in their projects. Ethics guidelines often stress that responsibility for obtaining informed consent cannot be devolved to gatekeepers.36 Moreover, most development programmes—political or otherwise—are designed to provide benefits to those included. Many take place in the context of pre-existing or ongoing relationships between development organizations and programme beneficiaries. This creates a risk that individuals will feel obliged to participate in research lest they miss out on the benefits offered by development interventions. This is a concern because most (possibly all) guidelines also direct researchers to ensure that those who decline to participate in, or withdraw from, research activities do not suffer any negative consequences as a result. In the context of collaborations between researchers and development organizations, this means that there should be no connection between participation in research and eligibility for development project ‘benefits’. Where possible, the largest portion of the research should be conducted once a development intervention is finished, when there are no future benefits to be lost. However, when researchers seek consent from individuals who are receiving—or hope to receive—benefits from a related project, separating the role of the development organization from that of the academic, and the benefits of the project from voluntary participation in research, may become extremely difficult. For us, this issue has—so far—not been as severe because our research has focused on individuals who are part of Africa’s political elite: Members of Parliament and leaders of prominent civil society groups. Such individuals are generally more educated, assertive, and wealthy, and therefore less prone to feeling pressured to join research projects that run side-by-side with development interventions. Nonetheless, we are careful to stress that refusal to grant us an interview or fill in a survey has no bearing on the distribution of WFD resources. In other situations, the problem is more acute. For example, researchers investigating the impact of development assistance delivered by the International Rescue Committee on collective action in Liberia quickly recognized that individuals in relatively poor and isolated communities might feel obliged to participate in order to access desperately needed services.37 In this instance, waiting to invite communities to participate in the research component of the programme until after the community development projects delivered by the International Rescue Committee were formally complete was essential to ethically securing voluntary consent.38 Even when research takes place after development interventions are complete, it is possible that individuals might feel obliged to participate because of the benefits they have already received. This makes it important to distinguish between the research team and the team responsible for implementing a development intervention. Having these two groups undertake their work at different times and in different venues helps. However, the distinction between them may not be obvious to outsiders unless researchers take steps to make it more visible. A field experiment led by Paul Collier and Pedro Vicente in Nigeria provides a useful example of how this might be done.39 For their research, Collier and Vicente collaborated with ActionAid International Nigeria to examine the impact of a campaign against electoral violence that was designed to encourage participants to oppose voter intimidation. Distinct research and campaign teams operated independently, save for the baseline survey conducted prior to ActionAid International Nigeria’s campaign, during which one member of the campaign team accompanied the research team to identify the research sites. The two teams took care to use different ‘branding’ to ensure the distinction between them was clearly visible to the communities in which they were operating. While the campaigners wore t-shirts and caps branded with the anti-violence campaign, survey enumerators wore name tags and displayed the credentials of the survey company. As with our other suggestions, these kinds of strategies will not eradicate the problem, but they will enable researchers to better manage it. Practical challenges in collaborative research In addition to ethical concerns, collaborative research raises practical challenges, though in many cases the boundaries between the two are blurry. One (primarily) practical challenge is that of reconciling good research design with good programme design. The two are not necessarily synonymous. Most comparative research designs rely on variation in the dependent variable (what is to be explained),40 which can create problems because funders and development organizations typically want to focus their resources on maximizing the chances of success. They may be reluctant to include ‘control’ cases, or to run projects that are unlikely to succeed, making it difficult to generate the variation desired by academics. Indeed, many development organizations have deeply entrenched cultures of risk aversion, a fact pointed out by the former head of USAID.41 This attitude to risk has significant implications for what academics can study in collaboration with policy makers. In some cases, topics that are of interest to academics may be off-limits because they do not appear to be viable to development organizations. In other cases, practitioners’ interest in increasing the impact of a programme by ‘bundling’ a number of activities together may sit in tension with researchers’ desire to disaggregate the effect of each component of an intervention.42 The tensions between good research design and good programme design are not insurmountable, but may require a new approach. Development organizations are increasingly alert to the fact that seeking to avoid all risk is not only impossible, but often counter-productive. In 2013, the World Bank declared an intention to shift from an institutional culture of extreme risk aversion to one of informed risk taking.43 Growing concerns about both terrorism and migration have led to greater recognition of the fact that difficult cases cannot simply be ignored. These changes may create opportunities to reconcile research and programme design, but taking them will require academics to persuade their collaborators that particular risks are worth taking in order to gain greater analytical leverage. In the African context, this is particularly significant because many of the most pressing development challenges occur in countries where the political context generates the greatest risk of failure, such as the Central African Republic, Sudan, and Zimbabwe. Another practical problem that can arise in these kinds of collaborations is that of time horizons. Practitioners tend to need answers to specific, time-bound problems, while academic research is an ongoing, iterative process.44 When academics ask more ambitious and complex research questions, it may be impossible to specify how long it will take to get answers. This often leads to an assumption that practitioners have much shorter time horizons than academics, rendering certain research topics unviable. However, our experience suggests that the gap between academic and practitioner time horizons is not always as great as is assumed; this issue is often one of organizational constraints rather than attitude. In the case of our project, WFD staff consistently emphasized the importance of taking a long-term approach to programme design and evaluation. The real challenge for us has not been that WFD wants answers as soon as possible, but that our attempts to find those answers need to fit into WFD’s programming and funding cycles. These place constraints on when programmes (and thus research) can start, and on how long they can run. This explains why our collaboration is only gradually moving from policy papers that focus on a retrospective evaluation of WFD programmes and strategy, to more forward-looking research that is integrated into programmes from their inception through to implementation and eventually evaluation. Overcoming the constraints imposed by programming and funding cycles is not something that academics can do on their own. Instead, it requires a longer conversation with funders and policy makers about how budgets and timeframes can be better designed to allow for longer-term research projects that are not undermined by arbitrary milestones or inappropriate deadlines. Making collaborative research work As our discussion of ethical and practical issues makes clear, it is not easy to make collaborative research work, but it is important if we are to make progress in terms of deepening our understanding of issues such as the effectiveness of international development and democracy programmes on the continent. We have identified a number of ways in which the ethical challenges involved in such projects can be managed. These include prohibiting personal gain to lead researchers, employing an independent advisory board, amplifying the voice of beneficiaries and critics, establishing balanced research partnerships with Africa-based organizations, and taking steps to ensure that consent is not coerced. However, these measures will count for little unless they occur within the context of a legal agreement or memorandum of understanding that guarantees academics intellectual property rights over their research. Academics are often asked to sign contracts that grant them certain intellectual property rights, but also circumscribe those rights. This is frequently done through non-disclosure agreements, and by requiring researchers to submit publications to policy partners for review before publication. Such constraints are particularly common in work that involves a security aspect, such as research conducted for the United States Africa Command, which is one of a number of reasons why such projects are often viewed as particularly problematic.45 Here, balance is key. It is understandable for development organizations and policy makers to seek some measure of influence over how the research they help to produce is used, particularly with respect to confidential information that they and their partners make available. Contracts that prohibit independent publication of research findings might be tolerated by researchers acting as paid consultants outside of their university role.46 However, in co-produced research and institutional collaborations, it is essential to maintain the freedom of academics to publish their findings in peer-reviewed journals, even when they cast the programmes of partners in a less than favourable light. Contracts that require some degree of consultation prior to publication may be reasonable. Contracts that give development organizations a veto over what can be published should be avoided. Indeed, the UK’s Research Integrity Office advises academic institutions to ‘protect researchers from inappropriately restrictive or coercive contracts that, for example, prevent researchers from publishing work’.47 Given the existence of such ‘red lines’, academics must be judicious when entering into any new research collaboration. We have been very fortunate in our collaboration to have a partner whose organizational culture values independent research while tolerating constructive criticism. The fact that we have such a partner is not a matter of chance, however. We only entered into the collaboration because it was clear to us that WFD was genuinely committed to academic freedom, as evidenced by the fact that we were granted full intellectual property rights and not asked to sign a non-disclosure agreement. As we have discussed above, there are a number of practical steps that can be taken to help make collaborative research work in an effective and ethical manner. Ultimately, however, the most important step is deciding who—and who not—to work with. Footnotes 1. DFID, ‘Review of embedding evaluation in the Department for International Development’ (London, DFID, 2014). 2. African Leadership Initiative, ‘Parliamentary scorecard 2009–2010: Assessing the performance of Ugandan legislators’ (African Leadership Institute, Kampala, 2011). 3. Catherine Olivier, Matthew R. Hunt, and Valéry Ridde, ‘NGO–researcher partnerships in global health research: Benefits, challenges, and approaches that promote success’, Development in Practice 26, 4 (2016), pp. 444–55. 4. Laura Roper, ‘Achieving successful academic-practitioner research collaborations’, Development in Practice 12, 3–4 (2002), pp. 338–45. 5. William G. Moseley, ‘Collaborating in the field, working for change: Reflecting on partnerships between academics, development organizations and rural communities in Africa’, Singapore Journal of Tropical Geography 28, 3 (2007), pp. 334–47, p. 337. 6. Ryan Briggs and Scott Weathers, ‘Gender and location in African politics scholarship: The other white man’s burden?’, African Affairs, 115, 460 (2016), pp. 466-489. 7. Further details about this project are available on the project website https://democracypromotion.wordpress.com. 8. A similar observation is made by Daniel Stevens, Rachel Hayman, and Anna Mdee, ‘‘Cracking collaboration’ between NGOs and academics in development research’, Development in Practice 23, 8 (2013), pp. 1071–77. 9. Research Councils UK, ‘Pathways to Impact’, Research Councils UK, <http://www.rcuk.ac.uk/innovation/impacts/> (27 April 2017). 10. Australian Research Council, ‘Research impact principles and framework’, Australian Research Council Website, <http://www.arc.gov.au/research-impact-principles-and-framework> (8 May 2017). 11. Science Foundation Ireland, ‘Research impact’, Science Foundation Ireland Website <http://www.sfi.ie/funding/sfi-research-impact/> (8 May 2017). 12. Research Councils UK, ‘Official Development Assistance Global Challenges Research Fund guidance’, Research Councils UK, <http://www.rcuk.ac.uk/documents/international/gcrfodaguidance-pdf/> (27 April 2017), p. 3. 13. These figures are based on the database of REF2014 Impact Case Studies available at <http://impact.ref.ac.uk/CaseStudies/Search1.aspx.>. 14. DFID, ‘Review of embedding evaluation in the Department for International Development’. 15. Drew B. Cameron, Anjini Mishra, and Annette N. Brown, ‘The growth of impact evaluation for international development: How much have we learned?’, Journal of Development Effectiveness 8, 1 (2016), pp. 1–21, Table 5. 16. USAID, ‘Monitoring, Evaluation, Research and Learning Innovations Program (MERLIN)’, US Agency for International Development <https://www.usaid.gov/GlobalDevLab/about/monitoring-evaluation-research-and-learning-innovations-program> (27 April 2017). 17. The Westminster Foundation for Democracy is a non-departmental public body that receives its core funding from the Foreign and Commonwealth Office, and additional grants from DFID. 18. Helen Tilley, Africa as a living laboratory: Empire, development, and the problem of scientific knowledge, 1870–1950 (University of Chicago Press, Chicago, IL, 2011). 19. Linda Nordling, ‘San people of Africa draft code of ethics for researchers’, Science, 17 March 2017 <http://www.sciencemag.org/news/2017/03/san-people-africa-draft-code-ethics-researchers> (3 August 2017). 20. Nic Cheeseman, ‘An introduction to African politics’, Volume 1, in Nic Cheeseman (ed), African politics: Critical concepts in political science, 4 volumes (Routledge, Abingdon, 2016), pp. 1–25. 21. Authors’ calculations, based on Figure 3 in Graeme Blair, Radha K. Iyengar, Jacob N. Shapiro, ‘Where policy experiments are conducted in economics and political science: The missing autocracies’, Working Paper (20 May 2013). 22. Devra C. Moehler, ‘Democracy, governance, and randomized development assistance’, The Annals of the American Academy of Political and Social Science 628, 1 (2010), pp. 30–46. 23. Ana L. De La O and Leonard Wantchekon, ‘Experimental research on democracy and development’ in James N. Druckman, Donald P. Green, James H. Kuklinski, and Arthur Lupia (eds.), Cambridge handbook of experimental political science (Cambridge University Press, Cambridge, 2011), pp. 384-96. 24. Christopher B. Barrett and Michael R. Carter, ‘The power and pitfalls of experiments in development economics: Some non-random reflections’, Applied Economic Perspectives and Policy 32, 4 (2010), pp. 515–48. 25. Michael Bamberger, Michele Tarsilla, and Sharlene Hesse-Biber, ‘Why so many ‘rigorous’ evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute’, Evaluation and Program Planning 55, April (2016), pp. 155–62. 26. Morten Jerven, ‘Research note: Africa by numbers: Reviewing the database approach to studying African economies’, African Affairs 115, 459 (2016), pp. 342–58. 27. Cheeseman, ‘An introduction to African politics'. 28. Gerard Prinsen, Ilse Hartog, and Martijn Vink ‘‘It Would Be Great to See an Example…’: Collaboration between international NGOs and national knowledge institutes in six African countries’, Higher Education Policy 30, 2 (2017), pp. 203–23. 29. Briggs and Weathers, ‘Gender and location in African politics scholarship’. 30. Cameron, Mishra, and Brown, ‘The growth of impact evaluation for international development’. 31. Megan Bradley, ‘On the agenda: North–South research partnerships and agenda-setting processes’, Development in Practice 18, 6 (2008), pp. 673–85. 32. Johnson Muchunguzi Ishengoma, ‘North–South research collaborations and their impact on capacity building: A Southern perspective’, in Tor Halvorsen and Jorun Nossum (eds), North–South knowledge networks: Towards equitable collaboration between academics, donors and universities (African Minds, Cape Town, 2016), pp. 149–86. 33. Gilles Carbonnier and Tiina Kontinen, ‘North-South research partnership: Academia meets development?’, EADI Policy Paper Series, 2014, <https://www.eadi.org/typo3/fileadmin/Documents/Publications/EADI_Policy_Paper/EADI_policy_paper_Carbonnier_Kontinen_FINAL.pdf> (2 August 2017). 34. Artwell Nhemachena, Nelson Mlambo, and Maria Kaundjua, ‘The notion of the ‘field’ and the practices of researching and writing Africa: Towards decolonial praxis’, Journal of Pan African Studies 9, 7 (2016), pp. 15-37, p. 17. 35. For example, see Association of Social Anthropologists of the UK and Commonwealth, ‘Ethical guidelines for good research practice’, 2011, <https://www.theasa.org/downloads/ASA%20ethics%20guidelines%202011.pdf> (8 May 2017); Australian Council for International Development, Principles and guidelines for ethical research and evaluation in development (ACFID, Canberra, 2016). 36. Social Research Association, Ethical guidelines (Social Research Association, London, 2003). 37. James D. Fearon, Macartan Humphreys, and Jeremy M. Weinstein, ‘How does development assistance affect collective action capacity? Results from a field experiment in post-conflict Liberia’, American Political Science Review 109, 3 (2015), pp. 450–69. 38. Fearon, Humphreys, and Weinstein, ‘How does development assistance affect collective action capacity?’. 39. Paul Collier and Pedro C. Vicente, ‘Votes and violence: Evidence from a field experiment in Nigeria’, The Economic Journal 124, 574 (2014), pp. F327–55. 40. Alexander L. George and Andrew Bennett, Case studies and theory development in the social sciences (MIT Press, Cambridge, MA, 2005), Ch. 8. 41. Andrew Natsios, ‘The clash of the counter-bureaucracy and development’ (Essay, Center for Global Development, Washington, DC, 2010). 42. De La O and Wantchekon, ‘Experimental research on democracy and development’. 43. World Bank, ‘World Development Report 2014: Risk and opportunity—Managing risk for development’ (World Bank, Washington, DC, 2013). 44. Roper, ‘Achieving successful academic-practitioner research collaborations’. 45. Jeremy Keenan, ‘US militarization in Africa: What anthropologists should know about AFRICOM’, Anthropology Today 25, 5 (2008), pp. 16–20; David H. Price, Weaponizing anthropology (Ak Press, Oakland, CA, 2011). 46. For example, The International Institute for Environment and Development (IIED) adopts the position that prohibitions on independent publication or disclosure or research findings ‘may be justified in order to fulfil IIED’s mission under certain conditions’; IIED, Integrity and ethics in research, partnership and policy Engagement, 2017, <https://www.iied.org/sites/default/files/IIED%20Research%20Ethics%20Policy%20%28March%202017%29.pdf> (3 August 2017). 47. UK Research Integrity Office, Good practice in research: Authorship, 2017, <http://ukrio.org/wp-content/uploads/UKRIO-Guidance-Note-Authorship-v1.0.pdf> (3 August 2017). © The Author 2017. Published by Oxford University Press on behalf of Royal African Society. All rights reserved http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png African Affairs Oxford University Press

The potential and pitfalls of collaborating with development organizations and policy makers in Africa

Loading next page...
 
/lp/ou_press/the-potential-and-pitfalls-of-collaborating-with-development-0lgQGSyEwW
Publisher
Oxford University Press
Copyright
© The Author 2017. Published by Oxford University Press on behalf of Royal African Society. All rights reserved
ISSN
0001-9909
eISSN
1468-2621
D.O.I.
10.1093/afraf/adx041
Publisher site
See Article on Publisher Site

Abstract

Abstract A growing number of academics are engaging in collaborative research projects with development organizations and policy makers. Increasingly, this includes efforts to co-produce research, rather than simply share information. These new ways of doing research raise important ethical and practical issues that are rarely discussed but deserve attention. Sub-Saharan Africa is the region of the world in which these new approaches are particularly prevalent, and one where the challenges created by those approaches tend to manifest in distinct or acute ways. In this Research Note, we draw on a collaborative research project with the Westminster Foundation for Democracy to illuminate these difficulties. We also offer suggestions for how to manage the challenges that arise when academics conduct research with policy makers and development organizations. Ensuring that such collaborations are both effective and ethical is not easy, but it must be done if we are to develop better informed policy and scholarship. In the last decade, academic researchers and development organizations such as bilateral aid agencies, multilateral organizations, and international NGOs have found themselves working together more frequently than in the past. Sometimes this has been the product of genuine enthusiasm about the opportunities offered by collaboration. In other cases, it has been the product of outside pressures, particularly from funders and review bodies, an issue we return to later. Researchers' collaborations with development organizations and policy makers have also changed in nature. Previously, scholarly policy engagement tended to either take the form of academics being employed as private consultants outside of their university roles, or policy makers funding academic work in areas of specific interest. By contrast, the last decade has witnessed the proliferation of new collaborations that go beyond simply evaluating completed projects or answering specific questions. In an increasing number of cases, research is now being integrated into development projects from the outset, with researchers playing an active role in shaping interventions. In these cases, development organizations and policy makers, such as the United Kingdom’s Department for International Development (DFID), have worked with teams of researchers to design and implement the monitoring and evaluation of major interventions.1 In one such project, researchers from Columbia University examined the impact of scorecards in monitoring the performance of Members of Parliament and promoting accountability in Uganda.2 In programmes like this one, academics are involved from a very early stage and help to make decisions that shape policy and practice in profound ways. Thus, what is distinctive about this latest phase of collaboration is its scope and institutional embeddedness: many of the new partnerships run for years, involve teams of researchers rather than an individual, and are co-hosted within universities. This greater entanglement of the academic and policy worlds gives rise to the ethical and practical challenges we discuss in this Research Note. The trend towards more frequent and more extensive research collaboration between academics, development organizations and policy makers is not confined to sub-Saharan Africa, though as we discuss below, it raises some particularly thorny issues in that context. Nor is this trend confined to the social sciences; if anything, the social sciences are late to the party. Partnerships between academic researchers and development organizations have a long history in more technical fields such as global health, agriculture and sanitation, though those areas have also witnessed an increase in such collaborations in the last few years.3 We focus on political science and development because the literature is less well developed in this area, and it is where we have the most evidence and experience. The growing number of collaborations has been incentivised by the pressure that funders place on academics, particularly in the United Kingdom, to achieve research impact, and on practitioners to demonstrate the efficacy of their policies, a point we explore in greater detail below. Of course, satisfying the demands of funders is not the only reason to team up. Collaborative research offers many potential benefits. While those working within development organizations gain access to expert advice, analysis, and cutting edge methodological techniques, academics gain access to fresh sources of evidence, opportunities to test novel ideas, and new routes through which their research might influence practice and policy-making. Indeed, at first-glance, collaborations between academics and development practitioners look like a win–win proposition.4 However, such projects also generate risks and challenges. Academics and practitioners do not necessarily have the same objectives and priorities. Nor do they always work within the same time horizons. Out-dated stereotypes as to what ‘academic’ and ‘policy’ work entails can also hamper effective collaboration.5 Perhaps more importantly, collaborations with development organizations and policy makers raise serious structural and systemic risks, including those of structural bias, losing objectivity and deviating from ethical standards relating to consent. While some of these risks arise regardless of where research is carried out, others manifest in distinct or particularly acute ways when research is conducted in the African context because of existing inequalities in the production of knowledge about the continent. As a recent article in African Affairs demonstrated, despite democratization and the ‘Africa rising’ narrative, the proportion of journal articles published on the continent that are written by those living in it has fallen in recent years.6 Given this, there are difficult questions to be asked about who benefits when Africa is used as a laboratory for social science, how collaborative research can avoid perpetuating power imbalances entrenched by colonialism, and whether encouraging researchers to work more closely with development organizations will further marginalize African voices in the production of knowledge about the continent. In this Research Note, we use the Political Economy of Democracy Promotion Project7 as the starting point for a more detailed examination of these issues. This research programme is a collaboration between the International Development Department of the University of Birmingham and the Westminster Foundation for Democracy, the United Kingdom’s leading democracy assistance organization. It aims to identify the conditions under which democracy promotion activities are successful and to explain how such interventions can be improved. Using this project as an entry point, we explore the challenges that arise when academics collaborate with policy makers and development organizations. In order to structure the discussion, we distinguish between ethical and practical challenges, though we recognize that there is often overlap between these; many ethical problems have a practical dimension, while practical issues often raise ethical dilemmas. We do not claim to have ‘solved’ all of the issues that we identify in our project. Indeed, one of the main arguments of this Research Note is that many of these issues cannot simply be avoided or wished away; rather, they need to be constantly managed. Nor will the solutions that we have adopted be appropriate in all cases. Nevertheless, we believe that our project offers useful insights into the kinds of challenges that such collaborations generate, and how they can be tackled more effectively. The motivation behind collaboration The increase in both the frequency and the depth of collaboration between academics and development organizations has been driven on both sides by the demands of funders.8 On the academic side, scholars face increased pressure to explain how they will create ‘pathways to impact’ for their research. This trend has been most pronounced in the United Kingdom, where the provision of funding is now contingent on the provision of such an explanation,9 but similar policies have been adopted by funders in other countries such as Australia10 and Ireland.11 These kinds of requirement make it clear that funders expect proposals to set out convincing strategies and ring-fenced budgets for influencing policy makers and practitioners working in relevant fields. Such strategies are generally seen as more convincing when stakeholders close to the problem under scrutiny ‘are actively involved in the research … through the whole life cycle from initiation, design, progression, knowledge exchange and application of the research.’12 In the United Kingdom, the emphasis on research impact goes even further; academics are now formally evaluated on the policy impact of their work under the government’s Research Excellence Framework. This requires every academic department in the country to write an ‘Impact Environment Statement’ and provide ‘Impact Case Studies’ detailing projects with significant influence. In 2014, the vast majority of such case studies focused on ‘impacts’ in developed countries; researchers submitted 4,596 Impact Case Studies whose impact was located in Europe, while only 685 were based in Africa.13 However, this figure is relatively high when one takes into account the fact that only a small minority of research conducted within UK higher education institutions focuses on Africa. Moreover, we expect the number of Impact Case Studies on the continent to rise in the future given moves to channel a significant amount of UK Official Development Assistance to UK Research Councils via the Global Challenges Research Fund. On the other side, development organizations face mounting pressure from their funders and an increasingly critical media to demonstrate two things. First, that the interventions they deliver are based on evidence. Second, that those interventions have a demonstrable (and ideally, measurable) impact on development outcomes. Among funders, DFID has worked particularly hard to build a reputation as an organization that generates, and uses, high-quality and robust evidence about what works in international development, investing heavily in both external research and its internal evaluation systems. In some cases, projects come with specific guidelines regarding expenditure on monitoring and evaluation, for example that such activities should receive around 10 percent of the overall budget. This investment in research and evaluation has been particularly marked in relation to Africa. In 2014, an internal review conducted by DFID found that its programmes in Africa were more likely to be evaluated, either by DFID’s internal evaluation team or by independent experts commissioned by DFID, than programmes in any other region.14 While DFID’s investment in research and evaluation is distinctive, it is by no means unique. Over the last few decades, there has been a marked increase in the number of published impact evaluations of international development interventions, almost a third of which have taken place in Africa.15 In 2015, the US Agency for International Development (USAID) launched the Monitoring, Evaluation, Research and Learning Innovations Program. This programme is designed to allow USAID and its partners ‘to source, co-create, and co-design development solutions that innovate on traditional approaches to monitoring, evaluation, research and learning.’16 The fiscal constraints faced by many traditional donors, together with the election of leaders such as President Trump who exhibit clear scepticism about the efficacy and ideological justification for development aid, mean that the pressure on aid and educational budgets is unlikely to ease any time soon. As a result, academics, policy makers and practitioners will continue to face strong incentives to join forces for some time to come. Critical friends, not cheerleaders Our collaboration with the Westminster Foundation for Democracy (WFD) started in late 2015.17 WFD’s contribution to the project includes a funded post-doctoral position and the provision of access to three things: (i) internal WFD documents (including programme proposals, reports and evaluations); (ii) people (through formal interviews and informal exchanges, the latter facilitated by the presence of the post-doctoral researcher at WFD’s London office on a semi-regular basis); and (iii) events (WFD programme activities). In exchange, the academic research team comprised, at present, of the authors of this Research Note have produced a series of policy papers, some of which have formed the basis of academic articles. One of the explicit aims of the project is that the academic research will help to strengthen the approach of WFD. While the project has given rise to many gains, it also generates a number of ethical risks. One of the most obvious ethical risks that emerge in long-term collaborations like this is that academics will be co-opted, losing their objectivity. To do good collaborative research, academics need to find a way of being critical friends to practitioners, rather than cheerleaders. Evaluations commissioned by development organizations, and written by academics acting as consultants, are inevitably approached with a degree of scepticism because they tend to pull their punches. This is often true in sub-Saharan Africa, where the acknowledged difficulty of promoting development or democracy makes it tempting to tone down criticism of practitioners. Overly generous evaluations are especially likely when a researcher’s continued employment is contingent on their partner organization being happy with the research they produce, or when researchers are sympathetic to the goals of development interventions. The latter represents a particular challenge in collaborative work because researchers are unlikely to partner with development organizations unless they accept at least some basic assumptions made by those organizations, including the assumption that such organizations ought to intervene to promote development. This may make a degree of bias inevitable. There are no hard and fast rules that can ward off favouritism, especially where it is unconscious, but there are ways to reduce the problem. Our project with WFD provides one example of what this might look like in practice. While WFD funds a post-doctoral position linked to the project, the post-doctoral fellow is not employed directly by WFD. This ensures that the university, rather than WFD, is responsible for assessing their performance and making any decisions about their continued employment. In addition, while the post-doctoral fellow works in the WFD office on as semi-regular basis to facilitate data collection, the majority of their work is conducted at locations outside of WFD. This helps to ensure a degree of critical distance, as does the fact that the remainder of the project team (including the academic lead) is not embedded within WFD. More generally, it is important that the lead academic on such projects (often called the Principal Investigator) does not receive a direct financial benefit from their involvement in the form of a salary increase or other perks. This establishes a degree of formal independence that can prevent institutional capture. It is also advisable that the outputs of any such collaboration are reviewed by an advisory board of individuals who have no ties to the organization or projects being evaluated, providing an external check on the process. If these rules are followed, the risk of producing biased findings can be significantly reduced, though not eliminated. It is therefore important to also maintain transparency at all times, and clearly state that research outputs are the product of collaborative projects. Our experience also suggests that academics should avoid assuming that development practitioners want research to be biased in their favour. Indeed, some practitioners are just as worried as academics about the potential loss of objectivity inherent in this kind of research. In our case, a significant percentage of mid-level and senior staff at WFD have doctorates, which has fostered an organizational culture that casts objective research as a good thing. WFD’s staff tend to see research as a way of generating more robust evidence that will allow them to demonstrate the value of their work at a time when the United Kingdom’s aid budget is under increasing pressure. They know that evidence will only be convincing if it is produced in an objective and transparent way. In other words, most WFD staff believe they are better off having critical friends, rather than cheerleaders. This is not to say that our research collaboration has been perceived in a positive manner by every single person linked to WFD. Initially, there were some who clearly felt uncomfortable with too many details of their projects being made public. We found practical ways to manage these concerns. Sometimes this was as simple as explaining that we did not have to ‘name names’—anonymity was an option. In other cases, we circulated drafts of research outputs (specifically, policy papers) for comment in advance. This was done on the understanding that while we would not change our findings, we could clarify certain points if staff felt the factual details of their programmes had been misrepresented. Unsurprisingly, we have found that practitioners are far more tolerant of criticism when they have had a chance to discuss and respond to it prior to publication. Leading the way in methodological innovation, or turning Africa into a laboratory? A central question that faces research in the African context is who benefits: researchers or research participants? As Helen Tilley has argued, Western scholars have a long history of using Africa as a ‘living laboratory’ to produce research, some of which helped to maintain colonial empires.18 The San people of Southern Africa, a population that is frequently the subject of academic research, recently launched their own code of ethics for researchers, partly due to dissatisfaction with the benefits that research has provided to their community.19 How to conduct research in a fair and ethical way is a particularly pressing issue given the rising number of collaborative research projects that use Africa as a site for experimental methods, including randomized controlled trials (RCTs) and ‘laboratory’ techniques.20 Between 1995 and 2013 four African countries—Kenya, Malawi, Uganda and Ghana—accounted for around 25 percent of all RCTs conducted in non-OECD countries and published in the top 25 economics and political science journals, or listed on the websites of major organizations supporting RCTs in development economics.21 At the same time, African examples have become particularly prevalent in field experiments linked to democracy and governance programmes, which typically require academics to work collaboratively with practitioners. When Devra Moehler compiled a (non-exhaustive) list of such projects in 2010, she found that just over half of them took place on the continent.22 While it is good to see African studies at the forefront of methodological innovation, the growing popularity of these methods has proved controversial, triggering renewed debates about what constitutes ‘good research’, along with concern about ethical and moral issues. Though experimental methods such as RCTs have advantages, they raise their own methodological challenges23 and have important limitations that are often overlooked.24 For example, while RCTs are good for testing whether an intervention achieved its stated goal, they often fail to identify the unintended consequences of development programmes.25 Some also fear that the spread of experimental methods may marginalize qualitative research, which remains essential regardless of advances on the quantitative front,26 and is the most common form of research conducted within African universities.27 Collaborative research between academics and development organizations has the potential to exacerbate concerns that Western scholars have reduced Africa to a ‘testing ground’ for their theories. The majority of the policy/research collaborations established in the United States or United Kingdom are led by American and British academics and some lack African involvement at the top levels. There is also evidence that the limited resources available to many African universities discourages collaboration between development organizations and Africa-based researchers, as do (often unfounded) concerns among some policy makers about the ‘usability’ of the research that the latter produce.28 However, it is also true that bodies such as DFID have invested in forming three-legged partnerships with African institutions. We suspect that in many cases uneven representation results from the ease of working with researchers who are close by, the desire to partner with prestigious institutions, and in some cases the imperative of spending a proportion of budgets within the home country of the funder. The limited presence of Africa-based researchers in the management of large projects is problematic for a number of reasons. Most notably, while a lack of local knowledge may undermine research and policy outputs, African researchers miss out on access to privileged information and funding to collect new data. The latter is of particular concern given the extremely limited core funding that is available to research institutions on the continent. The exclusion of African scholars from these opportunities also means that collaborations between academics and development organizations might further marginalize African researchers in disciplines in which they are already underrepresented.29 An analysis of the impact evaluations of international development interventions published between 1981 and 2012 found that over time the share of authors with institutional affiliations in North America and Western and Northern Europe increased but the proportion from sub-Saharan Africa declined.30 While impact evaluations and collaborations are not synonymous, this trend suggests that collaborations between researchers and development organizations may have undesirable side-effects. One way to deal with this is to ensure that such collaborations include ring-fenced funds to support research partnerships with Africa-based organizations. However, such research partnerships will need to be carefully designed as they raise a number of ethical issues in themselves.31 In particular, the fact they are funded and designed in the West creates a significant risk that they will simply perpetuate imbalanced relationships between Western and African researchers.32 Recent survey and interview-based evidence indicates that in existing north–south research partnerships the role of southern researchers is often limited to collecting data for country case studies while northern researchers take the lead in identifying research questions, deciding on research methods, conducting cross-country analysis and—as a consequence—authoring publications.33 In light of this, collaborative research partnerships will need to consciously prioritize the co-development of research questions, the co-design of research methods, the equal division of research tasks, and the co-authorship of publications. The fact that the development agencies of Western governments fund a significant proportion of the RCTs and other experimental research projects also raise concerns that academics are becoming increasingly involved in promoting external interests on the continent. Even when research is intended to benefit African citizens, some question whether—by feeding into the policies of Western donors, rather than those of African states—it has become a new ‘tool for indirect rule’.34 This is a valid question to ask of many collaborations, including our project; WFD is engaged in the kind of democracy support programmes that are sometimes attacked by African governments as an infringement on sovereignty and an attempt to impose foreign values. How can such concerns be mitigated? One potential avenue through which to do this is to create more space for the voices of programme beneficiaries—and critics—to be heard. While the intended beneficiaries of development programmes are routinely surveyed to provide feedback, such checks often represent a token exercise that does little to speak to the concerns of African citizens. This has been a problem for WFD, not because WFD does not care about this kind of feedback, but because it is exceptional for beneficiaries to provide direct criticism of WFD activities. Workshop participants asked to identify which session they found ‘least useful’ will typically (sometimes, unanimously) report that there ‘was no least useful session’. This reluctance to give negative feedback is—we suspect—partly due to politeness, and partly due to a worry that doing so would harm the potential for future engagement. Although this challenge is hard to overcome, where research involves deeper collaboration in the form of co-production, academics can use different techniques such as longer interviews, focus groups, and anonymous questionnaires to elucidate more accurate responses, and thus improve the way in which programmes are designed, delivered and evaluated. Academics introduced as being independent from, but working with, development practitioners may also be better placed to uncover alternative perspectives. However, this potential ‘solution’ raises a further ethical challenge regarding the quality of consent. Protecting the quality of consent In many academic codes of conduct, receiving informed and genuine consent from research subjects is one of, if not the most, significant component in carrying out ethical research.35 It is essential to ensure that all participation in research activities is voluntary and that research participants never feel coerced, something that will often require care and forethought when research is conducted in collaboration with development organizations. This is because such bodies typically act as gatekeepers of resources and opportunities by determining who will be included in their projects. Ethics guidelines often stress that responsibility for obtaining informed consent cannot be devolved to gatekeepers.36 Moreover, most development programmes—political or otherwise—are designed to provide benefits to those included. Many take place in the context of pre-existing or ongoing relationships between development organizations and programme beneficiaries. This creates a risk that individuals will feel obliged to participate in research lest they miss out on the benefits offered by development interventions. This is a concern because most (possibly all) guidelines also direct researchers to ensure that those who decline to participate in, or withdraw from, research activities do not suffer any negative consequences as a result. In the context of collaborations between researchers and development organizations, this means that there should be no connection between participation in research and eligibility for development project ‘benefits’. Where possible, the largest portion of the research should be conducted once a development intervention is finished, when there are no future benefits to be lost. However, when researchers seek consent from individuals who are receiving—or hope to receive—benefits from a related project, separating the role of the development organization from that of the academic, and the benefits of the project from voluntary participation in research, may become extremely difficult. For us, this issue has—so far—not been as severe because our research has focused on individuals who are part of Africa’s political elite: Members of Parliament and leaders of prominent civil society groups. Such individuals are generally more educated, assertive, and wealthy, and therefore less prone to feeling pressured to join research projects that run side-by-side with development interventions. Nonetheless, we are careful to stress that refusal to grant us an interview or fill in a survey has no bearing on the distribution of WFD resources. In other situations, the problem is more acute. For example, researchers investigating the impact of development assistance delivered by the International Rescue Committee on collective action in Liberia quickly recognized that individuals in relatively poor and isolated communities might feel obliged to participate in order to access desperately needed services.37 In this instance, waiting to invite communities to participate in the research component of the programme until after the community development projects delivered by the International Rescue Committee were formally complete was essential to ethically securing voluntary consent.38 Even when research takes place after development interventions are complete, it is possible that individuals might feel obliged to participate because of the benefits they have already received. This makes it important to distinguish between the research team and the team responsible for implementing a development intervention. Having these two groups undertake their work at different times and in different venues helps. However, the distinction between them may not be obvious to outsiders unless researchers take steps to make it more visible. A field experiment led by Paul Collier and Pedro Vicente in Nigeria provides a useful example of how this might be done.39 For their research, Collier and Vicente collaborated with ActionAid International Nigeria to examine the impact of a campaign against electoral violence that was designed to encourage participants to oppose voter intimidation. Distinct research and campaign teams operated independently, save for the baseline survey conducted prior to ActionAid International Nigeria’s campaign, during which one member of the campaign team accompanied the research team to identify the research sites. The two teams took care to use different ‘branding’ to ensure the distinction between them was clearly visible to the communities in which they were operating. While the campaigners wore t-shirts and caps branded with the anti-violence campaign, survey enumerators wore name tags and displayed the credentials of the survey company. As with our other suggestions, these kinds of strategies will not eradicate the problem, but they will enable researchers to better manage it. Practical challenges in collaborative research In addition to ethical concerns, collaborative research raises practical challenges, though in many cases the boundaries between the two are blurry. One (primarily) practical challenge is that of reconciling good research design with good programme design. The two are not necessarily synonymous. Most comparative research designs rely on variation in the dependent variable (what is to be explained),40 which can create problems because funders and development organizations typically want to focus their resources on maximizing the chances of success. They may be reluctant to include ‘control’ cases, or to run projects that are unlikely to succeed, making it difficult to generate the variation desired by academics. Indeed, many development organizations have deeply entrenched cultures of risk aversion, a fact pointed out by the former head of USAID.41 This attitude to risk has significant implications for what academics can study in collaboration with policy makers. In some cases, topics that are of interest to academics may be off-limits because they do not appear to be viable to development organizations. In other cases, practitioners’ interest in increasing the impact of a programme by ‘bundling’ a number of activities together may sit in tension with researchers’ desire to disaggregate the effect of each component of an intervention.42 The tensions between good research design and good programme design are not insurmountable, but may require a new approach. Development organizations are increasingly alert to the fact that seeking to avoid all risk is not only impossible, but often counter-productive. In 2013, the World Bank declared an intention to shift from an institutional culture of extreme risk aversion to one of informed risk taking.43 Growing concerns about both terrorism and migration have led to greater recognition of the fact that difficult cases cannot simply be ignored. These changes may create opportunities to reconcile research and programme design, but taking them will require academics to persuade their collaborators that particular risks are worth taking in order to gain greater analytical leverage. In the African context, this is particularly significant because many of the most pressing development challenges occur in countries where the political context generates the greatest risk of failure, such as the Central African Republic, Sudan, and Zimbabwe. Another practical problem that can arise in these kinds of collaborations is that of time horizons. Practitioners tend to need answers to specific, time-bound problems, while academic research is an ongoing, iterative process.44 When academics ask more ambitious and complex research questions, it may be impossible to specify how long it will take to get answers. This often leads to an assumption that practitioners have much shorter time horizons than academics, rendering certain research topics unviable. However, our experience suggests that the gap between academic and practitioner time horizons is not always as great as is assumed; this issue is often one of organizational constraints rather than attitude. In the case of our project, WFD staff consistently emphasized the importance of taking a long-term approach to programme design and evaluation. The real challenge for us has not been that WFD wants answers as soon as possible, but that our attempts to find those answers need to fit into WFD’s programming and funding cycles. These place constraints on when programmes (and thus research) can start, and on how long they can run. This explains why our collaboration is only gradually moving from policy papers that focus on a retrospective evaluation of WFD programmes and strategy, to more forward-looking research that is integrated into programmes from their inception through to implementation and eventually evaluation. Overcoming the constraints imposed by programming and funding cycles is not something that academics can do on their own. Instead, it requires a longer conversation with funders and policy makers about how budgets and timeframes can be better designed to allow for longer-term research projects that are not undermined by arbitrary milestones or inappropriate deadlines. Making collaborative research work As our discussion of ethical and practical issues makes clear, it is not easy to make collaborative research work, but it is important if we are to make progress in terms of deepening our understanding of issues such as the effectiveness of international development and democracy programmes on the continent. We have identified a number of ways in which the ethical challenges involved in such projects can be managed. These include prohibiting personal gain to lead researchers, employing an independent advisory board, amplifying the voice of beneficiaries and critics, establishing balanced research partnerships with Africa-based organizations, and taking steps to ensure that consent is not coerced. However, these measures will count for little unless they occur within the context of a legal agreement or memorandum of understanding that guarantees academics intellectual property rights over their research. Academics are often asked to sign contracts that grant them certain intellectual property rights, but also circumscribe those rights. This is frequently done through non-disclosure agreements, and by requiring researchers to submit publications to policy partners for review before publication. Such constraints are particularly common in work that involves a security aspect, such as research conducted for the United States Africa Command, which is one of a number of reasons why such projects are often viewed as particularly problematic.45 Here, balance is key. It is understandable for development organizations and policy makers to seek some measure of influence over how the research they help to produce is used, particularly with respect to confidential information that they and their partners make available. Contracts that prohibit independent publication of research findings might be tolerated by researchers acting as paid consultants outside of their university role.46 However, in co-produced research and institutional collaborations, it is essential to maintain the freedom of academics to publish their findings in peer-reviewed journals, even when they cast the programmes of partners in a less than favourable light. Contracts that require some degree of consultation prior to publication may be reasonable. Contracts that give development organizations a veto over what can be published should be avoided. Indeed, the UK’s Research Integrity Office advises academic institutions to ‘protect researchers from inappropriately restrictive or coercive contracts that, for example, prevent researchers from publishing work’.47 Given the existence of such ‘red lines’, academics must be judicious when entering into any new research collaboration. We have been very fortunate in our collaboration to have a partner whose organizational culture values independent research while tolerating constructive criticism. The fact that we have such a partner is not a matter of chance, however. We only entered into the collaboration because it was clear to us that WFD was genuinely committed to academic freedom, as evidenced by the fact that we were granted full intellectual property rights and not asked to sign a non-disclosure agreement. As we have discussed above, there are a number of practical steps that can be taken to help make collaborative research work in an effective and ethical manner. Ultimately, however, the most important step is deciding who—and who not—to work with. Footnotes 1. DFID, ‘Review of embedding evaluation in the Department for International Development’ (London, DFID, 2014). 2. African Leadership Initiative, ‘Parliamentary scorecard 2009–2010: Assessing the performance of Ugandan legislators’ (African Leadership Institute, Kampala, 2011). 3. Catherine Olivier, Matthew R. Hunt, and Valéry Ridde, ‘NGO–researcher partnerships in global health research: Benefits, challenges, and approaches that promote success’, Development in Practice 26, 4 (2016), pp. 444–55. 4. Laura Roper, ‘Achieving successful academic-practitioner research collaborations’, Development in Practice 12, 3–4 (2002), pp. 338–45. 5. William G. Moseley, ‘Collaborating in the field, working for change: Reflecting on partnerships between academics, development organizations and rural communities in Africa’, Singapore Journal of Tropical Geography 28, 3 (2007), pp. 334–47, p. 337. 6. Ryan Briggs and Scott Weathers, ‘Gender and location in African politics scholarship: The other white man’s burden?’, African Affairs, 115, 460 (2016), pp. 466-489. 7. Further details about this project are available on the project website https://democracypromotion.wordpress.com. 8. A similar observation is made by Daniel Stevens, Rachel Hayman, and Anna Mdee, ‘‘Cracking collaboration’ between NGOs and academics in development research’, Development in Practice 23, 8 (2013), pp. 1071–77. 9. Research Councils UK, ‘Pathways to Impact’, Research Councils UK, <http://www.rcuk.ac.uk/innovation/impacts/> (27 April 2017). 10. Australian Research Council, ‘Research impact principles and framework’, Australian Research Council Website, <http://www.arc.gov.au/research-impact-principles-and-framework> (8 May 2017). 11. Science Foundation Ireland, ‘Research impact’, Science Foundation Ireland Website <http://www.sfi.ie/funding/sfi-research-impact/> (8 May 2017). 12. Research Councils UK, ‘Official Development Assistance Global Challenges Research Fund guidance’, Research Councils UK, <http://www.rcuk.ac.uk/documents/international/gcrfodaguidance-pdf/> (27 April 2017), p. 3. 13. These figures are based on the database of REF2014 Impact Case Studies available at <http://impact.ref.ac.uk/CaseStudies/Search1.aspx.>. 14. DFID, ‘Review of embedding evaluation in the Department for International Development’. 15. Drew B. Cameron, Anjini Mishra, and Annette N. Brown, ‘The growth of impact evaluation for international development: How much have we learned?’, Journal of Development Effectiveness 8, 1 (2016), pp. 1–21, Table 5. 16. USAID, ‘Monitoring, Evaluation, Research and Learning Innovations Program (MERLIN)’, US Agency for International Development <https://www.usaid.gov/GlobalDevLab/about/monitoring-evaluation-research-and-learning-innovations-program> (27 April 2017). 17. The Westminster Foundation for Democracy is a non-departmental public body that receives its core funding from the Foreign and Commonwealth Office, and additional grants from DFID. 18. Helen Tilley, Africa as a living laboratory: Empire, development, and the problem of scientific knowledge, 1870–1950 (University of Chicago Press, Chicago, IL, 2011). 19. Linda Nordling, ‘San people of Africa draft code of ethics for researchers’, Science, 17 March 2017 <http://www.sciencemag.org/news/2017/03/san-people-africa-draft-code-ethics-researchers> (3 August 2017). 20. Nic Cheeseman, ‘An introduction to African politics’, Volume 1, in Nic Cheeseman (ed), African politics: Critical concepts in political science, 4 volumes (Routledge, Abingdon, 2016), pp. 1–25. 21. Authors’ calculations, based on Figure 3 in Graeme Blair, Radha K. Iyengar, Jacob N. Shapiro, ‘Where policy experiments are conducted in economics and political science: The missing autocracies’, Working Paper (20 May 2013). 22. Devra C. Moehler, ‘Democracy, governance, and randomized development assistance’, The Annals of the American Academy of Political and Social Science 628, 1 (2010), pp. 30–46. 23. Ana L. De La O and Leonard Wantchekon, ‘Experimental research on democracy and development’ in James N. Druckman, Donald P. Green, James H. Kuklinski, and Arthur Lupia (eds.), Cambridge handbook of experimental political science (Cambridge University Press, Cambridge, 2011), pp. 384-96. 24. Christopher B. Barrett and Michael R. Carter, ‘The power and pitfalls of experiments in development economics: Some non-random reflections’, Applied Economic Perspectives and Policy 32, 4 (2010), pp. 515–48. 25. Michael Bamberger, Michele Tarsilla, and Sharlene Hesse-Biber, ‘Why so many ‘rigorous’ evaluations fail to identify unintended consequences of development programs: How mixed methods can contribute’, Evaluation and Program Planning 55, April (2016), pp. 155–62. 26. Morten Jerven, ‘Research note: Africa by numbers: Reviewing the database approach to studying African economies’, African Affairs 115, 459 (2016), pp. 342–58. 27. Cheeseman, ‘An introduction to African politics'. 28. Gerard Prinsen, Ilse Hartog, and Martijn Vink ‘‘It Would Be Great to See an Example…’: Collaboration between international NGOs and national knowledge institutes in six African countries’, Higher Education Policy 30, 2 (2017), pp. 203–23. 29. Briggs and Weathers, ‘Gender and location in African politics scholarship’. 30. Cameron, Mishra, and Brown, ‘The growth of impact evaluation for international development’. 31. Megan Bradley, ‘On the agenda: North–South research partnerships and agenda-setting processes’, Development in Practice 18, 6 (2008), pp. 673–85. 32. Johnson Muchunguzi Ishengoma, ‘North–South research collaborations and their impact on capacity building: A Southern perspective’, in Tor Halvorsen and Jorun Nossum (eds), North–South knowledge networks: Towards equitable collaboration between academics, donors and universities (African Minds, Cape Town, 2016), pp. 149–86. 33. Gilles Carbonnier and Tiina Kontinen, ‘North-South research partnership: Academia meets development?’, EADI Policy Paper Series, 2014, <https://www.eadi.org/typo3/fileadmin/Documents/Publications/EADI_Policy_Paper/EADI_policy_paper_Carbonnier_Kontinen_FINAL.pdf> (2 August 2017). 34. Artwell Nhemachena, Nelson Mlambo, and Maria Kaundjua, ‘The notion of the ‘field’ and the practices of researching and writing Africa: Towards decolonial praxis’, Journal of Pan African Studies 9, 7 (2016), pp. 15-37, p. 17. 35. For example, see Association of Social Anthropologists of the UK and Commonwealth, ‘Ethical guidelines for good research practice’, 2011, <https://www.theasa.org/downloads/ASA%20ethics%20guidelines%202011.pdf> (8 May 2017); Australian Council for International Development, Principles and guidelines for ethical research and evaluation in development (ACFID, Canberra, 2016). 36. Social Research Association, Ethical guidelines (Social Research Association, London, 2003). 37. James D. Fearon, Macartan Humphreys, and Jeremy M. Weinstein, ‘How does development assistance affect collective action capacity? Results from a field experiment in post-conflict Liberia’, American Political Science Review 109, 3 (2015), pp. 450–69. 38. Fearon, Humphreys, and Weinstein, ‘How does development assistance affect collective action capacity?’. 39. Paul Collier and Pedro C. Vicente, ‘Votes and violence: Evidence from a field experiment in Nigeria’, The Economic Journal 124, 574 (2014), pp. F327–55. 40. Alexander L. George and Andrew Bennett, Case studies and theory development in the social sciences (MIT Press, Cambridge, MA, 2005), Ch. 8. 41. Andrew Natsios, ‘The clash of the counter-bureaucracy and development’ (Essay, Center for Global Development, Washington, DC, 2010). 42. De La O and Wantchekon, ‘Experimental research on democracy and development’. 43. World Bank, ‘World Development Report 2014: Risk and opportunity—Managing risk for development’ (World Bank, Washington, DC, 2013). 44. Roper, ‘Achieving successful academic-practitioner research collaborations’. 45. Jeremy Keenan, ‘US militarization in Africa: What anthropologists should know about AFRICOM’, Anthropology Today 25, 5 (2008), pp. 16–20; David H. Price, Weaponizing anthropology (Ak Press, Oakland, CA, 2011). 46. For example, The International Institute for Environment and Development (IIED) adopts the position that prohibitions on independent publication or disclosure or research findings ‘may be justified in order to fulfil IIED’s mission under certain conditions’; IIED, Integrity and ethics in research, partnership and policy Engagement, 2017, <https://www.iied.org/sites/default/files/IIED%20Research%20Ethics%20Policy%20%28March%202017%29.pdf> (3 August 2017). 47. UK Research Integrity Office, Good practice in research: Authorship, 2017, <http://ukrio.org/wp-content/uploads/UKRIO-Guidance-Note-Authorship-v1.0.pdf> (3 August 2017). © The Author 2017. Published by Oxford University Press on behalf of Royal African Society. All rights reserved

Journal

African AffairsOxford University Press

Published: Jan 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off