Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The effectiveness and evaluation of conservation planning

The effectiveness and evaluation of conservation planning Introduction There is a lack of evidence‐based knowledge on the effectiveness of different conservation interventions, and limited monitoring and evaluation to generate better knowledge ( Pullin & Knight 2001 ; Sutherland . 2004 ; Ferraro & Pattanayak 2006 ). Without evidence, justification of investment in different interventions is thought to be guided by experience at best ( Cook . 2010 ) and anecdote and received wisdom at worst ( Sutherland . 2004 ). To date, there has been no comprehensive reporting or evidence to demonstrate the costs and benefits of investing in conservation planning. Program evaluation, a systematic process for judging effectiveness of programs ( Patton 2008 ), could provide a conceptual framework and a set of tools to enhance our knowledge of the value of conservation planning. In this review, we make the case for explicit program evaluation of regional‐scale planning exercises—those applied across domains defined by ecoregions, catchments, or jurisdictions with extents in the order of 10 2 to 10 5 km 2 . Our review has three specific aims: (1) to outline the benefits and challenges of more explicit and comprehensive evaluation of conservation planning; (2) to identify the range of indicators available to evaluate the outcomes of conservation planning; and (3) to explore conceptual, operational, organizational, and policy implications of evaluating planning exercises in the future. We hope this review highlights the importance of evaluation in promoting defensible conservation planning processes, adds realism to expectations about planning and, ultimately, helps to improve conservation outcomes. Over the last two decades, the application of conservation planning ( Margules & Pressey 2000 ) has expanded rapidly. It has influenced conservation priorities by international organizations ( Groves . 2002 ; Sanderson . 2002 ), guided policy decisions by government agencies ( Airame . 2003 ; Pressey . 2009 ), and resulted in hundreds of publications in the academic literature. In particular, considerable advances have been made in producing conservation plans—maps of priority conservation areas—with hundreds of plans developed around the world at a variety of scales. However, despite several prominent success stories of planning leading to real conservation gains ( Finkel 1998 ; Airame . 2003 ; Fernandes . 2005 ), there has been limited empirical evidence to demonstrate actual outcomes ( Knight . 2008 ). Without more comprehensive evaluation of the tangible outcomes of conservation planning, investments in planning are not accountable, decisions are not defensible, and learning from past experiences is limited. This is all the more important because each of the hundreds of planning exercises undertaken entails substantial costs ( Table 1 ). 1 Costs of conservation planning (all estimates exclude costs of applying actions) Planning process Lead organization Cost estimate (US$) Duration Area (km 2 ) Source Description Cape Action Plan for the Environment Project, South Africa South African National Botanical Institute (SANBI) ca. 1 million 1998–2000 90,000 Cowling & Pressey 2003 Project comprised four major steps: (1) situation assessment (including conservation planning component); (2) development of strategy to involve stakeholders and institutions; (3) action plan formulation, and (4) fundraising. Great Barrier Reef Marine Park rezoning plan Great Barrier Reef (GBR) Marine Park Authority ca. 6–8 million 1999–2004 344,400 Osmond et al. 2010 Costs covered staff salaries, staff time, and staff operating costs for the new zoning plan for the entire GBR spread over a 5‐year period. The Australian government also funded a structural adjustment package (∼$100 million) designed to assist fishers, fishery related businesses, employees, and communities affected by the rezoning. Adirondack State Park Landscape, USA Wildlife Conservation Society † Monetary value unavailable. Costs reported in relation to staff time (37 person‐weeks) 2000–2003 19,700 Didier et al. 2009 Stages of planning process were measured in relation to staff time (person‐weeks). Most time devoted to selection of focal species (14 weeks); mapping of biological landscape (8 weeks); and mapping of human landscape (8 weeks) Maputaland Transnational Conservation Plan, Mozambique/South Africa/Swaziland University of Kent (DICE) ca. 215,143 2003–2005 17,000 Smith & Leader‐Williams 2006 Costs covered convening of training workshops, staff time, development of GIS databases, production of decision support system (CLUZ), and design of tri‐national conservation plan through participatory workshops with key regional stakeholders followed by publicity and presentation of results. Kimbe Bay Conservation Action Planning, Papua New Guinea The Nature Conservancy ca. 400,000 2004–2007 13,000 Green et al. 2009 Total cost for the scientific design process in Kimbe Bay (excluding community engagement and implementation) was divided principally between scientific research (54%), staff time (35%) and workshops (10%). California Central Coast Marine Protected Areas Network, USA California Department of Fish and Game ca. 2.5 million 2005–2007 3,000 Raab 2006 Expenditures included: (1) executive/general administration and project management (25%); (2) facilitation and outreach (18%); (3) Department of Fish and Game lead staff and costs (18%); and (4) data preparation and analysis (12%). For the purpose of this review, we define conservation planning broadly as the design of conservation areas. Conservation areas can include IUCN protected area categories I–VI, as well as places where diverse management arrangements, such as zoning, offsets, easements, invasive species control, and restoration are applied. Most conservation planning today can be termed “systematic” ( Margules & Pressey 2000 ), involving explicit, quantitative objectives and leading to systems of potential conservation areas with emergent properties related to complementarity and connectivity ( Margules & Pressey 2000 ). The lack of evidence for the effectiveness of planning applies to systematic methods and to other approaches such as ranking procedures ( Smith & Theberge 1987 ) and expert‐based prioritizations ( Cowling . 2003 ). We focus here on whether designing conservation areas leads to actual conservation outcomes. Specifically, therefore, we refer here to evaluation of conservation assessment ( Knight . 2006a ), excluding the evaluation of stages involving implementation of actions or subsequent management of conservation areas (stages 10 and 11 of Pressey & Bottrill 2009 ) covered elsewhere ( Margoluis & Salafsky 1998 ; Ferraro & Pattanayak 2006 ; Hockings . 2006 ). In the sections that follow, we discuss the rationale for, and possible approaches to, measuring the effectiveness of conservation planning. We draw extensively on literature, but also present new qualitative data on planning outcomes gathered from semistructured interviews with experts involved in developing and/or implementing planning exercises. Qualitative data on effectiveness of planning The qualitative data collected in semistructured interviews were intended to fill large gaps in the literature concerning perceptions of the effectiveness of planning. Interviewees were selected using snowball sampling, by which selected interviewees identify other appropriate participants ( Lewis‐Beck . 2004 ). Among our participants, we interviewed people involved in the development of plans (“planners,” n = 15) and people responsible for updating plans, representing stakeholders, or providing technical support in applying actions arising from conservation plans (“implementers,” n = 13). We aimed for a representative sample of people working across socioeconomic contexts and environmental realms. More details on the affiliations of participants and the interview instrument are described in the Supporting Information. The interview format followed an open framework of themed questions that were conversational and flexible ( Mason 2002 ). Responses were analyzed using a content analysis of interview transcripts to draw out trends and frequency of responses. The interviews revealed perceptions of both positive and negative outcomes emerging from planning exercises by those intimately involved in their development and application. Evaluation and conservation planning Evaluation is a systematic method for collecting, analyzing, and assessing information on the effectiveness of projects and policies in relation to stated goals ( Patton 2008 ) and takes multiple forms. The most important basic distinction in evaluation is between summative and formative approaches ( Rossi . 2004 ). Formative approaches, including process evaluation and needs assessment, strengthen or improve the program being evaluated. Summative approaches, including outcome and impact evaluations, examine effects or outcomes of programs. We are primarily concerned here with the application of summative evaluation approaches. Table 2 describes definitions of key terms used in evaluation referred to in this review. 2 Key evaluation terms in the context of conservation planning Term Definition Example Purpose The intended aim(s) of conservation planning Identification of priority areas for management Goal Statements about visions of what planning processes should achieve in a specific time frame, influenced by the purpose of conservation planning To conserve a representative sample of a region's biodiversity Input Resources, including time, data or money, invested in developing the outputs and outcomes of the planning process Spatial data on land classes across a planning region Output Products produced during the planning process that contribute to outcomes of conservation planning Map of potential conservation areas that meet quantitative objectives for species representation Outcome Positive or negative changes to forms of natural, financial, social, human, or institutional capital in the region where the planning process occurs, and perhaps also in other regions Changes in existing legislation to protect new areas identified by a conservation plan (institutional outcome), or improved coordination of conservation activities between NGOs working across the planning region (social outcome) Impact The overall contribution of outcomes to the achievement of goals. Impact might be determined using counterfactual analysis, or the assessment of outcomes in the absence of conservation intervention. Avoided loss of native forest, measured in hectares, attributable to the outcomes of planning (hectares of forest that would have been lost in the absence of planning minus actual hectares lost) Indicator Variables used to assess, qualitatively or quantitatively, changes in or occurrences of outputs, outcomes or impacts New skills acquired by staff following a training workshop, demonstrating increased human capital in the planning team Over the last decade, evaluation in the conservation sector has developed to track performance of conservation interventions and policies, with approaches including results‐based management and outcomes monitoring ( Salafsky . 2002 ; Kapos . 2008 ). There are still, however, substantial gaps in knowledge of the effectiveness of many conservation interventions and of the advantages and limitations of different interventions. In this section, we discuss the potential merits of evaluation for conservation planning and review the extent to which evaluation is already applied. Evaluation has multiple benefits for developers, managers and funders of planning processes ( Rossi . 2004 ). First, and most importantly, evaluation can assess whether planning has met its goals and produced intended outcomes, or identify unintended consequences, positive or negative. Second, evaluation provides accountability by measuring cost‐effectiveness. Third, evaluation can provide insights into which parts of the planning process contribute most to effective outcomes. Finally, evaluation can cut through and resolve data‐free debates about the relative merits of different approaches to planning. Overall, evaluation can enhance institutional learning ( Knight . 2006a ), inform choices between different approaches, and build greater confidence among funders. Existing approaches to evaluation of conservation planning Formal evaluation of conservation planning has not been well documented and there has been limited reflection on specific exercises. Possible reasons for this lack of attention include: a tendency throughout the conservation sector to promote successes and bury failures ( Redford & Taber 2000 ); limited staff or financial resources allocated to evaluation; a failure to track outcomes from the outset of planning processes; the protracted nature of implementation of plans; and poor connections between those responsible for developing and implementing plans ( Knight . 2011 ). Some lessons for the evaluation of conservation planning come from evaluation in the broad areas of natural resource management and land use policy and planning. Program evaluation of the Albemarle‐Pamlico Estuarine Study ( Korfmacher 1998 ) observed positive short‐term outcomes such as knowledge transfer between managers and greater integration of scientific information. However, this study also cautioned that expectations about planning outcomes should be managed because demonstrating achievement of long‐term objectives can be difficult, ( Korfmacher 1998 ). Through interviews with municipal planners, Stokes (2009) found that greater public awareness and education were required to integrate biodiversity needs into local development strategies. Responses by different stakeholders engaged in coastal and marine coastal resource management emphasized the importance of fairness and legitimacy in public consultations ( Dalton 2006 ). For conservation planning specifically, some recent studies have documented experiences, compared approaches, assessed specialist tools, and investigated specific outcomes. Knight . (2008) assessed the extent to which systematic assessments have led to implemented actions. Other planners have reflected on personal or organizational experiences in particular planning regions ( Medley 2004 ; Smith . 2006 ; Green . 2009 ), provided perspectives on planning within environmental realms ( Leslie 2005 ; Osmond . 2010 ), or reviewed specific approaches ( Didier . 2009 ; Henson . 2009 ; Morrison . 2009 ). While these studies provide useful information, they are largely narrative, context‐specific, and narrow in scope and purpose. Collectively, these studies do not provide comprehensive, quantitative methods for measuring the effectiveness of planning. The few assessments of planning exercises have tended to focus on quantifiable outputs (e.g., number of hectares reserved, maps produced, and development of software, Knight . 2006b ) rather than indicators that directly demonstrate changes in ecological or social systems as a result of planning ( Figure 1 ). Indicators of outcomes have focused primarily on ecological criteria (e.g., percentage of species populations in reserve systems, Klein . 2008 ). Less explored are the financial, social, and institutional effects of planning such as conflict resolution, mainstreaming of conservation thinking into public and private sectors, and leverage of funding. Furthermore, we are not aware that any study has reported impact, or what changes to biodiversity might be directly attributed to planning. Given protected areas are concentrated in areas of least value for extractive uses ( Scott . 2001 ; Joppa & Pfaff 2009 ), even outcomes such as formal protection do not necessarily lead to net benefits for biodiversity and other natural values. 1 Results chain illustrating linkages between processes, outputs, outcomes, and impacts of social and natural aspects of conservation planning. Processes are specific events or series of decisions and actions that deliver outputs and outcomes. Outputs are products and knowledge created during planning exercises. Outcomes are the short‐ and medium‐term effects of the planning exercise on biodiversity and people. Impacts are the ultimate changes that people undertaking conservation planning hope to achieve. The pathway in gray illustrates a potential negative social outcome that could emerge from a planning process. Challenges to evaluation of conservation planning Evaluation of conservation planning is confronted by several conceptual and methodological challenges. Effective conservation planning can be defined in many ways. Outcomes can be described in visionary, qualitative terms (e.g., the planning exercise improved protection of land cover types in reserves), or more quantitatively (e.g., partnerships and collaborations between public and private stakeholders increased by 20%). Developers of plans, implementing agencies, stakeholders, and technical experts might have very different perspectives on goals and successful outcomes. In the face of this diversity, there are no established methods for evaluation of conservation planning. Measuring outcomes also faces the difficulty of attributing benefits to aspects of planning processes, distinct from confounding factors ( Ferraro & Pattanayak 2006 ). How, for example, can we measure the increased funding resulting only from planning, and not from unrelated advocacy or policy? The use of counterfactuals, or control areas without planning, is generally not feasible due to the complexity of physical, biological, social, and political contexts for planning. The sheer number of variables that would have to be matched, combined with limited resources devoted to evaluation, make controls impractical ( Margoluis . 2009 ). The difficulty of program evaluation is also exacerbated by the protracted implementation of most plans. Few planning exercises involve rapid, complete application of actions across regional designs that facilitate pre‐ and postimplementation comparisons. Examples of this minority are the Regional Forest Agreement process in New South Wales ( Pressey . 2009 ) and the re‐zoning of the Great Barrier Reef Marine Park ( Fernandes . 2005 ). More typically, especially on private land and in community‐managed inshore waters in developing countries, implementation proceeds incrementally over years or decades, during which time initial plans are modified and their influence on outcomes declines. A framework for evaluation of conservation planning To assess the effectiveness of conservation planning, an outcomes‐based approach is needed. We propose a framework for identifying and categorizing outcomes ( Table 3 ) against which results of planning exercises can be compared. Within this framework, potential indicators are described to track progress towards short‐ and medium‐term outcomes. Indicators could detect changes during planning processes, perhaps extending over several years, or during the implementation of conservation actions, which could take many years. 3 Potential outcomes from conservation planning with corresponding indicators and approaches for tracking changes to indicators categorized by five forms of capital. Information on types of outcomes was derived from responses by conservation planners and implementers of plans to questions about outcomes emerging from planning processes. Also listed are frequencies of occurrence of outcomes in interview responses when mentioned by respondents ( n = 28). Approaches focus on outcomes monitoring which involves tracking changes to indicators (e.g., species, budgets, attitudes) before and after planning (and perhaps during protracted planning processes) Capital Outcome Frequency in responses (%) Indicator with example(s) Approach (pre‐ and postplanning) Natural Representation of biodiversity 18 Percentages of biological features occurring in new conservation areas, e.g.,% change of critical habitat occurrence Compare number or extent of different features occurring in new and existing areas using spatial data on ecological patterns and processes Reduction in loss or degradation of natural values 26 Extent and intensity of threatening processes, e.g., deforestation; exploitation Compare extent and intensity of threatening processes occurring in conservation areas and matched areas outside conservation areas using data on distribution and intensity of resource use Persistence of biodiversity 18 Population sizes of selected species relative to minimum viable populations or metapopulations Compare estimated population sizes and connectivity between populations across all conservation areas Maintenance of ecosystem services 7 Rate of service provision e.g., freshwater volume (m 3 ) Compare rates or extents of services provided Financial Transparency in conservation investments 14 Investment patterns linked to stated priorities, e.g., alignment of investments with relative priority of areas identified Compare locations of investments by agencies and NGOs in relation to systematically identified priorities Efficiency of operations 14 Expenditures by implementing agencies in relation to conservation benefits, e.g., amounts allocated in specific areas or actions relative to priorities Based on financial statements and budgetary data, compare spending relative to conservation benefits achieved Maximized benefit given limited budget 18 Benefit to biodiversity per $ spent, e.g., extent of critical habitat secured/ha given acquisition costs Compare cost‐efficiency of final portfolio with minimum‐cost solution or amount required to secure a random selection of conservation areas with the same total extent Leverage of additional funds or in‐kind support 21 Proportion of additional funds received, e.g., % change in annual budget of implementing agency attributable to new donors Compare sources and extent of funding received using financial statements of planning and implementing agencies Social Collaboration among agencies 18 Frequency and type of partnerships or relationships, e.g., number of project partners working together Compare connectedness and exchanges between actors/agencies using social network analysis Coordination between different actors 11 Alignment between projects and strategies by different actors, e.g., complementarity between priorities and activities of different conservation agencies and NGOs in planning region Compare priorities and strategies between different agencies and NGOs Trust in planning process 18 Perceptions of planning process and outputs by stakeholders, e.g., % of stakeholders with positive view of plan Compare perceived values of planning processes and products using data from questionnaires or interviews with stakeholders Sharing datasets between agencies 25 Datasets made accessible and available to different agencies in planning process, e.g., occurrence of data‐sharing agreements Compare datasets and other resources available for use by different agencies Shared vision 29 Consensus among stakeholders and planners about future outlook for Compare perceptions or positions of key stakeholders using content biodiversity and conservation in a region, e.g., document stating agreement by key actors and team analysis or interviews or questionnaires Attitudes of stakeholders 39 Disposition or feelings of stakeholders towards biodiversity and conservation; e.g., % of stakeholders that have positive attitude to local biodiversity Compare perceived values of conservation and/or biodiversity using data from questionnaires or interviews with stakeholders Human Raised awareness of biodiversity or conservation 39 Level of knowledge of biodiversity, conservation or other related issues among different individuals, e.g., % of stakeholders aware of priority biodiversity features in region Compare extent or level of knowledge among individuals using data from interviews or questionnaires New knowledge of ecological or social values 50 Information on ecological or social values created or synthesized during planning processes, e.g., new data on opportunity costs of small landholders Compare information available using content analysis of documents or surveys Learning applied in future plans 25 Use of new knowledge or skills applied in subsequent plans, e.g., application of new decision tool by members of planning team in subsequent planning processes Compare knowledge and skills of planning team using data generated by self‐assessment questionnaires Behavior of decision makers 14 Frequency of positive or negative behavior of decision makers; e.g., voting records of local policymakers Compare behavior of decision makers, such as policymakers, using data from content analysis of records of local government authorities Institutional Influence on future decision making by organization or partners 46 Occurrence of decisions based on recommendations of plan, e.g., allocation of funding or other resources to specific areas Compare investments in specific area or issues using content analysis of documents/reports Self‐sustaining strategies 7 Continuing implementation and planning revisions, e.g., adaptation of a regional plan to new information Investment in staff time and funds in updating data sets and prioritization analyses as new information becomes available Role of implementing agency 14 Ongoing implementation and updates by implementing organization, e.g., proportion of decisions and actions by one organization focused on the plan Compare affiliations of people responsible for decisions and actions using content analysis of work plans, minutes of meetings, and other documents Consideration of conservation issues in decision making by other sectors 11 Frequency of considerations by local government authorities, e.g., downloads of conservation plan from website, alignment of land use decisions with priority areas Compare local government documentation of approved development proposals and protective zonings relative to identified priority areas Integration of priorities into policies, conventions, or legislation 11 Explicit consideration of priorities in policy, e.g., occurrence of areas, conservation features, and/or actions in policy statements or regulations Compare consideration of priorities in policy using content analysis of documents or reports Influence on resource‐use planning 25 Avoidance by developers of priority areas, e.g., occurrence of development applications in priority areas Compare frequency of planning applications in priority areas using content analysis of development applications Protected areas expanded 25 Establishment of new areas within protected area network, e.g., gazettal of areas under national or regional legislation Compare network of existing protected areas using agency records or content analysis of documents Our framework is adapted from the DfiD (UK Department for International Development) Livelihoods Framework ( DfID 1999 ), which categorizes people's assets as different types of capital: natural, financial, social, human, and physical. We have substituted physical capital (e.g., assets produced or manufactured) for institutional capital (e.g., change to the structure or function of institutions). Capital is the product of investment, yielding a flow of benefits over time. This framework is appropriate to conservation planning for several reasons. First, it has been applied and tested in evaluation of conservation strategies such as Integrated Conservation and Development Projects (ICDPs) ( Garnett . 2007 ). Second, it expands evaluation beyond ecological assets to include the other potential outcomes of planning. Finally, it recognizes that different stakeholders will value different outcomes and so accommodates diverse views of effectiveness. For each type of capital, below, we describe potential outcomes and their relationship to goals and expectations, drawing on literature and our interviews. Natural capital Natural capital is the stock and flow of goods and services provided by ecosystems, including the diversity of species, regulating processes, and supporting services ( Costanza . 1997 ). Conservation and restoration of natural capital are key motivations for conservation planning, reflected by frequently stated goals to safeguard biodiversity from threatening processes ( Margules & Pressey 2000 ) and maintain ecosystem services ( Egoh . 2007 ; Cowling . 2008 ). Only a quarter or less of responses explicitly noted outcomes for biodiversity persistence, reduction in threats, or maintenance of ecosystem services ( Table 3 ). Evaluation of changes in natural capital from planning is constrained by the time lags needed to observe changes that occur beyond the lifespan of most planning exercises. Reporting on conservation planning has therefore focused primarily on outputs such as achievement of objectives for conservation features in potential conservation areas (e.g., Rebelo & Siegfried 1992 ; Pressey & Tully 1994 ; Hansen . 2011 ). Reporting on conservation actions resulting from planning has been rare, and either short‐term (e.g., Knight . 2011 ) or covering a few regions where complete and rapid application of conservation areas increased coverage of species or ecosystems (e.g., California Marine Life Protection Act planning initiative, Gleason . 2010 ). Even these few studies of positive outcomes are limited, however. Implementation of actions might not necessarily lead to positive trends in the abundance and composition of species of concern ( Kapos . 2009 ). Protection of areas of little interest for extractive uses ( Scott . 2001 ) could result in a failure to protect imminently threatened features elsewhere. Moreover, there is limited evidence of ecological outcomes in the extensive parts of the world where application of actions occurs gradually over many years. In a few cases, negative effects of planning on natural capital have been documented, such as preemptive deforestation of areas identified in planning documents ( Murray & Wear 1998 ; Venter . 2010 ). Overall, reporting on the influence of conservation planning on natural capital has been weak, concentrating on outputs and with little demonstration of impact. Financial capital Financial capital is the gain or savings of cash, property or goods that represent the wealth or economic value of an individual or organization. Improved financial capital resulting from planning might include mobilized funding or leverage of additional funds for conservation, measurable by changes in budget allocations, operating costs, and funding levels. Financial outcomes might also reflect cost savings or efficiency of operations due to priority‐setting and strategic allocation of resources, given that cost‐efficiency is a key principle of systematic conservation planning ( Carwardine . 2008 ). Nongovernment organizations have acknowledged financial benefits from planning in the form of raised awareness among potential donors ( Morrison . 2009 ). These benefits have not, however, been rigorously documented beyond highlighting by our interviewees ( Table 3 ). New funding or leveraged support might emerge only after the planning process is complete, although some funds might flow in the short term based on successes during early stages of planning. It might also be difficult to attribute financial gains to specific planning exercises relative to other potential influences, including those beyond the control of the organizations that benefit. Reporting of cost savings as a result of planning has largely been documented as the potential efficiency of conservation designs that avoid, where possible, areas with high acquisition or opportunity costs ( Ando . 1998 ; Stewart . 2003 ). This benefit is an output, analogous to the potential, not actual, gain in natural capital arising from well designed but notional conservation areas. Conservation planning also has potential negative effects on cost‐efficiency, such as increased land values resulting from areas earmarked for action ( Armsworth . 2006 ; Polasky 2006 ). Planning itself also has financial and opportunity costs: the staff and funds invested in planning could have been directed to other activities. Retrospective analyses measuring actual cost savings from planning versus business‐as‐usual spending have not been applied to implemented conservation actions. If financial efficiency is an intended consequence of planning, then greater efforts are needed to demonstrate actual financial outcomes. Social capital Social capital represents the relationships and interactions between individuals and groups with productive benefits ( Pretty & Ward 2001 ). Planning processes and decision‐making arrangements invariably shape social capital through activities such as stakeholder consultation and negotiations, expert workshops, and interactions within the planning team. A widely cited purpose of the planning process itself is to bring stakeholders together to share a common vision ( Loucks . 2004 ; Knight . 2006b ). Conservation planning is a social process in that conservation aims ultimately to alter the attitudes and behavior of people through incentives or regulation ( Cowling . 2004 ; Cowling and Wilhelm‐Rechmann 2007 ). Planning can also influence the priorities of other actors working on conservation within planning regions, recognizing the importance of cooperation and collaboration ( Groves . 2002 ). Changes in social capital can be defined broadly in four categories: reciprocity and exchanges, norms, connectedness, and trust ( Pretty & Ward 2001 ), all of which were highlighted by responses in our interviews ( Table 3 ). Reciprocity and exchanges involve sharing ideas and information between individual actors and organizations (e.g., Klein 2008 ) or collaborations between organizations (e.g., Higgins . 2007 ). Planning could, for example, positively alter the attitudes of policy makers towards biodiversity and conservation, an outcome frequently cited by our interview respondents ( Table 3 ). Norms refer to mutually agreed values that place group interests above those of individuals. An example is reaching agreement among stakeholders on common goals for conserving biodiversity (e.g., Gonzales . 2003 ). Connectedness refers to relationships between individuals and social groups that are influenced by planning, perhaps through conflict resolution during stakeholder workshops. Finally, trust is exemplified by acceptance of the results of planning processes by individuals or institutions. Positive changes in social capital are largely documented within case studies of specific planning exercises ( The Nature Conservancy 2003 ; Henson . 2009 ; Knight . 2011 ). These examples demonstrate interactions and relationships during the planning process, but there are no quantitative assessments of different aspects of social capital across multiple exercises or approaches, or of the extent to which these effects were achieved or sustained. While improved social capital is generally not a primary motivation for conservation planning, the need for more explicit monitoring and evaluation of social variables is underlined by increasing recognition of social influences on the achievement of planning goals ( Cowling & Wilhelm‐Rechmann 2007 ; Ban . in press ). Negative social outcomes of planning are also possible. These could include opposition to recommendations, fatigue from lengthy negotiations, perceptions that sharing of personal information was not repaid with respect, alienation of local stakeholders by external agencies ( Smith . 2009 ), or conflict between resource users and conservationists ( Gleason . 2010 ). Human capital Human capital comprises knowledge or skills that enable people to develop strategies to achieve their objectives ( DfID 1999 ), providing the foundation for the other four types of capital. In conservation planning, skills and knowledge might be improved through discussion about objectives or training or application in analytical methods or decision‐support tools. New knowledge of ecological and social values, for example, was the most frequently cited outcome of conservation planning highlighted by our interviewees ( Table 3 ). Sharing datasets from different agencies and combining data layers might also enhance knowledge of the social and ecological conditions of a planning region. Changes in human capital tend to be reported as outputs such as numbers of people trained or items of educational materials distributed. Less emphasis has been given to demonstrable outcomes such as changes in skills, increase in knowledge, and how these improvements might affect other planning outcomes, such as increased support for conservation actions, collaboration among partners, or better executed planning processes. Institutional capital Institutions are formal and informal rules that guide the behavior and decisions of individuals and organizations ( Ostrom 1990 ). Institutional capital is the capacity, structure, or functioning of institutions through formal means (e.g., laws and regulations) or informal arrangements (e.g., cultural norms applied in governing natural resource uses) ( Platje 2008 ). Almost half of our interview respondents noted influences of planning on the structure, policy, or practice of planning agencies or other partners ( Table 3 ). Conservation planning has the potential to influence institutional outcomes such as policies and practices, expectations of groups of people, and the priorities of organizations through lessons learned about effective or ineffective approaches. Longer‐term outcomes in natural capital might be facilitated by changes to institutional thinking or structures, perhaps reflected in strengthening of legislation to protect areas or avoidance of priority conservation areas by developers. Some evidence demonstrates that planning exercises have influenced legislation and regulations, in particular for the establishment of protected areas ( Pressey 1998 ; Fernandes . 2005 ; Gleason . 2010 ). Some organizations have embraced systematic approaches to planning as part of their missions in response to arguments for the potential of these methods combined with successful applied examples (e.g., the Nature Conservancy 2011 ). Notwithstanding the few success stories related to institutional capital, changes in formal or informal institutions arising from planning have not been demonstrated convincingly or comprehensively. Implications for improved evaluation of conservation planning There are substantial potential benefits from evaluation in terms of accountability, demonstrating impact, organizational learning, and maximizing benefits. At the same time, evaluation requires new thinking about the nature of the planning process itself and how it is approached. We explore below the conceptual, operational, organizational, and policy implications of improved evaluation of conservation planning. Conceptual implications Lack of agreement persists on how to define the effectiveness of conservation planning. The planning process includes not only the design of notional conservation areas (the assessment phase) but also subsequent implementation of actions and ongoing management of conservation areas ( Pressey & Bottrill 2009 ). Limiting evaluation to outputs ( Figure 1 ), where most demonstrations of potential benefits have focused, restricts evaluation to the assessment stages of planning, reinforcing the insulation between spatial analysis and the outcomes and impact that developers and implementers of plans say they want to achieve ( Bottrill . 2012 ). To frame appropriate and informative evaluation questions, indicators relevant to the ultimate goals of planning must be developed and applied. In addition, the focus of evaluation has been concentrated unduly on aspects of natural capital, including implementation of conservation areas. Not only is direct implementation of actions an incomplete indicator of benefits to natural capital ( Pressey & Bottrill 2009 ) but the four other types of capital ( Table 3 ) warrant their own attention. Extending evaluation to financial, social, human and institutional outcomes will avoid underestimating the benefits of planning and motivate improved approaches that maximize benefits defined more comprehensively. Operational implications Our framework suggests indicators to reflect outcomes of planning ( Table 3 ), but indicators alone do not translate into a rigorous and informative evaluation strategy. Like the process of conservation planning itself, an evaluation system is an integrated series of analyses and decisions. Frameworks and operational models for conservation planning, (e.g., Knight . 2006a ; Pressey & Bottrill 2009 ), do not explicitly consider evaluation. An evaluation strategy would formulate key evaluation questions, set objectives for evaluation, identify appropriate timing for collecting and analyzing data, and articulate ways of resourcing and providing incentives for evaluation. The advantages of considering evaluation early and explicitly in the planning process include timely collection of required data and ownership of evaluation results by stakeholders. Given the potential applications of evaluation throughout the planning process and its importance in shaping decisions, an evaluation strategy should be part of the initial scoping stage of planning, with adjustment as planning proceeds ( Pressey & Bottrill 2009 ). Organizational implications Measuring potential outcomes of planning appears to be a large task ( Table 3 ), even if we have only begun to understand its scope. As with planning itself, evaluation is often viewed as an activity that diverts valuable resources from conservation actions. Organizations must therefore consider the value of knowledge gained from evaluation versus the cost of gaining that knowledge. Rigorous and effective evaluation of planning will require organizations to commit not only considerable time and resources but to embrace learning in an explicit framework. Perhaps most challenging is the prospect of exposure to internal and, in many cases, external scrutiny. Overwhelmingly, reporting of conservation outcomes has promoted successes and buried failures ( Redford & Taber 2000 ). Reasons for this lack of transparency stem from a fear of losing donor funding and weak incentives for critical assessment or experimentation ( Kapos . 2008 ). Stronger incentives could come from government and donors themselves, if not from within organizations, in the interests of maintaining organizational knowledge and learning from mistakes to make conservation planning more effective. Policy implications Transparency of results associated with evaluation also has consequences for policymakers. Inadequate evidence of planning effectiveness has serious implications for the accountability of large sums spent on conservation planning by government agencies and NGOs ( Table 1 ), and could undermine confidence in identified priorities. Public accountability becomes an issue when governments and agencies make ad hoc decisions about spending on the environment. Audits of such decisions, like recent ones in Australia (Auditor‐General of Queensland 2010 ; Victorian Auditor‐General 2011 ) need to be broadened to consider not only management of established protected areas but the effectiveness of the whole process of conservation planning. Very large government spending programs, such as Australia's Natural Heritage Trust, should also be scrutinized with the framework we propose here. Accountability and evaluation have been embraced by other public sectors, such as public health and education, yet conservation has been slow to adopt evidence‐based approaches ( Pullin & Knight 2001 ). Obstacles to progress in evaluation of conservation planning include lack of data on social and ecological outcomes, limited sharing of data or lessons learned for proprietary reasons ( Keene & Pullin 2011 ), and the complexity of the social, economic, and ecological contexts for conservation ( Margoluis . 2009 ). However, Keene & Pullin (2011) suggest that, with improved data flows and self‐organization, the conservation sector might overcome these obstacles to achieve an “effectiveness revolution.” Acknowledgments The authors are grateful to Angela Guerrero, Andrew Knight, Morena Mills, and three anonymous reviewers for comments on an earlier manuscript. MCB thanks Dan Segan, Andrew Knight, Michael Mascia, Hugh Possingham, Ken Vance‐Borland, and our interview participants for inspiring discussions that influenced the concepts presented here. MCB was supported by a Northcote Graduate Scholarship and funding from the Australian Commonwealth Environment Research Facility. RLP acknowledges support from the Australian Research Council. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Conservation Letters Wiley

The effectiveness and evaluation of conservation planning

Download PDF
14 pages

Loading next page...
 
/lp/wiley/the-effectiveness-and-evaluation-of-conservation-planning-Il97gkemuC

References (67)

Publisher
Wiley
Copyright
"Copyright © 2012 Wiley Subscription Services, Inc., A Wiley Company"
eISSN
1755-263X
DOI
10.1111/j.1755-263X.2012.00268.x
Publisher site
See Article on Publisher Site

Abstract

Introduction There is a lack of evidence‐based knowledge on the effectiveness of different conservation interventions, and limited monitoring and evaluation to generate better knowledge ( Pullin & Knight 2001 ; Sutherland . 2004 ; Ferraro & Pattanayak 2006 ). Without evidence, justification of investment in different interventions is thought to be guided by experience at best ( Cook . 2010 ) and anecdote and received wisdom at worst ( Sutherland . 2004 ). To date, there has been no comprehensive reporting or evidence to demonstrate the costs and benefits of investing in conservation planning. Program evaluation, a systematic process for judging effectiveness of programs ( Patton 2008 ), could provide a conceptual framework and a set of tools to enhance our knowledge of the value of conservation planning. In this review, we make the case for explicit program evaluation of regional‐scale planning exercises—those applied across domains defined by ecoregions, catchments, or jurisdictions with extents in the order of 10 2 to 10 5 km 2 . Our review has three specific aims: (1) to outline the benefits and challenges of more explicit and comprehensive evaluation of conservation planning; (2) to identify the range of indicators available to evaluate the outcomes of conservation planning; and (3) to explore conceptual, operational, organizational, and policy implications of evaluating planning exercises in the future. We hope this review highlights the importance of evaluation in promoting defensible conservation planning processes, adds realism to expectations about planning and, ultimately, helps to improve conservation outcomes. Over the last two decades, the application of conservation planning ( Margules & Pressey 2000 ) has expanded rapidly. It has influenced conservation priorities by international organizations ( Groves . 2002 ; Sanderson . 2002 ), guided policy decisions by government agencies ( Airame . 2003 ; Pressey . 2009 ), and resulted in hundreds of publications in the academic literature. In particular, considerable advances have been made in producing conservation plans—maps of priority conservation areas—with hundreds of plans developed around the world at a variety of scales. However, despite several prominent success stories of planning leading to real conservation gains ( Finkel 1998 ; Airame . 2003 ; Fernandes . 2005 ), there has been limited empirical evidence to demonstrate actual outcomes ( Knight . 2008 ). Without more comprehensive evaluation of the tangible outcomes of conservation planning, investments in planning are not accountable, decisions are not defensible, and learning from past experiences is limited. This is all the more important because each of the hundreds of planning exercises undertaken entails substantial costs ( Table 1 ). 1 Costs of conservation planning (all estimates exclude costs of applying actions) Planning process Lead organization Cost estimate (US$) Duration Area (km 2 ) Source Description Cape Action Plan for the Environment Project, South Africa South African National Botanical Institute (SANBI) ca. 1 million 1998–2000 90,000 Cowling & Pressey 2003 Project comprised four major steps: (1) situation assessment (including conservation planning component); (2) development of strategy to involve stakeholders and institutions; (3) action plan formulation, and (4) fundraising. Great Barrier Reef Marine Park rezoning plan Great Barrier Reef (GBR) Marine Park Authority ca. 6–8 million 1999–2004 344,400 Osmond et al. 2010 Costs covered staff salaries, staff time, and staff operating costs for the new zoning plan for the entire GBR spread over a 5‐year period. The Australian government also funded a structural adjustment package (∼$100 million) designed to assist fishers, fishery related businesses, employees, and communities affected by the rezoning. Adirondack State Park Landscape, USA Wildlife Conservation Society † Monetary value unavailable. Costs reported in relation to staff time (37 person‐weeks) 2000–2003 19,700 Didier et al. 2009 Stages of planning process were measured in relation to staff time (person‐weeks). Most time devoted to selection of focal species (14 weeks); mapping of biological landscape (8 weeks); and mapping of human landscape (8 weeks) Maputaland Transnational Conservation Plan, Mozambique/South Africa/Swaziland University of Kent (DICE) ca. 215,143 2003–2005 17,000 Smith & Leader‐Williams 2006 Costs covered convening of training workshops, staff time, development of GIS databases, production of decision support system (CLUZ), and design of tri‐national conservation plan through participatory workshops with key regional stakeholders followed by publicity and presentation of results. Kimbe Bay Conservation Action Planning, Papua New Guinea The Nature Conservancy ca. 400,000 2004–2007 13,000 Green et al. 2009 Total cost for the scientific design process in Kimbe Bay (excluding community engagement and implementation) was divided principally between scientific research (54%), staff time (35%) and workshops (10%). California Central Coast Marine Protected Areas Network, USA California Department of Fish and Game ca. 2.5 million 2005–2007 3,000 Raab 2006 Expenditures included: (1) executive/general administration and project management (25%); (2) facilitation and outreach (18%); (3) Department of Fish and Game lead staff and costs (18%); and (4) data preparation and analysis (12%). For the purpose of this review, we define conservation planning broadly as the design of conservation areas. Conservation areas can include IUCN protected area categories I–VI, as well as places where diverse management arrangements, such as zoning, offsets, easements, invasive species control, and restoration are applied. Most conservation planning today can be termed “systematic” ( Margules & Pressey 2000 ), involving explicit, quantitative objectives and leading to systems of potential conservation areas with emergent properties related to complementarity and connectivity ( Margules & Pressey 2000 ). The lack of evidence for the effectiveness of planning applies to systematic methods and to other approaches such as ranking procedures ( Smith & Theberge 1987 ) and expert‐based prioritizations ( Cowling . 2003 ). We focus here on whether designing conservation areas leads to actual conservation outcomes. Specifically, therefore, we refer here to evaluation of conservation assessment ( Knight . 2006a ), excluding the evaluation of stages involving implementation of actions or subsequent management of conservation areas (stages 10 and 11 of Pressey & Bottrill 2009 ) covered elsewhere ( Margoluis & Salafsky 1998 ; Ferraro & Pattanayak 2006 ; Hockings . 2006 ). In the sections that follow, we discuss the rationale for, and possible approaches to, measuring the effectiveness of conservation planning. We draw extensively on literature, but also present new qualitative data on planning outcomes gathered from semistructured interviews with experts involved in developing and/or implementing planning exercises. Qualitative data on effectiveness of planning The qualitative data collected in semistructured interviews were intended to fill large gaps in the literature concerning perceptions of the effectiveness of planning. Interviewees were selected using snowball sampling, by which selected interviewees identify other appropriate participants ( Lewis‐Beck . 2004 ). Among our participants, we interviewed people involved in the development of plans (“planners,” n = 15) and people responsible for updating plans, representing stakeholders, or providing technical support in applying actions arising from conservation plans (“implementers,” n = 13). We aimed for a representative sample of people working across socioeconomic contexts and environmental realms. More details on the affiliations of participants and the interview instrument are described in the Supporting Information. The interview format followed an open framework of themed questions that were conversational and flexible ( Mason 2002 ). Responses were analyzed using a content analysis of interview transcripts to draw out trends and frequency of responses. The interviews revealed perceptions of both positive and negative outcomes emerging from planning exercises by those intimately involved in their development and application. Evaluation and conservation planning Evaluation is a systematic method for collecting, analyzing, and assessing information on the effectiveness of projects and policies in relation to stated goals ( Patton 2008 ) and takes multiple forms. The most important basic distinction in evaluation is between summative and formative approaches ( Rossi . 2004 ). Formative approaches, including process evaluation and needs assessment, strengthen or improve the program being evaluated. Summative approaches, including outcome and impact evaluations, examine effects or outcomes of programs. We are primarily concerned here with the application of summative evaluation approaches. Table 2 describes definitions of key terms used in evaluation referred to in this review. 2 Key evaluation terms in the context of conservation planning Term Definition Example Purpose The intended aim(s) of conservation planning Identification of priority areas for management Goal Statements about visions of what planning processes should achieve in a specific time frame, influenced by the purpose of conservation planning To conserve a representative sample of a region's biodiversity Input Resources, including time, data or money, invested in developing the outputs and outcomes of the planning process Spatial data on land classes across a planning region Output Products produced during the planning process that contribute to outcomes of conservation planning Map of potential conservation areas that meet quantitative objectives for species representation Outcome Positive or negative changes to forms of natural, financial, social, human, or institutional capital in the region where the planning process occurs, and perhaps also in other regions Changes in existing legislation to protect new areas identified by a conservation plan (institutional outcome), or improved coordination of conservation activities between NGOs working across the planning region (social outcome) Impact The overall contribution of outcomes to the achievement of goals. Impact might be determined using counterfactual analysis, or the assessment of outcomes in the absence of conservation intervention. Avoided loss of native forest, measured in hectares, attributable to the outcomes of planning (hectares of forest that would have been lost in the absence of planning minus actual hectares lost) Indicator Variables used to assess, qualitatively or quantitatively, changes in or occurrences of outputs, outcomes or impacts New skills acquired by staff following a training workshop, demonstrating increased human capital in the planning team Over the last decade, evaluation in the conservation sector has developed to track performance of conservation interventions and policies, with approaches including results‐based management and outcomes monitoring ( Salafsky . 2002 ; Kapos . 2008 ). There are still, however, substantial gaps in knowledge of the effectiveness of many conservation interventions and of the advantages and limitations of different interventions. In this section, we discuss the potential merits of evaluation for conservation planning and review the extent to which evaluation is already applied. Evaluation has multiple benefits for developers, managers and funders of planning processes ( Rossi . 2004 ). First, and most importantly, evaluation can assess whether planning has met its goals and produced intended outcomes, or identify unintended consequences, positive or negative. Second, evaluation provides accountability by measuring cost‐effectiveness. Third, evaluation can provide insights into which parts of the planning process contribute most to effective outcomes. Finally, evaluation can cut through and resolve data‐free debates about the relative merits of different approaches to planning. Overall, evaluation can enhance institutional learning ( Knight . 2006a ), inform choices between different approaches, and build greater confidence among funders. Existing approaches to evaluation of conservation planning Formal evaluation of conservation planning has not been well documented and there has been limited reflection on specific exercises. Possible reasons for this lack of attention include: a tendency throughout the conservation sector to promote successes and bury failures ( Redford & Taber 2000 ); limited staff or financial resources allocated to evaluation; a failure to track outcomes from the outset of planning processes; the protracted nature of implementation of plans; and poor connections between those responsible for developing and implementing plans ( Knight . 2011 ). Some lessons for the evaluation of conservation planning come from evaluation in the broad areas of natural resource management and land use policy and planning. Program evaluation of the Albemarle‐Pamlico Estuarine Study ( Korfmacher 1998 ) observed positive short‐term outcomes such as knowledge transfer between managers and greater integration of scientific information. However, this study also cautioned that expectations about planning outcomes should be managed because demonstrating achievement of long‐term objectives can be difficult, ( Korfmacher 1998 ). Through interviews with municipal planners, Stokes (2009) found that greater public awareness and education were required to integrate biodiversity needs into local development strategies. Responses by different stakeholders engaged in coastal and marine coastal resource management emphasized the importance of fairness and legitimacy in public consultations ( Dalton 2006 ). For conservation planning specifically, some recent studies have documented experiences, compared approaches, assessed specialist tools, and investigated specific outcomes. Knight . (2008) assessed the extent to which systematic assessments have led to implemented actions. Other planners have reflected on personal or organizational experiences in particular planning regions ( Medley 2004 ; Smith . 2006 ; Green . 2009 ), provided perspectives on planning within environmental realms ( Leslie 2005 ; Osmond . 2010 ), or reviewed specific approaches ( Didier . 2009 ; Henson . 2009 ; Morrison . 2009 ). While these studies provide useful information, they are largely narrative, context‐specific, and narrow in scope and purpose. Collectively, these studies do not provide comprehensive, quantitative methods for measuring the effectiveness of planning. The few assessments of planning exercises have tended to focus on quantifiable outputs (e.g., number of hectares reserved, maps produced, and development of software, Knight . 2006b ) rather than indicators that directly demonstrate changes in ecological or social systems as a result of planning ( Figure 1 ). Indicators of outcomes have focused primarily on ecological criteria (e.g., percentage of species populations in reserve systems, Klein . 2008 ). Less explored are the financial, social, and institutional effects of planning such as conflict resolution, mainstreaming of conservation thinking into public and private sectors, and leverage of funding. Furthermore, we are not aware that any study has reported impact, or what changes to biodiversity might be directly attributed to planning. Given protected areas are concentrated in areas of least value for extractive uses ( Scott . 2001 ; Joppa & Pfaff 2009 ), even outcomes such as formal protection do not necessarily lead to net benefits for biodiversity and other natural values. 1 Results chain illustrating linkages between processes, outputs, outcomes, and impacts of social and natural aspects of conservation planning. Processes are specific events or series of decisions and actions that deliver outputs and outcomes. Outputs are products and knowledge created during planning exercises. Outcomes are the short‐ and medium‐term effects of the planning exercise on biodiversity and people. Impacts are the ultimate changes that people undertaking conservation planning hope to achieve. The pathway in gray illustrates a potential negative social outcome that could emerge from a planning process. Challenges to evaluation of conservation planning Evaluation of conservation planning is confronted by several conceptual and methodological challenges. Effective conservation planning can be defined in many ways. Outcomes can be described in visionary, qualitative terms (e.g., the planning exercise improved protection of land cover types in reserves), or more quantitatively (e.g., partnerships and collaborations between public and private stakeholders increased by 20%). Developers of plans, implementing agencies, stakeholders, and technical experts might have very different perspectives on goals and successful outcomes. In the face of this diversity, there are no established methods for evaluation of conservation planning. Measuring outcomes also faces the difficulty of attributing benefits to aspects of planning processes, distinct from confounding factors ( Ferraro & Pattanayak 2006 ). How, for example, can we measure the increased funding resulting only from planning, and not from unrelated advocacy or policy? The use of counterfactuals, or control areas without planning, is generally not feasible due to the complexity of physical, biological, social, and political contexts for planning. The sheer number of variables that would have to be matched, combined with limited resources devoted to evaluation, make controls impractical ( Margoluis . 2009 ). The difficulty of program evaluation is also exacerbated by the protracted implementation of most plans. Few planning exercises involve rapid, complete application of actions across regional designs that facilitate pre‐ and postimplementation comparisons. Examples of this minority are the Regional Forest Agreement process in New South Wales ( Pressey . 2009 ) and the re‐zoning of the Great Barrier Reef Marine Park ( Fernandes . 2005 ). More typically, especially on private land and in community‐managed inshore waters in developing countries, implementation proceeds incrementally over years or decades, during which time initial plans are modified and their influence on outcomes declines. A framework for evaluation of conservation planning To assess the effectiveness of conservation planning, an outcomes‐based approach is needed. We propose a framework for identifying and categorizing outcomes ( Table 3 ) against which results of planning exercises can be compared. Within this framework, potential indicators are described to track progress towards short‐ and medium‐term outcomes. Indicators could detect changes during planning processes, perhaps extending over several years, or during the implementation of conservation actions, which could take many years. 3 Potential outcomes from conservation planning with corresponding indicators and approaches for tracking changes to indicators categorized by five forms of capital. Information on types of outcomes was derived from responses by conservation planners and implementers of plans to questions about outcomes emerging from planning processes. Also listed are frequencies of occurrence of outcomes in interview responses when mentioned by respondents ( n = 28). Approaches focus on outcomes monitoring which involves tracking changes to indicators (e.g., species, budgets, attitudes) before and after planning (and perhaps during protracted planning processes) Capital Outcome Frequency in responses (%) Indicator with example(s) Approach (pre‐ and postplanning) Natural Representation of biodiversity 18 Percentages of biological features occurring in new conservation areas, e.g.,% change of critical habitat occurrence Compare number or extent of different features occurring in new and existing areas using spatial data on ecological patterns and processes Reduction in loss or degradation of natural values 26 Extent and intensity of threatening processes, e.g., deforestation; exploitation Compare extent and intensity of threatening processes occurring in conservation areas and matched areas outside conservation areas using data on distribution and intensity of resource use Persistence of biodiversity 18 Population sizes of selected species relative to minimum viable populations or metapopulations Compare estimated population sizes and connectivity between populations across all conservation areas Maintenance of ecosystem services 7 Rate of service provision e.g., freshwater volume (m 3 ) Compare rates or extents of services provided Financial Transparency in conservation investments 14 Investment patterns linked to stated priorities, e.g., alignment of investments with relative priority of areas identified Compare locations of investments by agencies and NGOs in relation to systematically identified priorities Efficiency of operations 14 Expenditures by implementing agencies in relation to conservation benefits, e.g., amounts allocated in specific areas or actions relative to priorities Based on financial statements and budgetary data, compare spending relative to conservation benefits achieved Maximized benefit given limited budget 18 Benefit to biodiversity per $ spent, e.g., extent of critical habitat secured/ha given acquisition costs Compare cost‐efficiency of final portfolio with minimum‐cost solution or amount required to secure a random selection of conservation areas with the same total extent Leverage of additional funds or in‐kind support 21 Proportion of additional funds received, e.g., % change in annual budget of implementing agency attributable to new donors Compare sources and extent of funding received using financial statements of planning and implementing agencies Social Collaboration among agencies 18 Frequency and type of partnerships or relationships, e.g., number of project partners working together Compare connectedness and exchanges between actors/agencies using social network analysis Coordination between different actors 11 Alignment between projects and strategies by different actors, e.g., complementarity between priorities and activities of different conservation agencies and NGOs in planning region Compare priorities and strategies between different agencies and NGOs Trust in planning process 18 Perceptions of planning process and outputs by stakeholders, e.g., % of stakeholders with positive view of plan Compare perceived values of planning processes and products using data from questionnaires or interviews with stakeholders Sharing datasets between agencies 25 Datasets made accessible and available to different agencies in planning process, e.g., occurrence of data‐sharing agreements Compare datasets and other resources available for use by different agencies Shared vision 29 Consensus among stakeholders and planners about future outlook for Compare perceptions or positions of key stakeholders using content biodiversity and conservation in a region, e.g., document stating agreement by key actors and team analysis or interviews or questionnaires Attitudes of stakeholders 39 Disposition or feelings of stakeholders towards biodiversity and conservation; e.g., % of stakeholders that have positive attitude to local biodiversity Compare perceived values of conservation and/or biodiversity using data from questionnaires or interviews with stakeholders Human Raised awareness of biodiversity or conservation 39 Level of knowledge of biodiversity, conservation or other related issues among different individuals, e.g., % of stakeholders aware of priority biodiversity features in region Compare extent or level of knowledge among individuals using data from interviews or questionnaires New knowledge of ecological or social values 50 Information on ecological or social values created or synthesized during planning processes, e.g., new data on opportunity costs of small landholders Compare information available using content analysis of documents or surveys Learning applied in future plans 25 Use of new knowledge or skills applied in subsequent plans, e.g., application of new decision tool by members of planning team in subsequent planning processes Compare knowledge and skills of planning team using data generated by self‐assessment questionnaires Behavior of decision makers 14 Frequency of positive or negative behavior of decision makers; e.g., voting records of local policymakers Compare behavior of decision makers, such as policymakers, using data from content analysis of records of local government authorities Institutional Influence on future decision making by organization or partners 46 Occurrence of decisions based on recommendations of plan, e.g., allocation of funding or other resources to specific areas Compare investments in specific area or issues using content analysis of documents/reports Self‐sustaining strategies 7 Continuing implementation and planning revisions, e.g., adaptation of a regional plan to new information Investment in staff time and funds in updating data sets and prioritization analyses as new information becomes available Role of implementing agency 14 Ongoing implementation and updates by implementing organization, e.g., proportion of decisions and actions by one organization focused on the plan Compare affiliations of people responsible for decisions and actions using content analysis of work plans, minutes of meetings, and other documents Consideration of conservation issues in decision making by other sectors 11 Frequency of considerations by local government authorities, e.g., downloads of conservation plan from website, alignment of land use decisions with priority areas Compare local government documentation of approved development proposals and protective zonings relative to identified priority areas Integration of priorities into policies, conventions, or legislation 11 Explicit consideration of priorities in policy, e.g., occurrence of areas, conservation features, and/or actions in policy statements or regulations Compare consideration of priorities in policy using content analysis of documents or reports Influence on resource‐use planning 25 Avoidance by developers of priority areas, e.g., occurrence of development applications in priority areas Compare frequency of planning applications in priority areas using content analysis of development applications Protected areas expanded 25 Establishment of new areas within protected area network, e.g., gazettal of areas under national or regional legislation Compare network of existing protected areas using agency records or content analysis of documents Our framework is adapted from the DfiD (UK Department for International Development) Livelihoods Framework ( DfID 1999 ), which categorizes people's assets as different types of capital: natural, financial, social, human, and physical. We have substituted physical capital (e.g., assets produced or manufactured) for institutional capital (e.g., change to the structure or function of institutions). Capital is the product of investment, yielding a flow of benefits over time. This framework is appropriate to conservation planning for several reasons. First, it has been applied and tested in evaluation of conservation strategies such as Integrated Conservation and Development Projects (ICDPs) ( Garnett . 2007 ). Second, it expands evaluation beyond ecological assets to include the other potential outcomes of planning. Finally, it recognizes that different stakeholders will value different outcomes and so accommodates diverse views of effectiveness. For each type of capital, below, we describe potential outcomes and their relationship to goals and expectations, drawing on literature and our interviews. Natural capital Natural capital is the stock and flow of goods and services provided by ecosystems, including the diversity of species, regulating processes, and supporting services ( Costanza . 1997 ). Conservation and restoration of natural capital are key motivations for conservation planning, reflected by frequently stated goals to safeguard biodiversity from threatening processes ( Margules & Pressey 2000 ) and maintain ecosystem services ( Egoh . 2007 ; Cowling . 2008 ). Only a quarter or less of responses explicitly noted outcomes for biodiversity persistence, reduction in threats, or maintenance of ecosystem services ( Table 3 ). Evaluation of changes in natural capital from planning is constrained by the time lags needed to observe changes that occur beyond the lifespan of most planning exercises. Reporting on conservation planning has therefore focused primarily on outputs such as achievement of objectives for conservation features in potential conservation areas (e.g., Rebelo & Siegfried 1992 ; Pressey & Tully 1994 ; Hansen . 2011 ). Reporting on conservation actions resulting from planning has been rare, and either short‐term (e.g., Knight . 2011 ) or covering a few regions where complete and rapid application of conservation areas increased coverage of species or ecosystems (e.g., California Marine Life Protection Act planning initiative, Gleason . 2010 ). Even these few studies of positive outcomes are limited, however. Implementation of actions might not necessarily lead to positive trends in the abundance and composition of species of concern ( Kapos . 2009 ). Protection of areas of little interest for extractive uses ( Scott . 2001 ) could result in a failure to protect imminently threatened features elsewhere. Moreover, there is limited evidence of ecological outcomes in the extensive parts of the world where application of actions occurs gradually over many years. In a few cases, negative effects of planning on natural capital have been documented, such as preemptive deforestation of areas identified in planning documents ( Murray & Wear 1998 ; Venter . 2010 ). Overall, reporting on the influence of conservation planning on natural capital has been weak, concentrating on outputs and with little demonstration of impact. Financial capital Financial capital is the gain or savings of cash, property or goods that represent the wealth or economic value of an individual or organization. Improved financial capital resulting from planning might include mobilized funding or leverage of additional funds for conservation, measurable by changes in budget allocations, operating costs, and funding levels. Financial outcomes might also reflect cost savings or efficiency of operations due to priority‐setting and strategic allocation of resources, given that cost‐efficiency is a key principle of systematic conservation planning ( Carwardine . 2008 ). Nongovernment organizations have acknowledged financial benefits from planning in the form of raised awareness among potential donors ( Morrison . 2009 ). These benefits have not, however, been rigorously documented beyond highlighting by our interviewees ( Table 3 ). New funding or leveraged support might emerge only after the planning process is complete, although some funds might flow in the short term based on successes during early stages of planning. It might also be difficult to attribute financial gains to specific planning exercises relative to other potential influences, including those beyond the control of the organizations that benefit. Reporting of cost savings as a result of planning has largely been documented as the potential efficiency of conservation designs that avoid, where possible, areas with high acquisition or opportunity costs ( Ando . 1998 ; Stewart . 2003 ). This benefit is an output, analogous to the potential, not actual, gain in natural capital arising from well designed but notional conservation areas. Conservation planning also has potential negative effects on cost‐efficiency, such as increased land values resulting from areas earmarked for action ( Armsworth . 2006 ; Polasky 2006 ). Planning itself also has financial and opportunity costs: the staff and funds invested in planning could have been directed to other activities. Retrospective analyses measuring actual cost savings from planning versus business‐as‐usual spending have not been applied to implemented conservation actions. If financial efficiency is an intended consequence of planning, then greater efforts are needed to demonstrate actual financial outcomes. Social capital Social capital represents the relationships and interactions between individuals and groups with productive benefits ( Pretty & Ward 2001 ). Planning processes and decision‐making arrangements invariably shape social capital through activities such as stakeholder consultation and negotiations, expert workshops, and interactions within the planning team. A widely cited purpose of the planning process itself is to bring stakeholders together to share a common vision ( Loucks . 2004 ; Knight . 2006b ). Conservation planning is a social process in that conservation aims ultimately to alter the attitudes and behavior of people through incentives or regulation ( Cowling . 2004 ; Cowling and Wilhelm‐Rechmann 2007 ). Planning can also influence the priorities of other actors working on conservation within planning regions, recognizing the importance of cooperation and collaboration ( Groves . 2002 ). Changes in social capital can be defined broadly in four categories: reciprocity and exchanges, norms, connectedness, and trust ( Pretty & Ward 2001 ), all of which were highlighted by responses in our interviews ( Table 3 ). Reciprocity and exchanges involve sharing ideas and information between individual actors and organizations (e.g., Klein 2008 ) or collaborations between organizations (e.g., Higgins . 2007 ). Planning could, for example, positively alter the attitudes of policy makers towards biodiversity and conservation, an outcome frequently cited by our interview respondents ( Table 3 ). Norms refer to mutually agreed values that place group interests above those of individuals. An example is reaching agreement among stakeholders on common goals for conserving biodiversity (e.g., Gonzales . 2003 ). Connectedness refers to relationships between individuals and social groups that are influenced by planning, perhaps through conflict resolution during stakeholder workshops. Finally, trust is exemplified by acceptance of the results of planning processes by individuals or institutions. Positive changes in social capital are largely documented within case studies of specific planning exercises ( The Nature Conservancy 2003 ; Henson . 2009 ; Knight . 2011 ). These examples demonstrate interactions and relationships during the planning process, but there are no quantitative assessments of different aspects of social capital across multiple exercises or approaches, or of the extent to which these effects were achieved or sustained. While improved social capital is generally not a primary motivation for conservation planning, the need for more explicit monitoring and evaluation of social variables is underlined by increasing recognition of social influences on the achievement of planning goals ( Cowling & Wilhelm‐Rechmann 2007 ; Ban . in press ). Negative social outcomes of planning are also possible. These could include opposition to recommendations, fatigue from lengthy negotiations, perceptions that sharing of personal information was not repaid with respect, alienation of local stakeholders by external agencies ( Smith . 2009 ), or conflict between resource users and conservationists ( Gleason . 2010 ). Human capital Human capital comprises knowledge or skills that enable people to develop strategies to achieve their objectives ( DfID 1999 ), providing the foundation for the other four types of capital. In conservation planning, skills and knowledge might be improved through discussion about objectives or training or application in analytical methods or decision‐support tools. New knowledge of ecological and social values, for example, was the most frequently cited outcome of conservation planning highlighted by our interviewees ( Table 3 ). Sharing datasets from different agencies and combining data layers might also enhance knowledge of the social and ecological conditions of a planning region. Changes in human capital tend to be reported as outputs such as numbers of people trained or items of educational materials distributed. Less emphasis has been given to demonstrable outcomes such as changes in skills, increase in knowledge, and how these improvements might affect other planning outcomes, such as increased support for conservation actions, collaboration among partners, or better executed planning processes. Institutional capital Institutions are formal and informal rules that guide the behavior and decisions of individuals and organizations ( Ostrom 1990 ). Institutional capital is the capacity, structure, or functioning of institutions through formal means (e.g., laws and regulations) or informal arrangements (e.g., cultural norms applied in governing natural resource uses) ( Platje 2008 ). Almost half of our interview respondents noted influences of planning on the structure, policy, or practice of planning agencies or other partners ( Table 3 ). Conservation planning has the potential to influence institutional outcomes such as policies and practices, expectations of groups of people, and the priorities of organizations through lessons learned about effective or ineffective approaches. Longer‐term outcomes in natural capital might be facilitated by changes to institutional thinking or structures, perhaps reflected in strengthening of legislation to protect areas or avoidance of priority conservation areas by developers. Some evidence demonstrates that planning exercises have influenced legislation and regulations, in particular for the establishment of protected areas ( Pressey 1998 ; Fernandes . 2005 ; Gleason . 2010 ). Some organizations have embraced systematic approaches to planning as part of their missions in response to arguments for the potential of these methods combined with successful applied examples (e.g., the Nature Conservancy 2011 ). Notwithstanding the few success stories related to institutional capital, changes in formal or informal institutions arising from planning have not been demonstrated convincingly or comprehensively. Implications for improved evaluation of conservation planning There are substantial potential benefits from evaluation in terms of accountability, demonstrating impact, organizational learning, and maximizing benefits. At the same time, evaluation requires new thinking about the nature of the planning process itself and how it is approached. We explore below the conceptual, operational, organizational, and policy implications of improved evaluation of conservation planning. Conceptual implications Lack of agreement persists on how to define the effectiveness of conservation planning. The planning process includes not only the design of notional conservation areas (the assessment phase) but also subsequent implementation of actions and ongoing management of conservation areas ( Pressey & Bottrill 2009 ). Limiting evaluation to outputs ( Figure 1 ), where most demonstrations of potential benefits have focused, restricts evaluation to the assessment stages of planning, reinforcing the insulation between spatial analysis and the outcomes and impact that developers and implementers of plans say they want to achieve ( Bottrill . 2012 ). To frame appropriate and informative evaluation questions, indicators relevant to the ultimate goals of planning must be developed and applied. In addition, the focus of evaluation has been concentrated unduly on aspects of natural capital, including implementation of conservation areas. Not only is direct implementation of actions an incomplete indicator of benefits to natural capital ( Pressey & Bottrill 2009 ) but the four other types of capital ( Table 3 ) warrant their own attention. Extending evaluation to financial, social, human and institutional outcomes will avoid underestimating the benefits of planning and motivate improved approaches that maximize benefits defined more comprehensively. Operational implications Our framework suggests indicators to reflect outcomes of planning ( Table 3 ), but indicators alone do not translate into a rigorous and informative evaluation strategy. Like the process of conservation planning itself, an evaluation system is an integrated series of analyses and decisions. Frameworks and operational models for conservation planning, (e.g., Knight . 2006a ; Pressey & Bottrill 2009 ), do not explicitly consider evaluation. An evaluation strategy would formulate key evaluation questions, set objectives for evaluation, identify appropriate timing for collecting and analyzing data, and articulate ways of resourcing and providing incentives for evaluation. The advantages of considering evaluation early and explicitly in the planning process include timely collection of required data and ownership of evaluation results by stakeholders. Given the potential applications of evaluation throughout the planning process and its importance in shaping decisions, an evaluation strategy should be part of the initial scoping stage of planning, with adjustment as planning proceeds ( Pressey & Bottrill 2009 ). Organizational implications Measuring potential outcomes of planning appears to be a large task ( Table 3 ), even if we have only begun to understand its scope. As with planning itself, evaluation is often viewed as an activity that diverts valuable resources from conservation actions. Organizations must therefore consider the value of knowledge gained from evaluation versus the cost of gaining that knowledge. Rigorous and effective evaluation of planning will require organizations to commit not only considerable time and resources but to embrace learning in an explicit framework. Perhaps most challenging is the prospect of exposure to internal and, in many cases, external scrutiny. Overwhelmingly, reporting of conservation outcomes has promoted successes and buried failures ( Redford & Taber 2000 ). Reasons for this lack of transparency stem from a fear of losing donor funding and weak incentives for critical assessment or experimentation ( Kapos . 2008 ). Stronger incentives could come from government and donors themselves, if not from within organizations, in the interests of maintaining organizational knowledge and learning from mistakes to make conservation planning more effective. Policy implications Transparency of results associated with evaluation also has consequences for policymakers. Inadequate evidence of planning effectiveness has serious implications for the accountability of large sums spent on conservation planning by government agencies and NGOs ( Table 1 ), and could undermine confidence in identified priorities. Public accountability becomes an issue when governments and agencies make ad hoc decisions about spending on the environment. Audits of such decisions, like recent ones in Australia (Auditor‐General of Queensland 2010 ; Victorian Auditor‐General 2011 ) need to be broadened to consider not only management of established protected areas but the effectiveness of the whole process of conservation planning. Very large government spending programs, such as Australia's Natural Heritage Trust, should also be scrutinized with the framework we propose here. Accountability and evaluation have been embraced by other public sectors, such as public health and education, yet conservation has been slow to adopt evidence‐based approaches ( Pullin & Knight 2001 ). Obstacles to progress in evaluation of conservation planning include lack of data on social and ecological outcomes, limited sharing of data or lessons learned for proprietary reasons ( Keene & Pullin 2011 ), and the complexity of the social, economic, and ecological contexts for conservation ( Margoluis . 2009 ). However, Keene & Pullin (2011) suggest that, with improved data flows and self‐organization, the conservation sector might overcome these obstacles to achieve an “effectiveness revolution.” Acknowledgments The authors are grateful to Angela Guerrero, Andrew Knight, Morena Mills, and three anonymous reviewers for comments on an earlier manuscript. MCB thanks Dan Segan, Andrew Knight, Michael Mascia, Hugh Possingham, Ken Vance‐Borland, and our interview participants for inspiring discussions that influenced the concepts presented here. MCB was supported by a Northcote Graduate Scholarship and funding from the Australian Commonwealth Environment Research Facility. RLP acknowledges support from the Australian Research Council.

Journal

Conservation Letters – Wiley

Published: Dec 1, 2012

Keywords: ; ; ; ; ; ;

There are no references for this article.