Prevalence, thresholds and the performance of presence–absence models

Prevalence, thresholds and the performance of presence–absence models The use of species distribution models to understand and predict species' distributions necessitates tests of fit to empirical data. Numerous performance metrics have been proposed, many of which require continuous occurrence probabilities to be converted to binary ‘present or absent’ predictions using threshold transformations. It is widely accepted that both continuous and binary performance metrics should be independent of prevalence (the proportion of locations that are occupied). However, because these metrics have been mostly assessed on a case‐specific basis, there are few general guidelines for measuring performance. Here, we develop a conceptual framework for classifying performance metrics, based on whether they are sensitive to prevalence, and whether they require binary predictions. We use this framework to investigate how these performance metric properties influence the predictions made by the models they select. A literature survey reveals that binary metrics are widely employed and that prevalence‐independent metrics are used more frequently than prevalence‐dependent metrics. However, we show that prevalence‐dependent metrics are essential to assess the numerical accuracy of model predictions and are more useful in applications that require occupancy estimates. Furthermore, we demonstrate that in comparison with continuous metrics, binary metrics often select models that have reduced ability to separate presences from absences, make predictions which over‐ or underestimate occupancy and give misleading estimates of uncertainty. Importantly, models selected using binary metrics will often be of reduced practical use even when applied to ecological problems that require binary decision‐making. We suggest that SDM performance should be assessed using prevalence‐dependent performance metrics whenever the absolute values of occurrence predictions are important and that continuous metrics should be used instead of binary metrics whenever possible. We thus recommend the wider application of prevalence‐dependent continuous metrics, particularly likelihood‐based metrics such as Akaike's Information Criterion (AIC), to assess the performance of presence–absence models. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Methods in Ecology and Evolution Wiley

Prevalence, thresholds and the performance of presence–absence models

Loading next page...
 
/lp/wiley/prevalence-thresholds-and-the-performance-of-presence-absence-models-OnZvZwcG0y
Publisher
Wiley
Copyright
Copyright © 2014 British Ecological Society
ISSN
2041-210X
eISSN
2041-210X
DOI
10.1111/2041-210X.12123
Publisher site
See Article on Publisher Site

Abstract

The use of species distribution models to understand and predict species' distributions necessitates tests of fit to empirical data. Numerous performance metrics have been proposed, many of which require continuous occurrence probabilities to be converted to binary ‘present or absent’ predictions using threshold transformations. It is widely accepted that both continuous and binary performance metrics should be independent of prevalence (the proportion of locations that are occupied). However, because these metrics have been mostly assessed on a case‐specific basis, there are few general guidelines for measuring performance. Here, we develop a conceptual framework for classifying performance metrics, based on whether they are sensitive to prevalence, and whether they require binary predictions. We use this framework to investigate how these performance metric properties influence the predictions made by the models they select. A literature survey reveals that binary metrics are widely employed and that prevalence‐independent metrics are used more frequently than prevalence‐dependent metrics. However, we show that prevalence‐dependent metrics are essential to assess the numerical accuracy of model predictions and are more useful in applications that require occupancy estimates. Furthermore, we demonstrate that in comparison with continuous metrics, binary metrics often select models that have reduced ability to separate presences from absences, make predictions which over‐ or underestimate occupancy and give misleading estimates of uncertainty. Importantly, models selected using binary metrics will often be of reduced practical use even when applied to ecological problems that require binary decision‐making. We suggest that SDM performance should be assessed using prevalence‐dependent performance metrics whenever the absolute values of occurrence predictions are important and that continuous metrics should be used instead of binary metrics whenever possible. We thus recommend the wider application of prevalence‐dependent continuous metrics, particularly likelihood‐based metrics such as Akaike's Information Criterion (AIC), to assess the performance of presence–absence models.

Journal

Methods in Ecology and EvolutionWiley

Published: Jan 1, 2014

Keywords: ; ; ; ; ; ; ; ; ;

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create folders to
organize your research

Export folders, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off