Access the full text.
Sign up today, get DeepDyve free for 14 days.
J. Hanley, B. McNeil (1982)
The meaning and use of the area under a receiver operating characteristic (ROC) curve.Radiology, 143 1
J. Cramer (1999)
Predicitive performance of the binary logit model in unbalanced samplesInvestigative Ophthalmology & Visual Science
J. Wiens (1994)
Species diversity in ecological communities: edited by Robert E. Ricklefs and Dolph Schluter University of Chicago Press, 1993. $105.00 hbk, $35.00 pbk (414 pages) ISBN 0 226 71822 0/0 226 71823 9Trends in Ecology and Evolution, 9
A. Jiménez‐Valverde, J. Lobo (2007)
Threshold criteria for conversion of probability of species presence to either–or presence–absenceActa Oecologica-international Journal of Ecology, 31
M. Austin (2007)
Species distribution models and ecological theory: A critical assessment and some possible new approachesEcological Modelling, 200
M. Zweig, G. Campbell (1993)
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.Clinical chemistry, 39 4
E. Wiley, K. McNyset, A. Peterson, C. Robins, Aimee Stewart (2003)
Niche Modeling and Geographic Range Predictions in the Marine Environment Using a Machine-learning Algorithm
I. Hanski (1998)
Metapopulation dynamicsNature, 396
R. Ricklefs, D. Schluter (1993)
Species diversity in ecological communities: historical and geographical perspectives.
S. Baker, P. Pinsky (2001)
A proposed design and analysis for comparing digital and analog mammography: special receiver operating characteristic methods for cancer screeningJournal of the American Statistical Association, 96
A. Jiménez‐Valverde, J. Lobo (2006)
The ghost of unbalanced species distribution data in geographical model predictionsDiversity and Distributions, 12
R. Real, A. Barbosa, J. Vargas (2006)
Obtaining Environmental Favourability Functions from Logistic RegressionEnvironmental and Ecological Statistics, 13
Mary Thompson, Walter Zucchini (1989)
On the statistical analysis of ROC curves.Statistics in medicine, 8 10
C. Ferri, P. Flach, J. Hernández‐Orallo, A. Senad (2005)
Proceedings of the Second Workshop on ROC Analysis in Machine LearningLecture Notes in Computer Science
George Forman, I. Cohen (2005)
Beware the Null Hypothesis: Critical Value Tables for Evaluating Classifiers
Ron Pulliam (1988)
Sources, Sinks, and Population RegulationThe American Naturalist, 132
L. Brotóns, W. Thuiller, M. Araújo, A. Hirzel (2004)
Presence-absence versus presence-only modelling methods for predicting bird habitat suitabilityEcography, 27
A. Peterson (2006)
Uses and requirements of ecological niche models and related distributional modelsBiodiversity Informatics, 3
S. Manel, H. Williams, S. Ormerod (2001)
Evaluating presence-absence models in ecology: the need to account for prevalenceJournal of Applied Ecology, 38
J. McPherson, W. Jetz, D. Rogers (2004)
The effects of species’ range sizes on the accuracy of distribution models: ecological phenomenon or statistical artefact?Journal of Applied Ecology, 41
Robin Engler, A. Guisan, Luca Rechsteiner (2004)
An improved approach for predicting the distribution of rare and endangered species from occurrence and pseudo-absence dataJournal of Applied Ecology, 41
E. Wiley, K. McNyset, A. Peterson, C. Robins, Aimee Stewart (2003)
Niche Modeling Perspective on Geographic Range Predictions in the Marine Environment Using a Machine-learning AlgorithmOceanography, 16
R. Brennan, Dale Prediger (1981)
Coefficient Kappa: Some Uses, Misuses, and AlternativesEducational and Psychological Measurement, 41
Canran Liu, P. Berry, T. Dawson, R. Pearson (2005)
Selecting thresholds of occurrence in the prediction of species distributionsEcography, 28
A. Fielding, J. Bell (1997)
A review of methods for the assessment of prediction errors in conservation presence/absence modelsEnvironmental Conservation, 24
J. McPherson, W. Jetz (2007)
Effects of species’ ecology on the accuracy of distribution modelsEcography, 30
J. Lobo, J. Verdú, C. Numa (2006)
Environmental and geographical factors affecting the Iberian distribution of flightless Jekelius species (Coleoptera: Geotrupidae)Diversity and Distributions, 12
N. Adams, D. Hand (1999)
Comparing classifiers when the misallocation costs are uncertainPattern Recognit., 32
H. Pulliam (2000)
On the relationship between niche and distributionEcology Letters, 3
J. Swets (1988)
Measuring the accuracy of diagnostic systems.Science, 240 4857
J. Candela, C. Rasmussen, Fabian Sinz, O. Bousquet, B. Scholkopf (2005)
Evaluating Predictive Uncertainty Challenge
J. Pearce, S. Ferrier (2000)
Evaluating the predictive performance of habitat models developed using logistic regressionEcological Modelling, 133
M. Pepe (2000)
Receiver Operating Characteristic MethodologyJournal of the American Statistical Association, 95
A. Bradley (1997)
The use of the area under the ROC curve in the evaluation of machine learning algorithmsPattern Recognit., 30
B. Schr (2006)
Constrain to perform: Regularization of habitat models
C. Ferri, Peter Flach, J. Hernández-Orallo (2005)
Modifying ROC Curves to Incorporate Predicted Probabilities
P. Hernández, C. Graham, L. Master, Deborah Albert (2006)
The effect of sample size and species characteristics on performance of different species distribution modeling methodsEcography, 29
J. Arntzen (2006)
From descriptive to predictive distribution models: a working example with Iberian amphibians and reptilesFrontiers in Zoology, 3
(2007)
AUC: a misleading measure © 2007 The Authors Global Ecology and Biogeography, Journal compilation
J. Candela, Magnini I, F. Lauria (2006)
Machine Learning Challenges. Evaluating Predictive Uncertainty, Visual Object Classification, and Recognising Tectual Entailment
J. Elith, Catherine Graham, Robert Anderson, Miroslav Dudı́k, Simon Ferrier, A. Guisan, R. Hijmans, F. Huettmann, J. Leathwick, Anthony Lehmann, Jin Li, Lúcia Lohmann, Bette Loiselle, G. Manion, Craig Moritz, Miguel Nakamura, Yoshinori Nakazawa, J. Overton, A. Peterson, Steven Phillips, Karen Richardson, R. Scachetti-Pereira, R. Schapire, Jorge Soberón, Stephen Williams, M. Wisz, N. Zimmermann (2006)
Novel methods improve prediction of species' distributions from occurrence dataEcography, 29
T. Fawcett (2004)
ROC graphs: notes and practical considerations for data mining researchersEnvironmental Conservation
F. Provost, Tom Fawcett (2000)
Robust Classification for Imprecise EnvironmentsMachine Learning, 42
N. Oreskes, K. Shrader-Frechette, K. Belitz (1994)
Verification, Validation, and Confirmation of Numerical Models in the Earth SciencesScience, 263
R. Anderson, D. Lew, A. Peterson (2003)
Evaluating predictive models of species’ distributions: criteria for selecting optimal modelsEcological Modelling, 162
R.Gil Pontius, Laura Schneider (2001)
Land-cover change model validation by an ROC method for the Ipswich watershed, Massachusetts, USAAgriculture, Ecosystems & Environment, 85
D. Hosmer, Stanley Lemesbow (1980)
Goodness of fit tests for the multiple logistic regression modelCommunications in Statistics-theory and Methods, 9
Editor: José Alexandre F. Diniz-Filho
M. Termansen, C. McClean, C. Preston (2006)
The use of genetic algorithms and Bayesian classification to model species distributionsEcological Modelling, 192
I. Vaughan, S. Ormerod (2005)
The continuing challenges of testing species distribution modelsJournal of Applied Ecology, 42
D. Hosmer, T. Hosmer, S. Cessie, S. Lemeshow (1997)
A comparison of goodness-of-fit tests for the logistic regression model.Statistics in medicine, 16 9
I. Vaughan, S. Ormerod (2003)
Improving the Quality of Distribution Models for Conservation by Addressing Shortcomings in the Field Collection of Training DataConservation Biology, 17
(2004)
ROC plotting and AUC calculation transferability test software
S. Baker, P. Pinsky (2001)
A Proposed Design and Analysis for Comparing Digital and Analog MammographyJournal of the American Statistical Association, 96
D. Hosmer, S. Lemeshow (1991)
Applied Logistic Regression
S. Lemeshow, D. Hosmer (1982)
A review of goodness of fit statistics for use in the development of logistic regression models.American journal of epidemiology, 115 1
Joaquin Quiñonero-Candela, Ido Dagan, B. Magnini, F. Alché-Buc (2006)
Machine Learning Challenges, Evaluating Predictive Uncertainty, Visual Object Classification and Recognizing Textual Entailment, First PASCAL Machine Learning Challenges Workshop, MLCW 2005, Southampton, UK, April 11-13, 2005, Revised Selected Papers, 3944
Jacob Cohen (1960)
A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 20
The area under the receiver operating characteristic (ROC) curve, known as the AUC, is currently considered to be the standard method to assess the accuracy of predictive distribution models. It avoids the supposed subjectivity in the threshold selection process, when continuous probability derived scores are converted to a binary presence–absence variable, by summarizing overall model performance over all possible thresholds. In this manuscript we review some of the features of this measure and bring into question its reliability as a comparative measure of accuracy between model results. We do not recommend using AUC for five reasons: (1) it ignores the predicted probability values and the goodness‐of‐fit of the model; (2) it summarises the test performance over regions of the ROC space in which one would rarely operate; (3) it weights omission and commission errors equally; (4) it does not give information about the spatial distribution of model errors; and, most importantly, (5) the total extent to which models are carried out highly influences the rate of well‐predicted absences and the AUC scores.
Global Ecology and Biogeography – Wiley
Published: Mar 1, 2008
Keywords: ; ; ; ; ;
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.