Access the full text.
Sign up today, get DeepDyve free for 14 days.
Åke Berg, U. Gärdenfors, T. Proschwitz (2004)
Logistic regression models for predicting occurrence of terrestrial molluscs in southern Sweden – importance of environmental data quality and model complexityEcography, 27
G. Cumming (2000)
Using habitat models to map diversity: pan‐African species richness of ticks (Acari: Ixodida)Journal of Biogeography, 27
C. Metz (1978)
Basic principles of ROC analysis.Seminars in nuclear medicine, 8 4
M. Zweig, G. Campbell (1993)
Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine.Clinical chemistry, 39 4
Setzler Bailey, R. Haines-Young, C. Watkins (2002)
Species presence in fragmented landscapes: modelling of species requirements at the national levelBiological Conservation, 108
G. Moisen, T. Frescino (2002)
Comparing five modelling techniques for predicting forest characteristicsEcological Modelling, 157
B. Huntley, P. Berry, W. Cramer, A. McDonald (1995)
Special Paper: Modelling Present and Potential Future Ranges of Some European Higher Plants Using Climate Response SurfacesJournal of Biogeography, 22
A. Fielding, P. Haworth (1995)
Testing the Generality of Bird‐Habitat ModelsConservation Biology, 9
T. D’heygere, P. Goethals, N. Pauw (2003)
Use of genetic algorithms to select input variables in decision tree models for the prediction of benthic macroinvertebratesEcological Modelling, 160
J. Olden, Donald Jackson (2001)
Fish–Habitat Relationships in Lakes: Gaining Predictive and Explanatory Insight by Using Artificial Neural NetworksTransactions of The American Fisheries Society, 130
A. Lehmann (1998)
GIS modeling of submerged macrophyte distribution using Generalized Additive ModelsPlant Ecology, 139
M. Robertson, C. Peter, M. Villet, B. Ripley (2003)
Comparing models for predicting species’ potential distributions: a case study using correlative and mechanistic predictive modelling techniquesEcological Modelling, 164
Stacy Özesmi, Uygar Özesmi (1999)
An artificial neural network approach to spatial habitat modelling with interspecific interactionEcological Modelling, 116
R. Schapire, Y. Singer, A. Singhal (1998)
Boosting and Rocchio applied to text filtering
R. Pearson, T. Dawson, P. Berry, P. Harrison (2002)
SPECIES: A Spatial Evaluation of Climate Impact on the Envelope of SpeciesEcological Modelling, 154
S. Brosse, J. Guégan, J. Tourenq, S. Lek (1999)
The use of artificial neural networks to assess fish abundance and spatial occupancy in the littoral zone of a mesotrophic lakeEcological Modelling, 120
M. Austin (2002)
Spatial prediction of species distribution: an interface between ecological theory and statistical modellingEcological Modelling, 157
A. Guisan, T. Edwards, T. Hastie (2002)
Generalized linear and generalized additive models in studies of species distributions: setting the sceneEcological Modelling, 157
A. Woolf, C. Nielsen, T. Weber, Tara Gibbs-Kieninger (2002)
Statewide modeling of bobcat, Lynx rufus, habitat in Illinois, USABiological Conservation, 104
S. Manel, H. Williams, S. Ormerod (2001)
Evaluating presence-absence models in ecology: the need to account for prevalenceJournal of Applied Ecology, 38
A. Guisan, N. Zimmermann (2000)
Predictive habitat distribution models in ecologyEcological Modelling, 135
U. Mooney (2000)
Using Information Extraction to Aid the Discovery of Prediction Rules from Text
Hao-Che Pu, Y. Hung (1995)
Use of artificial neural networksEnvironmental Management and Health, 6
S. Cantor, Charlotte Sun, G. Tortolero-Luna, R. Richards-Kortum, M. Follen (1999)
A comparison of C/B ratios from studies using receiver operating characteristic curve analysis.Journal of clinical epidemiology, 52 9
A. Guisan, S. Weiss, Andrew Weiss (1999)
GLM versus CCA spatial modeling of plant species distributionPlant Ecology, 143
Huntley Huntley (1995)
Modelling present and potential future ranges of some European higher plants using climate response surfacesJ. Biogeogr., 22
A. Fielding, J. Bell (1997)
A review of methods for the assessment of prediction errors in conservation presence/absence modelsEnvironmental Conservation, 24
David Stockwell, A. Peterson (2002)
Effects of sample size on accuracy of species distribution modelsEcological Modelling, 148
S. Manel, J. Dias, S. Ormerod (1999)
Comparing discriminant analysis, neural networks and logistic regression for predicting species distributions: a case study with a Himalayan river birdEcological Modelling, 120
A. Guisan, J. Theurillat, F. Kienast (1998)
Predicting the potential distribution of plant species in an alpine environmentJournal of Vegetation Science, 9
P. Berry, T. Dawson, P. Harrison, R. Pearson (2002)
Modelling potential impacts of climate change on the bioclimatic envelope of species in Britain and IrelandGlobal Ecology and Biogeography, 11
D. Hilbert, B. Ostendorf (2001)
The utility of artificial neural networks for modelling the distribution of vegetation in past, present and future climatesEcological Modelling, 146
Yvonne Collingham, R. Wadsworth, B. Huntley, P. Hulme (2000)
Predicting the spatial distribution of non‐indigenous riparian weeds: issues of spatial scale and extentJournal of Applied Ecology, 37
P. Osborne, J. Alonso, Dr Bryant (2001)
Modelling landscape-scale habitat use using GIS and remote sensing : a case study with great bustardsJournal of Applied Ecology, 38
J. Pearce, S. Ferrier (2000)
Evaluating the predictive performance of habitat models developed using logistic regressionEcological Modelling, 133
J. Olden (2003)
A Species‐Specific Approach to Modeling Biological Communities and Its Potential for ConservationConservation Biology, 17
Mark Robertson, Neil Caithness, M. Villet (2001)
A PCA‐based modelling technique for predicting environmental suitability for organisms from presence recordsDiversity and Distributions, 7
R. Pearson, T. Dawson, Canran Liu (2004)
Modelling species distributions in Britain: a hierarchical integration of climate and land-cover dataEcography, 27
J. Cramer (2003)
Logit Models from Economics and Other Fields
G. Luck (2002)
The habitat requirements of the rufous treecreeper (Climacteris rufa). 2. Validating predictive habitat modelsBiological Conservation, 105
A. Guisan, J. Theurillat (2000)
Equilibrium modeling of alpine plant distribution: how far can we go?Phytopathology, 30
Predicting species distributions is becoming increasingly important since it is relevant to resource assessment, environmental conservation and biodiversity management ( Fielding and Bell 1997 , Manel et al. 1999 , Austin 2002 , D'heygere et al. 2003 ). Many modeling techniques have been used for this purpose, e.g. generalized linear models (GLM), generalized additive models (GAM), classification and regression trees (CARTs), principal components analysis (PCA), artificial neural networks (ANNs) ( Guisan and Zimmermann 2000 , Moisen and Frescino 2002 , Guisan et al. 2002 , Berg et al. 2004 ). And most of the techniques give the results as the probability of species presence, e.g. GLM, GAM and some algorithms of ANNs, or environmental suitability for the target species, e.g. PCA ( Robertson et al. 2003 ) and some algorithms of ANNs. However, in conservation and environmental management practice, the information presented as species presence/absence may be more practical than presented as probability or suitability. Therefore, a threshold is needed to transform the probability or suitability data to presence/absence data. A threshold is also needed when assessing model performance using the indices derived from the confusion matrix ( Manel et al. 2001 ), which also facilitates the interpretation of modelling results. Before reviewing threshold determination approaches, we will review these model assessment indices first because some of these indices are also the only or primary component of some threshold determination approaches. Model assessment indices Many indices can be used in the assessment of the predictions of species distributions, including sensitivity, specificity, overall prediction success (OPS), Kohen's kappa statistic, the odds ratio, and the normalized mutual information statistic (NMI). And some of them have been incorporated into the approaches to determining thresholds. Fielding and Bell (1997) gave a comprehensive review ( Manel et al. 2001 ). All these indices ( Table 1 ) need the information from the confusion matrix, which consists of four elements: true positive or presence (a), false positive or presence (b), false negative or absence (c) and true negative or absence (d). Since the value of an individual element in the confusion matrix may take zero, the odds ratio and NMI cannot be calculated in some cases. Precision, recall and F are three indices used in the field of information retrieval. Precision is the proportion of the retrieved items that are relevant, i.e. the proportion of predicted presences that are real presences, recall is the proportion of the relevant items that are retrieved, which is equal to sensitivity, and F is the harmonic average of precision and recall ( Nahm and Mooney 2000 ). F varies from 0, when almost no relevant items are retrieved, i.e. almost no real presences are predicted as presences, to 1, when all and only the relevant items are retrieved, i.e. all and only the real presences are predicted as presences. α is a parameter, which gives weights (α and 1−α) to the two components of F. Moreover, when α=0.5, F is strongly towards the lower of the two values (precision and recall); therefore, this measure can only be high when both precision and recall are high. 1 Indices for assessing the predictive performance of species distribution models, a is true positives (or presences), b is false positives (or presences), c is false negatives (or absences), d is true negatives (or absences), n (=a+b+c+d) is the total number of sites and α is a parameter between 0 and 1 (inclusive). Index Formula Sensitivity (or Recall, R) a/(a+c) Specificity d/(b+d) Precision (P) a/(a+b) Overall prediction success (OPS) (a+d)/n Kappa Odds ratio (ad)/(cb) Normalized mutual information statistic (NMI) F Kappa and OPS are two widely used indices ( Guisan et al. 1999 , Manel et al. 1999 , Hilbert and Ostendorf 2001 , Luck 2002 , Moisen and Frescino 2002 ). It should be noted that OPS can be deceptively high when frequencies of zeros and ones in binary data are very different ( Fielding and Bell 1997 , Pearce and Ferrier 2000 , Moisen and Frescino 2002 ). However, Kappa measures the proportion of correctly predicted sites after the probability of chance agreement has been removed ( Moisen and Frescino 2002 ). Threshold determination approaches There are many approaches to determining thresholds, which fall into two categories: subjective and objective. A representative in the first category is taking 0.5 as the threshold, which is widely used in ecology ( Manel et al. 1999, 2001 , Luck 2002 , Stockwell and Peterson 2002 , Bailey et al. 2002 , Woolf et al. 2002 ). Sometimes 0.3 ( Robertson et al. 2001 ) and 0.05 ( Cumming 2000 ) are also used as thresholds. These choices are very arbitrary and lack any ecological basis ( Osborne et al. 2001 ). Sometimes, a specific level, e.g. 95%, of sensitivity or specificity is desired or deemed acceptable, and it is predetermined ( Cantor et al. 1999 ). Thus, the corresponding threshold can be found. This approach is also subjective because a specific level for some attribute (e.g. sensitivity or specificity, etc.) is predetermined by the researchers. There are many objective approaches. With these approaches, thresholds are chosen to maximize the agreement between observed and predicted distributions. Cramer (2003) also realized the problem with fixed threshold approach, especially taking 0.5 as the threshold. He stated that with unbalanced samples, this gives nonsense results. Therefore, the sample frequency, i.e. the prevalence of species occurrence (which is defined as the proportion of species occurrences among all the sites), and the mean value of the predicted probabilities of species presence were recommended as the threshold. Fielding and Haworth (1995) used a threshold that was calculated as the mid‐point between the mean probabilities of occupancy for the present and absent groups. For other objective approaches, usually, either a specific index, e.g. kappa, or the trade‐off between two conflicting properties, e.g. sensitivity and specificity, is optimized in various ways. Kappa maximization approach is popular in ecology ( Huntley et al. 1995 , Lehmann 1998 , Guisan et al. 1998 , Collingham et al. 2000 , Berry et al. 2001 , Pearson et al. 2002 ). Similarly, OPS and F can also be used in the determination of thresholds ( Shapire et al. 1998 ). The sum of sensitivity and specificity can be maximized to give the threshold ( Manel et al. 2001 ), which is equivalent to finding a point on the ROC (receiver operating characteristics) curve (i.e. sensitivity against 1‐specificity) whose tangent slope is equal to 1 ( Cantor et al. 1999 ). The point at which sensitivity and specificity are equal can also be chosen to determine the threshold ( Cantor et al. 1999 ). This approach can also be applied to precision and recall ( Shapire et al. 1998 ). Another approach is to select the point on the ROC curve that is closest to the upper‐left corner (0,1) in the ROC plot since the point in this corner represents a perfect classification with 100% sensitivity and specificity ( Cantor et al. 1999 ). Similarly, the point on the P‐R (i.e. precision‐recall) curve that is closest to the upper‐right corner (1,1) in the P‐R plot can also be used to determine the threshold since the point in this corner represents a perfect classification with 100% precision and recall. Some researchers went further to identify the appropriate threshold by incorporating the relative cost of FP (false positive) and FN (false negative) errors and prevalence ( Zweig and Campbell 1993 , Fielding and Bell 1997 ) or by incorporating the C/B ratio (the ratio of net FP cost and net true positive benefit) and prevalence ( Metz 1978 , Cantor et al. 1999 ). The threshold is corresponding to the point on the ROC curve at which the slope of the tangent is (C/B)×(1−p)/p or (FPC/FNC)×(1−p)/p, where p is the prevalence (of species’ presence) and FPC and FNC are the cost of false positive and false negative respectively. Although there are so many approaches to determining the threshold, there is no comparative study on their behaviours, so we don't know their relative performance. In this paper we compared twelve different approaches to determining thresholds ( Table 2 ), and investigated their behaviours in various situations, i.e. different prevalence for model‐building data and test data, using artificial neural networks, which have been recognized by many researchers, Ozesmi and Ozesmi (1999) , Brosse et al. (1999) , Manel et al. (1999) , Olden and Jackson (2001) , Berry et al. (2002) , Pearson et al. (2002, 2004) and Olden (2003) , as a modeling technique better than other traditional techniques in modeling complex phenomena with non‐linear relationships. We realized that the probability‐based approaches were used for predicted probabilities, and our modeling result is predicted suitability for species presence. However, we believe this will not hinder our effort to use these approaches since the “suitabilities” we get are ranged from 0 to 1. 2 Threshold‐determining approaches studied in this paper. Code Approach Definition Reference Subjective approach 1 Fixed threshold approach Taking a fixed value, usually 0.5, as the threshold Manel et al. (1999) , Bailey et al. (2002) Objective approaches Single index‐based approaches: 2 Kappa maximization approach Kappa statistic is maximized Huntley et al. (1995) , Guisan et al. (1998) 3 OPS maximization approach Overall prediction success (OPS) is maximized Model‐building data‐only‐based approach: 4 Prevalence approach Taking the prevalence of model‐building data as the threshold Cramer (2003) Predicted probability/suitability‐based approaches: 5 Average probability/suitability approach Taking the average predicted probability/suitability of the model‐building data as the threshold Cramer (2003) 6 Mid‐point probability/suitability approach Mid‐point between the average probabilities of or suitabilities for the species’ presence for occupied and unoccupied sites Fielding and Haworth (1995) Sensitivity and specificity‐combined approaches: 7 Sensitivity‐specificity sum maximization approach The sum of sensitivity and specificity is maximized Cantor et al. (1999) , Manel et al. (2001) 8 Sensitivity‐specificity equality approach The absolute value of the difference between sensitivity and specificity is minimized Cantor et al. (1999) 9 ROC plot‐based approach The threshold corresponds to the point on ROC curve (sensitivity against 1‐specificity) which has the shortest distance to the top‐left corner (0,1) in ROC plot Cantor et al. (1999) Precision and recall‐combined approaches: 10 Precision‐recall break‐even point approach The absolute value of the difference between precision and recall is minimized Shapire et al. (1998) 11 P‐R plot‐based approach The threshold corresponds to the point on P‐R (Precision‐Recall) curve which has the shortest distance to the top‐right corner (1,1) in P‐R plot 12 F maximization approach The index F is maximized. In this study, α=0.5 is used in F, i.e. there is no preference to precision and recall Shapire et al. (1998) Materials and methods Species and environmental data Two species, Fagus sylvatica (beech) and Puccinellia maritima (common salt marsh grass), with differing European distributions were used in this study. Fagus sylvatica is widespread in Europe and extends northwards to the edge of the boreal zone and eastwards into Poland and Romania. Puccinellia maritima is a maritime species that is found around the coast of Europe, although it is absent from parts of southern Spain and the Adriatic coast. Their current European distributions were obtained as presence/absence data and mapped to a 0.5° latitude×0.5° longitude grid ( Fig. 1 a, b). Five bioclimatic variables were selected as predictors, which are absolute minimum temperature expected over a 20‐yr period, annual maximum temperature, growing degree days above a base temperature of 5°C, mean soil water availability for the summer half year (May–September), and accumulated annual soil water deficit. These data are also at the scale of 0.5°×0.5°. They were described in detail by Berry et al. (2001) and Pearson et al. (2002) . 1 The observed distributions of F. sylvatica (a) and P. maritima (b). Design of modelling experiment Multilayer feed‐forward ANNs with back‐propagation algorithm were trained with SAS software (release 8.1). The networks contained one input layer, one hidden layer and one output layer. There were 5 neurons in the input layer, which correspond to the 5 input variables, 5 neurons in the hidden layer, and 1 neuron in the output layer. This architecture was chosen after many modelling experiments with varying neurons in the hidden layer. The five environmental variables were standardized to have zero mean and unit standard deviation. In order to investigate the performance of the threshold‐determining approaches in varying situations, we set seven levels of prevalence for both model‐building data (including training data and validation data) and test data, i.e. 5, 10, 25, 50, 75, 90 and 95%. The sample size is 100 for each of the training, validation and test datasets. For each level of the prevalence for model‐building data, one dataset for training was created by randomly sampling specified numbers of presences and absences without replacement from the original presences pool and the original absences pool respectively; then another dataset for validation and two datasets for testing were created sequentially from the left‐over data without replacement. An ANN was trained using the created training dataset and validation dataset, and the resulting model was applied to the two test datasets for each of the seven levels of prevalence. This procedure was repeated five times for each level of prevalence for the model‐building data. There are 10 sets of predictions for each combination of the levels of prevalence for model‐building data and test data, 70 sets of predictions for each level of the prevalence for model building data, and 490 sets of predictions in total for each species. For each set of model‐building data, the threshold was determined by each of the twelve approaches ( Table 2 for details). Then, these thresholds were applied to each testing dataset, and the four assessment indices, including sensitivity, specificity, OPS and Kappa, were calculated. Results Overall assessment The twelve approaches to determining thresholds were assessed using all the 490 sets of predictions from the 490 combinations of training datasets and test datasets with various prevalences for each species ( Fig. 2 ). It can be seen that the trend for the two species is similar. The only exception is that approach 2 is better than approach 1 for F. sylvatica , but it is worse than approach 1 for P . maritima in specificity. The ranking of the twelve approaches given by specificity is different from those given by the other three indices (sensitivity, OPS and kappa), the latter three give similar ranking and OPS and kappa give almost the same ranking for the twelve approaches. It can be seen that approaches 4, 5, 7, 8, and 9 are relatively better than the other approaches (1, 2, 3, 6, 10, 11 and 12) according to the four indices, especially OPS and kappa. In the following two sections, we will take approaches 4 and 2 as representatives for the above two groups to investigate their further behaviours. 2 Overall assessment of the twelve threshold‐determining approaches using four indices: sensitivity (a, e), specificity (b, f), OPS (c, g) and kappa (d, h) for F. sylvatica (a, b, c, d) and P. maritima (e, f, g, h). The bars show the±1 standard errors. Assessment using model‐building datasets with different prevalence The twelve approaches were investigated using the training data with different prevalence, and the trends for the two species are the same. The results of the two approaches (2 and 4) for F. sylvatica are shown in Fig. 3 . The sensitivity and specificity for approach 2 are severely affected by the prevalence of model‐building data, and those for approach 4 are not. The ranking of the approaches changes when the prevalence of model‐building data varies according to sensitivity and specificity. But according to OPS and kappa, the ranking of the approaches remains relatively stable, i.e. approach 4 is almost always better than approach 2. Detailed investigation shows that among the five good approaches (4, 5, 7, 8 and 9), approach 7 is relatively more sensitive to the prevalence of model‐building data and approaches 4 and 5 are more robust when the prevalence of model‐building data changes. It is obvious that there is the least difference among different approaches when the prevalence of model‐building data is 50%. 3 Assessment of two threshold‐determining approaches, i.e. approach 2 (dashed line) and 4 (solid line), which represent bad ones and good ones respectively, using sensitivity (a), specificity (b), OPS (c) and Kappa (d) for F. sylvatica when the prevalence of model‐building data is different. The bars show the±1 standard errors. Assessment using test datasets with different prevalence The twelve approaches were further investigated using the test datasets with different prevalence when the prevalence of model‐building data is fixed, and the results for P. maritima are consistent with those for F. sylvatica . The results of the two representative approaches (2 and 4) for F. sylvatica are shown in Fig. 4 . When the prevalence of test data changes, ranking of different approaches keeps relatively stable according to sensitivity and specificity; but it varies according to OPS and kappa. In addition, approach 4 is less severely affected by the prevalence of test data than approach 2 according to OPS and kappa. In this respect, approach 4 is also better than approach 2. 4 Assessment of the two threshold‐determining approaches, i.e. approach 2 (dashed line) and 4 (solid line), which represent bad ones and good ones respectively, using sensitivity, specificity, OPS and Kappa for F. sylvatica when the prevalence of test data is different and the prevalence of model‐building data is fixed at 10% and 90%. The bars show the±1 standard errors. Discussion Finding a threshold and making the presence/absence prediction is the final step in species distribution modeling, and it is necessary in, for example, the estimation of species range and the assessment of the impact of climate change. It is important to give an accurate presence/absence prediction in these situations. However, in other situations, other considerations should be included. For example, in species reintroduction programs, we may limit the reintroduction sites to the most suitable areas; but in some conservation planning programs, we may take a less restrictive strategy, that is, purposely including some less suitable areas in protection. It is expected that the larger the predicted probability/suitability of presence at a site, the more suitable is the site to the reintroduction of the species. However, since the prevalence of model‐building data has significant effect on the predicted probability/suitability of presence, i.e. the higher the prevalence, the bigger the predicted probability/suitability ( Cramer 2003 , Liu et al. unpubl.), this makes it difficult to decide the more suitable or less suitable sites. Therefore, even in those applications with some subjective decision‐makings involved, it is still necessary to find the appropriate threshold and take the “objective” presence/absence prediction as a reference. In this study, we treated two kinds of errors, e.g. false positive and false negative, as equally important and gave no preference to either side. But approach 12 can be adapted to the situation in which one of two conflicting sides is emphasized by changing the parameter α from 0 to 1. A smaller α emphasizes recall, and bigger α emphasizes precision. However, it is difficult to say to what degree one side is emphasized. In this situation, the subjective approach may be suitable, e.g. a “minimum acceptable error” could be defined that depended on the intended application of the model. For example, compared with false negatives, we could tolerate more false positives when we set up a conservation area for a particularly endangered species. If the purpose of the model was to identify experimental sites, where we could find a species, we should minimize the false positive error rate ( Fielding and Bell 1997 ). When some kind of cost for the false positive or false negative and/or benefit for true positive or true negative needs to be taken into account, Metz's (1978) approach can be adopted because the cost of false positive and the benefit of true positive as well as prevalence were explicitly considered. However, the relevant cost and benefit are difficult to determine in environmental and ecological practice. Zweig and Campbell (1993) suggested that if FPC>FNC, the threshold should favor specificity, while sensitivity should be favored if FNC>FPC ( Fielding and Bell 1997 ). Because estimation of the cost and benefit may add more uncertainty to the problem, caution must be taken when this approach is adopted. It is interesting to note that among the twelve approaches we studied, both sensitivity and specificity for approaches 4, 5, 7, 8 and 9 are high (>0.8) and are higher than those for the other approaches. Since the bigger the sensitivity, the smaller the false negatives rate, and the bigger the specificity, the smaller the false positives rate, therefore, both false positives rate and false negatives rate are low (<0.2) for the approaches 4, 5, 7, 8 and 9. These approaches are recommended to use. The other approaches either have low false negatives rate and high false positives rate (e.g. approach 12), or have high false negatives rate and low false positives rate (e.g. approach 10), or have both high false positives rate and high false negatives rate (e.g. approaches 1 and 3), therefore, these approaches are not recommended. It is not unexpected that the fixed threshold approach (threshold=0.5) is one of the worst. Guisan and Theurillat (2000) found that the threshold histogram is not centered on 0.5 with symmetric tails in each opposite direction (toward 0 and 1), rather all values range between 0.05 and 0.65 with a mean at 0.35 and an asymmetric shape. In fact, the prevalence of model‐building data affects all the results. The output is biased towards the larger of the two groups ( Fielding and Bell 1997 , Cramer 2003 ), occupied sites and unoccupied sites. When the prevalence is small, a 0.5 threshold would classify most of the sites as unoccupied ( Cumming 2000 ). However, the prevalence approach is one of the most robust, i.e. although it is not the best in every situation, it is good, at least not bad, even in the worst situation. Actually it is one of the best as assessed using sensitivity, OPS and kappa. This is also not unexpected. In fact, in another study we found that the probabilities corresponding to the maximum OPS and maximum kappa for the test data are positively correlated to the prevalence of model‐building data (Liu et al. unpubl.). We suggested that a good presence/absence prediction would be obtained by taking the prevalence of model‐building data as the threshold. This hypothesis was verified by this study. From this study we also found that when the prevalence of model‐building data is 50%, there is little difference among the twelve approaches as measured by the four indices ( Fig. 3 ), and the relative difference (=difference of the maximum and minimum among the twelve values for each index divided by the maximum) is <5% for all the four indices and the two species. Furthermore, in addition to approaches 1 and 4, approaches 2, 3 and 7 (and also 11 for P . maritima ), 8 and 10, and 5 and 6 reach exactly the same result respectively for each of the two species. The convergence of approaches 1 and 4 is obvious. The convergence of approaches 5 and 6 can be easily deduced because there are equal number of occupied sites and unoccupied sites. The convergence of approaches 2, 3 and 7 means that kappa, OPS and the sum of sensitivity and specificity were maximized at the same time. The convergence of approaches 8 and 10 means that specificity=sensitivity (=recall)=precision at the same time. This means that many conditions are satisfied or nearly satisfied at the same time in this situation. Therefore, the best result will most probably be obtained by any approach, even the poor ones, which is verified by this study ( Fig. 3 c, d). This is encouraging since it supports our recommendation that it is better to use model‐building data with prevalence of 50% in species distribution modeling (Liu et al. unpubl.). Conclusion The prevalence approach and average probability/suitability approach are simple and effective, and they are at least as good as the more complicated approaches, i.e. sensitivity‐specificity sum maximization approach, sensitivity‐specificity equality approach and the ROC plot‐based approach. These five approaches fall into the group of good ones. Unfortunately, one of the widely used approaches, i.e. fixed threshold approach, is the worst one, which is not therefore recommended. Another popular approach, i.e. kappa maximization approach, is also not a good one. We also recommend that if possible, using datasets with prevalence of 50% to build models since in addition to other advantages, it is easier to find the optimal threshold. Acknowledgements This work is funded by the Postdoctoral Fellowship from UK Royal Society to C. Liu. The distributions and original modelling work for these species were done under RegIS and MONARCH projects awarded to the ECI. RegIS was a jointly funded project between the UK's MAFF, DETR and UKWIR and MONARCH was funded by a consortium of government and non‐government nature conservation organizations, led by English Nature.
Ecography – Wiley
Published: Jun 1, 2005
You can share this free article with as many people as you like with the url below! We hope you enjoy this feature!
Read and print from thousands of top scholarly journals.
Already have an account? Log in
Bookmark this article. You can see your Bookmarks on your DeepDyve Library.
To save an article, log in first, or sign up for a DeepDyve account if you don’t already have one.
Copy and paste the desired citation format or use the link below to download a file formatted for EndNote
Access the full text.
Sign up today, get DeepDyve free for 14 days.
All DeepDyve websites use cookies to improve your online experience. They were placed on your computer when you launched this website. You can change your cookie settings through your browser.