Performance of Species Richness Estimators Across Assemblage Types and Survey Parameters Reese 2014

Global Ecology and Biogeography, (Global Ecol. Biogeogr.
) (2014) 23, 585–594

bs_bs_banner
RESEARCH Performance of species richness

PA P E R
estimators across assemblage types
and survey parameters
Gordon C. Reese1*, Kenneth R. Wilson1 and Curtis H. Flather2
1
Department of Fish, Wildlife, and ABSTRACT
Conservation Biology, Colorado State
University, Fort Collins, CO 80523, USA,
Aim A raw count of the species encountered across surveys usually underestimates
2
US Forest Service, Rocky Mountain Research species richness. Statistical estimators are often less biased. Nonparametric estima-
Station, Fort Collins, CO 80526, USA tors of species richness are widely considered the least biased, but no particular
estimator has consistently performed best. This is partly a function of estimators
responding differently to assemblage-level factors and survey design parameters.
Our objective was to evaluate the performance of raw counts and nonparametric
estimators of species richness across various assemblages and with different survey
designs.
Location We used both simulated and published field data.
Methods We evaluated the bias, precision and accuracy of raw counts and 13
nonparametric estimators using simulations that systematically varied assemblage
characteristics (number of species, species abundance distribution, total number of
individuals, spatial configuration of individuals and species detection probability),
sampling effort and survey design. Results informed the development of an esti-
mator selection framework that we evaluated with field data.
Results When averaged across assemblages, most nonparametric estimators were
less negatively biased than a raw count. Estimators based on the similarity of
repeated subsets of surveys were most accurate and their accumulation curves
appeared to reach asymptotes fastest. Number of species, species abundance distri-
bution and effort had the largest effects on performance, ultimately by affecting the
proportion of the species pool contained in a sample. Our estimator selection
framework showed promising results when applied to field data.
Main conclusions A raw count of the number of species in an area is far from the
best estimate of true species richness. Nonparametric estimators are less biased.
Newer largely unused, estimators perform better than more well known and longer
established counterparts under certain conditions. Given that there is generally a
trade-off between bias and precision, we believe that estimator variance, which is
often not reported when presenting species richness estimates, should always be
*Correspondence: Gordon C. Reese, included.
Department of Fish, Wildlife, and Conservation
Keywords
Biology, Colorado State University, Fort Collins,
CO 80523, USA. Biodiversity, community ecology, nonparametric estimator, sample coverage,
E-mail: greese126@gmail.com selection framework, simulation.
Attempts to quantify species diversity often include: (1) a count

INTRODUCTION
or estimate of the unique species in a delineated area (species
All aspects of biological diversity, from genes to ecosystems, can richness), (2) a measure of the uniformity of abundances among
inform decisions in ecological monitoring, conservation man- species (species evenness), or (3) some measure of species com-
agement and reserve design. How best to quantify diversity at a position (species similarity; see Magurran, 2004). Of these,
species level remains a topic of much debate (Brose et al., 2003). species richness (SR) is the most conceptually simple and
© 2014 John Wiley & Sons Ltd DOI: 10.1111/geb.12144

http://wileyonlinelibrary.com/journal/geb 585
G. C. Reese et al.
frequently used (Moreno et al., 2006; Drake, 2007; Lopez et al., Table 1 Levels of factors simulated to evaluate species richness
2012). estimators.
The number of species observed (Sobs) is the most straight-
forward estimate of the true number of species in an assemblage Description Symbol Levels
(Strue), but it has a known negative bias and is based on the naïve
Total (true) number of species Strue 25
assumption that all species are detected with a probability equal
100
to one (Palmer, 1990; Nichols et al., 1998). Fortunately, species 500
distribution and abundance patterns can be used to inform Total abundance across all species N 6250
statistically derived estimators of Strue. All estimators have under- 12500
lying assumptions, but the statistical estimators of SR are typi- Species abundance distribution Abund Log-series
cally less negatively biased than Sobs (Baltanás, 1992; Chiarucci Log-normal
et al., 2003; Walther & Moore, 2005). Spatial relationship between individuals Config Aggregated
Besides Sobs, there are three categories of SR estimators. One of a species Hyper-dispersed
category includes approaches for extrapolating a species accu- Random*
mulation curve to an asymptote, often using a negative expo- Mean detection probability of three p (0.5, 0.5, 0.5)
species abundance groups (0.9, 0.9, 0.9)
nential model (Holdridge et al., 1971), the Michaelis–Menten
(0.5, 0.7, 0.9)
equation (Michaelis & Menten, 1913) or a power model
(0.9, 0.7, 0.5)
(Arrhenius, 1921; Tjørve, 2009). A second category includes Spatial arrangement of surveyed grid Design Random
parametric methods that involve: (1) interpolating under a dis- cells Transect (linear)†
tribution fit to abundance data or (2) applying an estimator that Amount of landscape surveyed Effort 1% (100 cells)
assumes that all species are equally detectable. A third category 5% (500 cells)
includes nonparametric estimators, which make no assumption
about the underlying distribution of the data. *Individuals were spaced less regularly (aggregated) or more evenly
The performance of SR estimators has largely depended on (hyperdispersed) than expected by chance (random).
†Surveys were configured randomly or as random linear transects of 50
whether underlying assumptions are met, making factors such
grid cells.
as the species abundance distribution, species detection prob-
ability (p), survey effort and Strue (Baltanás, 1992; Keating &
Quinn, 1998; Brose et al., 2003) important to understanding
their performance. Ultimately, no estimator has consistently species but are constant across space. This assumption can be
performed best, but the nonparametric estimators have gener- violated when distributions are spatially heterogeneous, which
ally performed better than the other categories (see Table 1 in occurs regularly in natural systems (Legendre, 1993; Deblauwe
Cao et al. 2004; Table 3 in Walther & Moore 2005; Table 1 in et al., 2008).
Reese 2012). We thus focus on evaluating nonparametric esti- Our primary objective was to evaluate nonparametric SR esti-
mators, including several that are relatively untested (see mators across assemblages that were systematically varied both
Burnham & Overton, 1978; Pledger, 2000; Cao et al., 2001; Cao in their attributes (number of species, total number of individ-
et al., 2004), by comprehensively and consistently varying uals, species abundance distribution, spatial aggregation and
factors that can compromise estimator assumptions. species detection probability) and how they were sampled
Most of the nonparametric SR estimators can be categorized (effort and survey design). Given the large number of factors
as those that model either heterogeneity in p (commonly and the benefits of knowing the true factor values, we relied
denoted as model Mh; see Otis et al., 1978) or the similarity heavily on simulated data, but also evaluated estimators using
between replicate subsets of the survey data. Many in the first field data (see the Supporting Information). An important sec-
group were developed for population estimation using mark– ondary objective was to expand the estimator selection
recapture data under an assumption of geographic and demo- approach proposed by Brose et al. (2003). Their framework tar-
graphic closure, an assumption that also applies to all SR geted estimator accuracy and based selection on the ratio of
estimators. Brose et al. (2003) detailed additional challenges Sobs to the mean SR estimate, which is an estimate of sample
that arise when an Mh population estimator is used to estimate coverage (sc) or the fraction of a species pool represented in a
Strue. First, differences in detectability between species can be sample (i.e. Sobs/Strue). Our contribution included additional
larger and more difficult to model than those between the selection criteria based on bias, precision and an estimate of sc
individuals composing a population of one species. Second, based on Jaccard’s similarity coefficient.
when an Mh estimator is used to estimate population size from
encounter histories of individuals in surveys repeated at the
METHODS
same location, an assumption is that p will vary across indi-
viduals but remain constant over time. When those estimators
Simulation procedure
are instead used to estimate Strue from encounter histories of
species in surveys replicated at different locations, the compa- We evaluated the performances of Sobs and 13 nonparametric SR
rable assumption is that detection probabilities vary among estimators across simulated assemblages using the program
586 Global Ecology and Biogeography, 23, 585–594, © 2014 John Wiley & Sons Ltd
Performance of species richness estimators
SimAssem (Reese et al., 2013). We systematically varied the total 0.95d had to equal or exceed a RUV, where d is a randomly
number of species, total assemblage abundance, type of species selected distance [0, 1]. Second, the individual’s location had to
abundance distribution, spatial structure of individual occur- fall on or within landscape boundaries when placed distance d in
rences within a species and detection probability of species a random direction (0–359°) from the selected centre.
abundance groups in a factorial design (Table 1). Similarly, we Species-specific detection probabilities (factor p) were ran-
simulated a sampling procedure in which we varied the number domly drawn from beta distributions with expected means of
of surveyed grid cells and how they were selected. Based on our 0.5, 0.7 and 0.9. To isolate the effect of p, we constrained vari-
review of the literature and past experience, we selected these ances to the range 0.010–0.015. For two factor levels, we drew
factors and factor levels to cover a wide range of realistic sce- each p from the same distribution, with an expected mean of 0.5
narios. We generated 42 replicates of each factor combination or 0.9 (α and β parameters equalled 10 and 10, or 4.5 and 0.5,
(24,192 total simulations) as a compromise between computer respectively). We evaluated two additional factor levels where p
processing time and having a sufficient number of replicates to varied as a function of abundance (see Selmi & Boulinier, 2004;
ensure an adequate sample size after we removed replicates Pagano & Arnold, 2009). We grouped species into thirds based
involving estimation issues (e.g. lack of convergence for the on abundance and randomly selected abundance groups to
Mixture estimator). accommodate the additional species since the tested levels of
We simulated assemblages with 25, 100 and 500 species Strue are not divisible by three. For one factor level we assigned
(factor Strue), populating them with a total of 6250 and 12,500 each species in the least, moderate and most abundant groups a
individuals (factor N). These values restricted the average p that was randomly drawn from a distribution with an expected
number of individuals encountered per species to the range of mean of 0.5, 0.7 (α = 14, β = 6) or 0.9, respectively. We created
several datasets (see Williams, 1939; Lewis & Taylor, 1967; another factor level by reversing the expected means of the
Dallmeier et al., 1991). To simulate species abundance patterns abundance groups.
(factor Abund), we used log-normal (Preston, 1948) and log- In SimAssem, every cell of an overlaid 100 × 100 grid is a
series distributions (Fisher et al., 1943). SimAssem generated: potential survey site. We evaluated two levels of effort (factor
(1) log-normal distributions by drawing a random log-normal Effort) by sampling either 100 (1%) or 500 (5%) grid cells. In
variate (μ = 0, σ = 1) for each species, dividing each variate by addition, we used two survey designs (factor Design), random
the sum of all the variates and rescaling by multiplying each and a linear transect design. Each transect contained 50 adjacent
variate by N and (2) log-series distributions by calculating the cells in a randomly selected horizontal or vertical orientation
number of species to populate with z individuals, where z = 1, 2, and transects were added until the specified number of grid cells
. . . , N (see Magurran, 2004). For comparison, we also simulated was surveyed. Previously surveyed grid cells intersected by a new
evenly distributed assemblages with particulate-niche distribu- transect were applied to the transect length, but were not
tions (MacArthur, 1957) where each individual was randomly double-counted. An individual was encountered when two con-
assigned to a species. Given that: (1) MacArthur considered this ditions were met. First, a surveyed cell had to contain at least one
distribution unsatisfactory (Magurran, 2004) and (2) it is infre- individual. Second, a RUV, where one was generated for each
quently referenced in the literature, we provide the results in individual in the cell, had to be ≤ p.
Table S1 in Supporting Information (see also Reese, 2012).
We assigned x- and y-coordinates [0, 1] to each individual via
Species richness estimators
three spatial configuration options available in SimAssem
(factor Config). The random configuration located each individ- In addition to Sobs, we evaluated the performance of 13 estima-
ual with a pair of random uniform variates [0, 1]. For the hyper- tors, where some are variants of others (see Table 2 for details
dispersed configuration, SimAssem assigned each individual a and abbreviations). Those estimators belonging to the Mh class
square territory with a linear dimension of 1/√ni, where ni is the included two based on abundance patterns (i.e. the number of
abundance of species i. Territories were adjacent in the horizon- species with an exact number of individuals encountered) and
tal and vertical directions and collectively formed a grid across nine based on incidence patterns (i.e. the number of species
the entire landscape. Individuals were located a random distance encountered in an exact number of surveys). We used the Rmark
and random direction from the bottom left corner of their ter- package (Version 1.9.5; Laake & Rexstad, 2008) in the program
ritory. When a randomized location in an overlapping territory R (R Development Core Team, 2009) to generate Mixture esti-
fell outside the top or right landscape boundary, as happened mates based on two groups using program MARK (Version 6.0;
occasionally, the individual was randomly placed on the land- White & Burnham, 1999).
scape. To determine the locations around which species could Two additional estimators, CY-1 and CY-2, are based on the
aggregate with the aggregated (centres equal abun) option, similarity of two replicate subsets of surveys. A CY-1 estimate
SimAssem generated ni random uniform variates (RUV) for equals average SR across the replicate sets of surveys (SR )
each species i, and every RUV ≥ 0.98 increased by one the divided by the Jaccard coefficient (JC), JC = c/(a + b + c), where
number of randomly placed aggregate centres, resulting in a and b are the numbers of species unique to each subset and c
approximately one centre for every 50 individuals. Each individ- is the number of species common to both subsets. Thus, CY-1 is
ual of a species was then randomly allocated to a centre and only calculable when individuals are encountered in two or
located on the landscape when two conditions were met. First, more surveys. A CY-2 estimate equals the slope plus the inter-
Global Ecology and Biogeography, 23, 585–594, © 2014 John Wiley & Sons Ltd 587
G. C. Reese et al.
Table 2 Evaluated species richness

Estimator name Abbreviation Reference estimators.
Abundance-based coverage*† ACE Chao & Lee (1992)

Chao1 (bias-corrected)*† Chao1 Chao (1984)
Bootstrap*‡ Boot Smith & van Belle (1984)
Chao2 (bias-corrected)*‡ Chao2 Chao (1987)
Incidence-based coverage*‡ ICE Lee & Chao (1994)
1st-order jackknife*‡ Jack1 Burnham & Overton (1978)
2nd-order jackknife*‡ Jack2 Burnham & Overton (1978)
3rd-order jackknife*‡ Jack3 Burnham & Overton (1978)
4th-order jackknife*‡ Jack4 Burnham & Overton (1978)
5th-order jackknife*‡ Jack5 Burnham & Overton (1978)
Mixture-model*‡§ Mixture Pledger (2000)
CY-1¶ CY-1 Cao et al. (2001)
CY-2¶ CY-2 Cao et al. (2004)
Number of species observed Sobs
*Estimation by modelling heterogeneity in detection probability of species.

†Estimation using sample abundance patterns (i.e. number of individuals).
‡Estimation using sample incidence patterns (i.e. number of surveys).
§Estimation using maximum likelihood by modelling heterogeneity using mixtures.
¶Estimation using similarity of replicate surveys of species with the Jaccard coefficient.
cept of a regression line fitted to SR versus JC, where each value 2

⎛ Sêst( j ) Sêst ⎞
was the average of 100 iterations (see Cao et al., 2004).
∑
X
⎜ − ⎟
SimAssem used five regression points (SR − JC pairs) when j =1 S
⎝ true Strue ⎠
there were between 10 and 19 surveys with encounters and 10 SD =
X −1
regression points when there were 20 or more surveys with
encounters. where Ŝest Strue is the scaled estimate of the jth replicate, j = 1, 2,
Across estimators, the range of possible estimates is large. The . . . , X. We scaled the estimates in order to compare perfor-
bootstrap estimator can extrapolate to Sobs × 2 (Colwell & mances across the three levels of Strue. For accuracy, we used
Coddington, 1994), the jackknife estimators nearly to Sobs (the scaled mean square error (SMSE; Walther & Moore, 2005):
2
order of the estimator + 1), Chao2 to Sobs2/2 and CY-2 to SR 2
(Cao et al., 2004). Higher-order jackknife estimates can be < Sobs 1 X
⎛ Sêst( j ) − Strue ⎞
(Lam & Kleinn, 2008) or even negative. We therefore report the
SMSE =
X ∑ ⎜
j =1 ⎝ Strue ⎟⎠ .
proportion of estimates that were < Sobs.
Values of SD and SMSE are always positive and, as with SME,
Performance evaluation those closer to zero indicate better performance.
We treated all factors as random and evaluated relative factor
To evaluate estimator performance, we computed bias, preci- effects with random-effects models, using the ‘proc mixed’ pro-
sion, and accuracy (Walther & Moore, 2005). We assessed overall cedure in sas (version 9.3; SAS Institute, 1999), by computing
performance by combining all simulations (i.e. all factors and all variance components and proportionally allocating total vari-
factor levels) and performance for a specific factor level by com- ance into Strue, N, Abund, Config, p, Effort and Design. By using
bining the replicates of all remaining factors and factor levels. the averages (SME and SMSE) and standard deviations (SD) of
Estimator bias was evaluated with scaled mean error (SME; the replicates for each unique factor combination, the variance
Walther & Moore, 2005) components only reflect between-factor effects. We considered
only main effects, so the residual represents the variance
1 X
⎛ Sêst( j ) − Strue ⎞ explained by all interactions plus residual error.
SME =
X ∑ ⎜⎝
j =1 Strue ⎟⎠ We also evaluated performance against sc, which is possibly
the most important factor driving estimator performance
where X is the number of replicates across combined factors, j is (Baltanás, 1992; Brose et al., 2003). We used sc to group
the replicate, j = 1, 2, . . . , X, and Ŝest is an estimate of Strue. simulations into 10 equally sized bins (i.e. 0.0 < sc ≤ 0.1,
Negative and positive values of SME indicate average underes- 0.1 < sc ≤ 0.2, . . . , 0.9 < sc ≤ 1.0). Estimator performance was
timation and overestimation, respectively, and SME = 0 indi- averaged over each coverage range, which thereby assesses per-
cates an unbiased estimator. We evaluated estimator precision formance at various levels of sampling completeness. Except
with the sample standard deviation of a group of scaled when data are simulated, sc is usually not known. Brose et al.
estimates (SD): (2003) estimated sc by dividing Sobs by the average SR estimate.
Table 3 Average performance of species richness estimators Except for the bias of Mixture and the accuracy of higher-
across all factors (see Table 1) based on bias, measured as scaled order jackknife estimators (Jack3–Jack5), bias and accuracy
mean error (SME); precision, measured as standard deviation of worsened with increasing Strue (Table S1.1 in Supporting Infor-
scaled estimates (SD); and accuracy, measured as scaled mean mation). The relationship between Strue and estimator precision
square error (SMSE). Subscripts are estimator rank for a given
was much less consistent, with numerous instances of both posi-
performance measure.
tive and negative relationships. Only three estimators, Sobs, Boot
and Jack1, ranked consistently across all factor levels and only
Estimator Bias (SME) Precision (SD) Accuracy (SMSE)
for precision. The greatest change in the relative performance of
ACE −0.327 0.264 0.174 an estimator was the bias of Mixture which was the least biased
Boot −0.4912 0.276 0.3111 estimator with Strue = 100 and one of the two most biased esti-
Chao1 −0.3710 0.262 0.208 mators with the other factor levels.
Chao2 −0.379 0.251 0.207 Variation in the other assemblage factors also affected estima-
CY-1 −0.193 0.309 0.131 tor performance. Except for Jack5 and Mixture, estimator bias
CY-2 −0.151 0.3411 0.142 and accuracy improved with increasing evenness (i.e. in moving
ICE −0.326 0.265 0.173 from log-series to log-normal distributions) and all estimators
Jack1 −0.4111 0.287 0.2510
were more precise with log-series distributions (Table S1.2). The
Jack2 −0.338 0.298 0.196
relative bias of Mixture was inconsistent, ranking best with log-
Jack3 −0.275 0.3310 0.185
Jack4 −0.224 0.4112 0.229
series distributions and worst with log-normal distributions. By
Jack5 −0.182 0.5913 0.3813 all measures, the nonparametric estimators performed better at
Mixture 0.6414 13.4114 180.2214 the larger N level (Table S1.3). The precision of Sobs, however,
Sobs −0.5613 0.263 0.3812 decreased slightly with the increase in N. Mixture was the most
biased estimator at N = 6250 and the least biased at N = 12,500.
Performances varied little across configuration patterns, but esti-
We estimated sc (sc) by averaging the Jaccard similarity coeffi- mators were generally least biased and least precise when indi-
cients across the 100 iterations used to calculate CY-1. viduals were hyperdispersed (Table S1.4). The nonparametric
estimators performed better when average species detection
probability ( p ) equalled 0.9 than when p = 0.5 (Table S1.5).
R E S U LT S
When p increased with ranked abundance, estimators were more
precise and generally more biased and more accurate than when
Simulations
p decreased with abundance. Mixture was again either the most
The Mixture estimator failed to converge (i.e. it failed to produce or least biased estimator, depending on the factor level.
an estimate) in 0.19% of the simulations and in as many as five The effects of the tested survey factors, Effort and Design, were
replicates (i.e. simulations with identical factor levels). A small relatively consistent. The nonparametric estimators performed
number of simulations (0.39%) resulted in less than the number better at the larger Effort level (Table S1.6). Only the precision of
of encounters needed to compute CY-1 and CY-2, so those esti- Sobs decreased with the increase in Effort. Estimators were gen-
mates equalled zero. There were also simulations where one or erally, often to a small degree, less biased, more precise and more
more of the estimators returned an estimate < Sobs including accurate when individual survey locations were selected ran-
CY-2 (0.45%), Jack2 (3.60%), Jack3 (7.73%), Jack4 (13.45%) domly than when they were placed along randomly located
and Jack5 (18.10%). We removed simulations in which Mixture linear transects (Table S1.7).
did not converge and rebalanced the factorial by removing five Averaged across all estimators, Strue, Effort and Abund had the
or more replicates from every factor combination (i.e. resulting largest effects on estimator performance (Table S2). The results
in 37 replicates of every factor combination); when possible, we were generally similar with individual estimators; however, N or
removed replicates that also had one or more estimates < Sobs. p occasionally ranked higher. Main effects accounted for
Averaged over all simulations, Mixture was the only positively approximately 87%, 71% and 81% of the variation in bias, pre-
biased estimator and the only nonparametric estimator more cision and accuracy, respectively, but explained < 50% of the
biased than Sobs (Table 3). The least biased estimator, CY-2, variation in the performance of Mixture.
underestimated Strue by 15% on average. CY-1 was the most We also evaluated estimator performance against sample cov-
accurate estimator. Estimator rank depended on the evaluation erage (i.e. sc). Estimators were generally less biased and more
metric and we noted a general trade-off between bias and pre- accurate with a larger sc. There was no apparent trend in preci-
cision (i.e. the least biased estimators were the least precise, and sion. Our results suggest that the best estimator for bias reduc-
vice versa). However, Mixture was both the most biased and the tion depends largely on the sc range; only Jack5 is recommended
least precise estimator (Table 3). Relative ranks were rarely con- first in more than one sc range (Table 4). By contrast, Sobs and
sistent across factor levels, but estimators that performed best Boot were among the three most precise estimators for all cov-
overall were generally among the best at individual factor levels. erage ranges. CY-1 and Boot were the most accurate estimators
Below, we mainly focus on factor-level comparisons; additional in the two smallest and two largest coverage ranges, respectively,
estimator-specific results are presented in Reese (2012). and one of the jackknife estimators was most accurate in the
G. C. Reese et al.
Table 4 The three best performing

Bias (SME) Precision (SD) Accuracy (SMSE) species richness estimators, averaged
over all simulations (without the
Coverage range 1st 2nd 3rd 1st 2nd 3rd 1st 2nd 3rd particulate-niche distribution), in the
specified sample coverage (sc) range (the
0.0 < sc ≤ 0.1 (1891) −0.30 −0.32 −0.56 0.02 0.02 0.03 0.21 0.30 0.39
number of simulations in each range is
CY-2 CY-1 ICE Sobs Boot Jack1 CY-1 CY-2 ICE
given in parentheses). See Table 2 for
0.1 < sc ≤ 0.2 (2264) −0.25 −0.29 −0.51 0.03 0.04 0.05 0.24 0.28 0.29
estimator abbreviations and Table 3 for
CY-1 CY-2 Jack5 Sobs Boot Jack1 CY-1 CY-2 Jack5
specific performance metrics.
0.2 < sc ≤ 0.3 (3209) −0.11 −0.30 −0.30 0.03 0.04 0.05 0.17 0.18 0.19
Mixture CY-1 CY-2 Sobs Boot Jack1 Jack4 CY-2 Jack5
0.3 < sc ≤ 0.4 (3451) −0.22 −0.27 −0.27 0.03 0.04 0.07 0.14 0.15 0.15
Jack5 Jack4 CY-2 Sobs Boot Jack1 Jack3 Jack4 CY-2
0.4 < sc ≤ 0.5 (2719) −0.07 −0.13 −0.17 0.03 0.04 0.07 0.07 0.08 0.09
Jack5 Jack4 CY-2 Sobs Boot Jack1 Jack3 Jack2 CY-2
0.5 < sc ≤ 0.6 (2563) −0.04 −0.09 −0.09 0.03 0.04 0.08 0.06 0.06 0.08
Jack5 Jack4 CY-2 Sobs Boot Jack1 Jack2 CY-2 CY-1
0.6 < sc ≤ 0.7 (1313) 0.03 −0.03 −0.04 0.02 0.04 0.08 0.02 0.03 0.04
Jack3 CY-1 Jack2 Sobs Boot Jack1 Jack1 Jack2 CY-1
0.7 < sc ≤ 0.8 (1304) −0.03 0.05 −0.06 0.03 0.04 0.08 0.01 0.02 0.02
Jack1 Jack2 ICE Sobs Boot Jack1 Jack1 Boot ICE
0.8 < sc ≤ 0.9 (1065) −0.02 −0.03 0.03 0.02 0.04 0.08 0.00 0.01 0.01
ICE ACE Jack1 Sobs Boot Jack1 Boot Jack1 ACE
0.9 < sc ≤ 1.0 (1534) 0.00 0.00 0.00 0.03 0.03 0.04 0.00 0.00 0.00
Chao2 Chao1 ACE Sobs Mixture Boot Boot Mixture Sobs
SME, scaled mean error; SD, standard deviation of scaled estimates; SMSE, scaled mean square error.
other ranges. The CY-2 estimator was one of the three most were less than the ≥ 20% bias reported for other estimators (see
accurate estimators in the largest number of sc ranges (six) and Brose et al., 2003; Canning-Clode et al., 2008; Jobe, 2008). Bias
CY-1 was next best (four). reduction involves increased extrapolation, which could partly
explain the loss of precision that accompanied the least biased
estimators. As was found in our study, estimation involves a
DISCUSSION
tradeoff between bias and precision (see Burnham & Overton,
1979; Brose et al., 2003; Willie et al., 2012).
The case for statistical estimators
Among the most accurate and least biased estimators at many
Using Sobs as an estimate of SR frequently leads to large under- factor levels, CY-1 and CY-2 could be promising newer estima-
estimates, due to a strong dependence on sampling effort and tors (Table 3, Appendix S1). It is noteworthy that both per-
the assumption that all species are detected (Nichols et al., 1998; formed at their best with relatively uneven log-normal and log-
Kéry & Plattner, 2007). Other than Mixture, the nonparametric series distributions (Table S1) which might best approximate
estimators were generally less biased and more accurate than Sobs true species abundance patterns (Sugihara, 1980; Ulrich et al.,
(Table 3), supporting other studies (see Wagner & Wildi, 2002; 2010). However, CY-1 and CY-2 were outperformed at some
Brose et al., 2003; Walther & Moore, 2005). Thus, Sobs is far from factor levels (see Appendix S1). Cao et al. (2004) stated that
the best approach for estimating SR. As effort increases, Sobs will sample size would affect CY-2 less than other estimators and it
approach Strue, but estimators generally provide a more reliable was the least biased estimator at the smaller effort level (Table
approach (see Table S1.6). A measure of precision is another S1.6). However, CY-1 was more accurate and less affected (bias)
advantage provided by some estimators (Nichols et al., 1998; by variation in Effort than any other estimator (Table S2; see also
Reese, 2012). Drake, 2007). It is possible that the number of regression points
used in the program SimAssem, five or ten, does not fully expose
the performance potential of CY-2. Both estimators were rela-
Statistical estimator comparisons
tively imprecise and prone to overestimation with particulate-
Averaged across all factor levels, our results suggest that ACE, niche distributions (Table S1) and larger levels of N, Effort and p
CY-1, CY-2, ICE and Jack3 can provide less biased and more (Appendix S1), factor levels generally associated with larger sc
accurate estimates than the more popular Chao1, Chao2, Jack1 values.
and Jack2 estimators (Table 3). The higher-order jackknife and Another largely untested estimator, Mixture, often performed
bias-corrected Chao estimators were among the least and inter- relatively poorly. However, based on bias, Mixture ranked more
mediately biased estimators, respectively, supporting Brose et al. favourably in assemblages with an intermediate number of
(2003). In particular, the overall biases of Jack5, CY-1 and CY-2 species (Table S1.1), log-series distributions (Table S1.2), larger
N (Table S1.3) and either a larger p or a p that increased with individuals (Table S1.3). Positive relationships have been simi-
abundance groups (Table S1.5). Precision improved consider- larly reported with N in the form of density (Baltanás, 1992;
ably when effort increased from 100 to 500 cells (Table S1.6). Walther & Morand, 1998).
These results suggest that some factor combinations did not When the two levels where p was a function of ranked abun-
provide enough data to estimate probabilities for the two groups dance were removed, the correlation between sc and p increased
of the mixture model. An inspection of the raw results (G.C.R., from r2 = 0.04 to r2 = 0.18. When p generally decreased with
unpublished data) showed that the poorest performances abundance, estimators were less biased (there were exceptions
resulted from an increased number of large estimates. In with Boot, Mixture and Sobs), though often with less precision
other words, Mixture occasionally produced large estimates and accuracy, than when p increased with abundance (see Table
(e.g. 211,111) in assemblages with, for example, Strue = 500, S1.5). Less abundant species, particularly when p is small, are
Abund = log-normal, N = 6250, and p = 0.5, possibly a result of often not detected, but a larger p could result in such species
poor convergence. The conditions under which the performance being encountered, thereby reducing bias. Given the formulae of
of Mixture improved would indicate that it should be further many of the nonparametric estimators, bias will be further
evaluated in comparatively data-rich environments. reduced when such species are encountered in a small number
of surveys. Using either a fixed p for each abundance group or
larger differences in p might show more consistent trends in
bias and accuracy across estimators.
Factor effects
Estimator performance was little affected by Config, support-
The factors Strue, Effort, and Abund had the largest effects on ing both Wagner & Wildi (2002) and Brose et al. (2003). The
estimator performance, supporting Brose et al. (2003), and the three configuration patterns differed considerably for a single
strongest correlations with sc (i.e. the correlation between sc and species, but at the assemblage level the differences were minimal.
Strue = −0.66, Effort = 0.48, Abund = 0.37, N = 0.18, P = 0.04, This emergent property could partly explain the relatively weak
Config = 0.01 and Design = 0.01); note that the sign of correla- effects that Config had on estimator performance and the effects
tion is irrelevant for the nominal variables Abund, Config, Design of an assemblage-wide configuration pattern could be greater.
and p. This suggests that sc is the means by which factors affect Most estimators were more negatively biased with increased
performance and supports Brose et al. (2003). The negative cor- aggregation, which supports Baltanás (1992, their Fig. 3), but
relation that we found between Strue and sc, however, contradicts not Wagner & Wildi (2002). Furthermore, we did not find the
the positive and relatively weak correlation (i.e. r = 0.14) found positive relationship between accuracy and aggregation
by Brose et al. (2003). reported by Walther & Morand (1998).
The negative bias of most estimators was largest with the Survey design also had small effects on performance. Its
relatively uneven log-series distribution (Table S1), which sup- importance could be greater if, for example, SR varied along one
ports Wagner & Wildi (2002) and Brose et al. (2003). Wagner & or more gradients and the orientation of linear transects failed
Wildi (2002) found less bias with log-normal distributions than to fully represent an area. We are unaware of other investigations
with the distribution selected for its relative evenness, the of the relationships between survey design and SR estimates, but
broken-stick distribution. However, we did not find a significant differences between survey designs are important to other esti-
difference between simulated broken-stick and log-normal dis- mation issues (see Reese et al., 2005), including practicality due
tributions using Kolmogorov–Smirnov goodness of fit tests to logistics when implementing sampling regimes.
(G.C.R., unpublished analysis; for the test see Magurran, 2004),
which could explain our contradictory result when using
Estimator selection framework
particulate-niche distributions instead (Table S1). Previous
comparisons, therefore, might not have included enough vari- Near the boundaries of sc, we found that accuracy can be max-
ation to reveal actual relationships between estimator bias and imized with different estimators than those proposed in Brose
assemblage evenness. et al. (2003), specifically CY-1 when sc ≤ 20% and Boot when
At larger values of Strue, estimators were generally more biased, sc > 80% (Table 4). We also found the best performing estimator
supporting Baltanás (1992). Brose et al. (2003) reported a to vary between bias, precision and accuracy; therefore selection
contradictory result where estimators were more biased at should be application specific. Our selection framework and the
Strue = 25 than at Strue = 500; however, it is unclear whether that one by Brose et al. (2003) are based on sc (i.e. Sobs/Strue) a quan-
relationship held across their other levels of Strue (see their tity that, if known, would make estimation unnecessary. Across
Fig. 1). We also found that estimators were less accurate at larger our simulations, the correlation between sc and sc exceeded
values of Strue which contradicts Walther & Morand (1998; see 0.80. Thus, we recommend using the program SimAssem to
Table S1.1). Better performances at the larger Effort level, which process data and the reported sc to locate the coverage range of
supports Brose et al. (2003) and Wagner & Wildi (2002), could suggested estimators. Given the inconsistent performance of
be caused by more effort resulting in more data, particularly Mixture, we recommend caution when considering it for bias
encounters (see Table S1.6). The number of encounters can also reduction (see Table 4 and Appendix S1). We also tested our
be increased with larger values of N and p. Estimator perfor- selection framework against field data and present the results in
mance improved when N was increased from 6250 to 12,500 Appendix S3.
G. C. Reese et al.
and biased estimator can lead to a false sense of confidence when

Statistical estimation issues
compared with an imprecise, but unbiased, estimator. In our
Our results included estimates that were < Sobs, even negative, study, precision was based on empirical estimates of variance
and others that were unrealistically large. We therefore evaluated across the replicated simulations, but a single SR estimate is of
the effects of sensible substitutions. When the count associated limited value when not accompanied by some measure of its
with one or more of the negative terms of a jackknife estimator reliability. Although rarely reported in the literature, SR variance
is large, the final estimate can be < Sobs (Lam & Kleinn, 2008). estimators have been derived for many of the SR estimators and
Otis et al. (1978) considered the observed number, Sobs in species their performance should also be considered when selecting a
richness applications, a reasonable lower confidence interval species richness estimator (Reese, 2012).
boundary because at least that many species are known to exist.
We tested three approaches when a jackknife estimate was < Sobs.
First, an estimate < Sobs was set equal to Sobs. Second, we substi- ACKNOWLEDGEMENTS
tuted the next lower-order jackknife estimate that was ≥ Sobs.
We thank Amy Angert, E. Apple Snider, Thomas Stohlgren,
Third, we substituted the larger of Sobs or the largest jackknife
Bruno Walther and an anonymous referee for their helpful com-
estimate, regardless of its order. Burnham & Overton (1978)
ments on earlier versions of this manuscript.
introduced a statistical test for selecting a jackknife estimator
that was not tested here (but see Brose et al., 2003). Of the three
approaches, we found that substituting the larger of Sobs or the
REFERENCES
largest jackknife estimate resulted in the best performance and
therefore recommend that this approach be considered when Arrhenius, O. (1921) Species and area. Journal of Ecology, 9,
using the jackknife estimators (see Appendix S2). 95–99.
There were also simulations where encounters occurred in Baltanás, A. (1992) On the use of some methods for the estima-
too few surveys for the computation of CY-1 and CY-2 and the tion of species richness. Oikos, 65, 484–492.
CY-2 estimate was negative in other runs (0.14%), demonstrat- Brose, U., Martinez, N.D. & Williams, R.J. (2003) Estimating
ing the limitations of these estimators. A CY-2 estimate can be species richness: sensitivity to sample coverage and insensitiv-
< Sobs when SR and JC are negatively related, a chance relation- ity to spatial patterns. Ecology, 84, 2364–2377.
ship when data are repeatedly randomized. In both cases, we Burnham, K.P. & Overton, W.S. (1978) Estimation of the size of
tested the effect of replacing the non-negative or negative esti- a closed population when capture probabilities vary among
mate with Sobs. In simulations where CY-2 failed to produce an animals. Biometrika, 65, 623–633.
estimate, Mixture always either failed to converge or gave an Burnham, K.P. & Overton, W.S. (1979) Robust estimation of
unreasonably large estimate [e.g. ≥ (8059 × Sobs)], providing population size when capture probabilities vary among
further evidence that sparse data affected estimates. Systemati- animals. Ecology, 60, 927–936.
cally varying both the number of surveys and the number of Canning-Clode, J., Valdivia, N., Molis, M., Thomason, J.C. &
encounters will probably be required to define the thresholds at Wahl, M. (2008) Estimation of regional richness in marine
which valid Mixture estimates become possible. Relative to Sobs, benthic communities: quantifying the error. Limnology and
CY-2 returned the next largest estimate (approximately Sobs × Oceanography: Methods, 6, 580–590.
76) which we used as a proportionality threshold above which Cao, Y., Larsen, D.P. & Hughes, R.M. (2001) Estimating total
Mixture estimates were replaced with Sobs. These substitutions species richness in fish assemblage surveys: a similarity based
had minimal effects on the performances of CY-1 and CY-2, but approach. Canadian Journal of Fisheries and Aquatic Sciences,
occasionally dramatic effects on the performance of Mixture 58, 1782–1793.
(Appendix S2). Cao, Y., Larsen, D.P. & White, D. (2004) Estimating regional
species richness using a limited number of survey units.
Ecoscience, 11, 23–35.
Implications
Chao, A. (1984) Non-parametric estimation of the number of
Our study suggests that differences between assemblage charac- classes in a population. Scandinavian Journal of Statistics, 11,
teristics and survey designs partly explain reported differences 265–270.
in estimator performance. Inconsistencies in the use of perfor- Chao, A. (1987) Estimating the population size for capture–
mance metrics almost certainly further complicate comparisons recapture data with unequal catchability. Biometrics, 43, 783–
between studies (see Walther & Moore, 2005). Despite simulat- 791.
ing a wide range of conditions, additional real-world factors and Chao, A. & Lee, S.M. (1992) Estimating the number of classes
factor levels will undoubtedly complicate estimation efforts; via sample coverage. Journal of the American Statistical Asso-
therefore, the application of our results will be most beneficial ciation, 87, 210–217.
when combined with simulations using programs such as Chiarucci, A., Enright, N.J., Perry, G.L.W. & Miller, B.P. (2003)
SimAssem (Reese et al., 2013). Performance of nonparametric species richness estimators in
The most biased estimators were generally the most precise. a high diversity plant community. Diversity and Distributions,
This is a particularly dangerous combination because a precise 9, 283–295.
Colwell, R.K. & Coddington, J.A. (1994) Estimating terrestrial Moreno, C., Zuria, I., García-Zenteno, M., Sánchez-Rojas, G.,
biodiversity through extrapolation. Philosophical Transactions Castellanos, I., Martínez-Morales, M. & Rojas-Martínez, A.
of the Royal Society B: Biological Sciences, 345, 101–118. (2006) Trends in the measurement of alpha diversity in the
Dallmeier, F., Foster, R.B., Romano, C., Rice, R. & Kabel, M. last two decades. Interciencia, 31, 67–71.
(1991) A user’s guide to the Beni Biosphere Reserve biodiversity Nichols, J.D., Boulinear, T., Hines, J.E., Pollock, K.H. & Sauer,
plots. Smithsonian Institution, Washington, DC. J.R. (1998) Inference methods for spatial variation in species
Deblauwe, V., Barbier, N., Couteron, P., Lejeune, O. & Bogaert, J. richness and community composition when not all species are
(2008) The global biogeography of semi-arid periodic vegeta- detected. Conservation Biology, 12, 1390–1398.
tion patterns. Global Ecology and Biogeography, 17, 715–723. Otis, D.L., Burnham, K.P., White, G.C. & Anderson, D.R. (1978)
Drake, M.T. (2007) Estimating sampling effort for biomo- Statistical inference from capture data on closed animal popu-
nitoring of nearshore fish communities in small central Min- lations. Wildlife Monographs, 62, 1–135.
nesota lakes. North American Journal of Fisheries Management, Pagano, A.M. & Arnold, T.W. (2009) Estimating detection prob-
27, 1094–1111. abilities of waterfowl broods from ground-based surveys.
Fisher, R.A., Corbet, A.S. & Williams, C.B. (1943) The relation Journal of Wildlife Management, 73, 686–694.
between the number of species and the number of individuals Palmer, M.W. (1990) The estimation of species richness by
in a random sample of an animal population. Journal of extrapolation. Ecology, 71, 1195–1198.
Animal Ecology, 12, 42–58. Pledger, S. (2000) Unified maximum likelihood estimates for
Holdridge, L.R., Grenke, W.C., Hatheway, W.H., Liang, T. & Tosi, closed capture–recapture models using mixtures. Biometrics,
J.A. (1971) Forest environments in tropical life zones. Pergamon 56, 434–442.
Press, Oxford. Preston, F.W. (1948) The commonness, and rarity, of species.
Jobe, R.T. (2008) Estimating landscape-scale species richness: Ecology, 29, 254–283.
reconciling frequency- and turnover-based approaches. R Development Core Team (2009) R: a language and environ-
Ecology, 89, 174–182. ment for statistical computing. R Foundation for Statistical
Keating, K.A. & Quinn, J.F. (1998) Estimating species richness: Computing, Vienna, Austria. Available at: http://www
the Michaelis–Menten model revisited. Oikos, 81, 411–416. .r-project.org (accessed 24 October 2013).
Kéry, M. & Plattner, M. (2007) Species richness estimation and Reese, G.C. (2012) Simulating species assemblages and evaluating
determinants of species detectability in butterfly monitoring species richness estimators. PhD thesis, Colorado State Univer-
programmes. Ecological Entomology, 32, 53–61. sity, Fort Collins. Available at: http://hdl.handle.net/10217/
Laake, J. & Rexstad, E. (2008) RMark – an alternative approach 68193 (accessed 24 October 2013).
to building linear models in MARK. Program mark: ‘a gentle Reese, G.C., Wilson, K.R., Hoeting, J.A. & Flather, C.H. (2005)
introduction’, 7th edn (ed. by E. Cooch and G.C. White), Factors affecting species distribution predictions: a simulation
pp. C1–C115. Available at: http://www.phidot.org/software/ modeling experiment. Ecological Applications, 15, 554–564.
mark/docs/book/ (accessed 24 October 2013). Reese, G.C., Wilson, K.R. & Flather, C.H. (2013) Program
Lam, T.Y. & Kleinn, C. (2008) Estimation of tree species richness SimAssem: software for simulating species assemblages and
from large area forest inventory data: evaluation and compari- estimating species richness. Methods in Ecology and Evolution,
son of jackknife estimators. Forest Ecology and Management, 4, 891–896.
255, 1002–1010. SAS Institute (1999) SAS/STAT user’s guide, version 8. SAS Insti-
Lee, S.M. & Chao, A. (1994) Estimating population size via tute, Cary, NC.
sample coverage for closed capture–recapture models. Selmi, S. & Boulinier, T. (2004) Distribution–abundance rela-
Biometrics, 50, 88–97. tionship for passerines breeding in Tunisian oases: test of the
Legendre, P. (1993) Spatial autocorrelation: trouble or new para- sampling hypothesis. Oecologia, 139, 440–445.
digm? Ecology, 74, 1659–1673. Smith, E.P. & van Belle, G. (1984) Nonparametric estimation of
Lewis, T. & Taylor, L.R. (1967) Introduction to experimental species richness. Biometrics, 40, 119–129.
ecology. Academic Press, London. Sugihara, G. (1980) Minimal community structure: an explana-
Lopez, L.C.S., Fracasso, M.P.D.A., Mesquita, D.O., Palma, A.R.T. tion of species abundance patterns. The American Naturalist,
& Riul, P. (2012) The relationship between percentage of sin- 116, 770–787.
gletons and sampling effort: a new approach to reduce the bias Tjørve, E. (2009) Shapes and functions of species–area curves
of richness estimates. Ecological Indicators, 14, 164–169. (II): a review of new models and parameterizations. Journal of
MacArthur, R.H. (1957) On the relative abundance of bird Biogeography, 36, 1435–1445.
species. Proceedings of the National Academy of Sciences USA, Ulrich, W., Ollik, M. & Ugland, K.I. (2010) A meta-analysis of
43, 293–295. species-abundance distributions. Oikos, 119, 1149–1155.
Magurran, A.E. (2004) Measuring biological diversity. Blackwell Wagner, H.H. & Wildi, O. (2002) Realistic simulation of the
Publishing, Malden, MA. effects of abundance distribution and spatial heterogeneity on
Michaelis, M. & Menten, M.L. (1913) Der Kinetic der non-parametric estimators of species richness. Ecoscience, 9,
Invertinwirkung. Biochemische Zeitschrift, 49, 333–369. 241–250.
G. C. Reese et al.
Walther, B.A. & Moore, J.L. (2005) The concepts of bias, preci- Appendix S3 Case study of the performance of our proposed
sion and accuracy, and their use in testing the performance of selection framework and the asymptotic properties of
species richness estimators, with a literature review of estima- estimators.
tor performance. Ecography, 28, 815–829.
Walther, B.A. & Morand, S. (1998) Comparative performance of
species richness estimation methods. Parasitology, 116, 395– BIOSKETCHES
405.
Gordon C. Reese recently completed his PhD in the
White, G.C. & Burnham, K.P. (1999) Program MARK: survival
Graduate Degree Program in Ecology at Colorado State
estimation from populations of marked animals. Bird Study,
University. He is researching the factors affecting the
46, Supplement, 120–138.
performance of species richness estimators and also
Williams, C.B. (1939) An analysis of four years’ captures of
the relative performance of several approaches for
insects in a light trap. Part I. Transactions of the Royal Ento-
estimating the variance of species richness estimators.
mological Society of London, 89, 79–132.
Willie, J., Petre, C.A., Tagg, N. & Lens, L. (2012) Evaluation of Kenneth R. Wilson is a professor in the Department
species richness estimators based on quantitative perfor- of Fish, Wildlife, and Conservation Biology and
mance measures and sensitivity to patchiness and sample Graduate Degree Program in Ecology at Colorado State
grain size. Acta Oecologica, 45, 31–41. University. He has an interest in wildlife management,
conservation biology and ecology specifically related to:
S U P P O RT I N G I N F O R M AT I O N (1) impacts of human activities on wildlife, (2)
population ecology, and (3) understanding patterns of
Additional supporting information may be found in the online species richness.
version of this article at the publisher’s web-site.
Curtis H. Flather is a research ecologist with the US
Table S1 Performance of species richness estimators as a func- Forest Service, Rocky Mountain Research Station in
tion of species abundance distribution, log-normal, log-series Fort Collins, CO. His research programme focuses on
and particulate-niche, averaged across all factor levels. estimating the response of species assemblages to
Table S2 A variance component analysis of relative factor effects. natural and human-caused disturbance to support
Appendix S1 Estimator bias, precision and accuracy across resource planning within the agency.
factor levels (Tables S1.1–S1.7).
Appendix S2 Estimator bias, precision and accuracy across Editor: Gary Mittelbach
factor levels, after accounting for estimation issues (Tables
S2.1–S2.7).
Table S1 Performance of species richness estimators as a function of species abundance distribution, log-
normal (LN), log-series (LS), and particulate-niche (PN), averaged across all factor levels. Subscripts are
estimator rank for a given performance measure and abundance distribution. See Table 2 for estimator
abbreviations and Table 3 for specific performance metrics.
Bias (SME) Precision (SD) Accuracy (SMSE)

Estimator LN LS PNa LN LS PN LN LS PN
ACE -0.187 -0.479 0.002 0.254 0.185 0.273 0.093 0.256 0.073
Boot -0.3812 -0.6013 -0.2810 0.307 0.183 0.339 0.2411 0.3912 0.199
Chao1 -0.2410 -0.5011 -0.077 0.252 0.197 0.212 0.126 0.299 0.052
Chao2 -0.249 -0.5010 -0.076 0.241 0.196 0.211 0.125 0.298 0.051
CY-1 0.021 -0.406 0.4413 0.275 0.161 0.5611 0.071 0.182 0.5113
CY-2 0.042 -0.353 0.4212 0.3110 0.2410 0.5712 0.104 0.181 0.5012
ICE -0.186 -0.478 0.013 0.253 0.184 0.284 0.092 0.255 0.084
Jack1 -0.2911 -0.5312 -0.189 0.319 0.198 0.326 0.189 0.3210 0.147
Jack2 -0.208 -0.457 -0.118 0.318 0.229 0.305 0.138 0.257 0.105
Jack3 -0.155 -0.405 -0.065 0.3411 0.2611 0.337 0.137 0.233 0.116
Jack4 -0.104 -0.354 -0.034 0.4312 0.3412 0.4310 0.2010 0.244 0.188
Jack5 -0.063 -0.302 0.001 0.6513 0.5013 0.6713 0.4213 0.3411 0.4511
Mixture 1.3514 -0.071 3.8714 17.9314 6.0914 35.9814 323.3514 37.0814 1309.2614
Sobs -0.4613 -0.6514 -0.3711 0.306 0.172 0.338 0.3012 0.4513 0.2410
aParticulate-niche distributions were created by randomly and independently assigning each individual to a
species.
Notes: A large number of simulations with unreasonable estimates, i.e. estimates smaller than or
substantially greater than Sobs, were found in factor combinations with the particulate-niche distribution (in as
many as 33 of 37 replicates, i.e. once simulations in which Mixture failed to converge were removed and the
factorial was rebalanced by removing ≤5 replicates from each factor combination) as were the worst
performances (all measures) by CY-1, CY-2, and Mixture. Additionally, there is little empirical support for
particulate-niche distributions, compared to log-normal and log-series distributions; therefore, the results are
presented separately here.
The similarity of log-normal and broken-stick distributions (see text) could have contributed to Wagner and
Wildi (2002) finding that estimators are more negatively biased with an even distribution (i.e. a broken-stick)
than with an uneven distribution (i.e. a log-normal) which was opposite of our finding. We found that
estimators were most negatively biased in assemblages with relatively uneven log-series distributions,
supporting both Wagner and Wildi (2002) and Brose et al. (2003).
REFERENCES
Brose, U., Martinez, N.D. & Williams, R.J. (2003) Estimating species richness: sensitivity to sample coverage
and insensitivity to spatial patterns. Ecology, 84, 2364-2377.
Wagner, H.H. & Wildi, O. (2002) Realistic simulation of the effects of abundance distribution and spatial
heterogeneity on non-parametric estimators of species richness. Ecoscience, 9, 241-250.
Table S1, Reese et al., Performance of species richness estimators, Pg. 1

Table S2 A variance component analysis of relative factor effects. The values represent the percent variance
attributable to each of the independent factors. The residual represents the variance explained by all
interactions and residual error. Subscripts are factor rank for a given species richness estimator and
performance measure. See Table 2 for estimator abbreviations and Table 3 for specific performance metrics.
Bias (SME)
Estimator Strue N Abund Config Design Effort p Residual
ACE 17.732 1.026 57.131 0.347 0.048 17.603 2.065 4.074
Boot 42.021 3.645 19.623 0.017 0.008 28.242 1.216 5.254
Chao1 22.203 2.655 42.561 0.207 0.018 26.902 1.636 3.854
Chao2 22.023 2.585 42.831 0.147 0.038 26.992 1.646 3.784
CY-1 0.725 0.008 85.381 0.336 0.167 2.354 3.203 7.862
CY-2 3.735 0.008 75.411 0.416 0.167 7.183 3.784 9.322
ICE 17.262 0.946 58.581 0.227 0.088 16.703 2.115 4.114
Jack1 38.461 3.295 23.103 0.027 0.008 27.892 1.216 6.034
Jack2 33.741 2.875 26.603 0.027 0.008 27.852 1.256 7.684
Jack3 30.691 2.555 27.962 0.027 0.008 27.833 1.286 9.674
Jack4 29.391 2.325 27.332 0.017 0.008 27.213 1.286 12.474
Jack5 29.431 2.105 24.533 0.007 0.008 25.232 1.246 17.474
Mixture 6.462 3.454 3.225 0.008 0.117 6.283 1.846 78.631
Sobs 44.371 3.905 16.773 0.017 0.008 28.632 1.236 5.094
Mean 24.162 2.245 37.931 0.127 0.048 21.213 1.786 12.524
Precision (SD)
ACE 0.206 9.263 5.544 0.047 0.008 43.831 4.675 36.472
Boot 73.661 0.715 1.504 0.007 0.008 3.943 0.106 20.082
Chao1 9.673 4.984 3.515 0.097 0.008 31.462 2.466 47.831
Chao2 9.993 6.434 3.345 0.177 0.008 31.152 2.426 46.511
CY-1 3.826 7.684 12.503 0.008 0.017 45.021 4.185 26.792
CY-2 2.716 8.253 4.254 0.008 0.007 32.582 3.995 48.231
ICE 0.176 9.693 5.844 0.008 0.007 44.471 4.695 35.142
Jack1 76.731 0.775 0.874 0.008 0.007 3.463 0.086 18.092
Jack2 83.411 0.455 1.304 0.008 0.007 1.443 0.046 13.362
Jack3 87.441 0.105 2.523 0.008 0.007 0.134 0.016 9.792
Jack4 87.771 0.006 3.833 0.008 0.007 0.094 0.005 8.312
Jack5 86.551 0.006 4.613 0.008 0.007 0.944 0.005 7.902
Mixture 8.743 4.614 1.946 0.008 0.017 10.012 2.455 72.231
Sobs 74.881 0.435 3.073 0.008 0.007 2.784 0.126 18.722
Mean 43.271 3.815 3.904 0.027 0.008 17.953 1.806 29.252
Accuracy (SMSE)
ACE 17.053 2.205 36.591 0.147 0.098 36.212 1.436 6.284
Boot 45.261 4.075 10.953 0.017 0.008 32.622 1.456 5.644
Chao1 18.853 3.335 32.362 0.207 0.068 38.261 1.946 5.004
Chao2 18.603 3.395 33.092 0.127 0.078 37.781 1.936 5.034
CY-1 3.504 1.056 33.182 0.007 0.008 37.981 1.715 22.583
CY-2 5.414 2.805 12.583 0.007 0.008 43.831 1.636 33.742
ICE 16.593 2.265 37.611 0.088 0.107 35.752 1.456 6.154
Jack1 40.241 3.995 11.373 0.008 0.007 34.562 1.476 8.364
Jack2 32.122 4.095 11.514 0.008 0.007 38.721 1.606 11.963
Jack3 21.892 4.635 9.724 0.008 0.007 45.991 1.956 15.813
Jack4 18.013 4.924 2.196 0.008 0.007 47.181 2.465 25.242
Jack5 61.731 1.235 2.874 0.008 0.007 7.423 0.956 25.802
Mixture 3.022 2.294 1.585 0.008 0.437 2.683 1.026 88.991
Sobs 48.361 4.194 10.223 0.017 0.008 31.722 1.456 4.055
Mean 25.052 3.185 17.564 0.048 0.057 33.621 1.606 18.903
Table S2, Reese et al., Performance of species richness estimators, Pg. 1

Appendix S1: Estimator bias, precision, and accuracy across factor levels.
Table S1.1 Performance of species richness estimators as a function of the true number of species, 25, 100,
and 500 species, averaged across all factor levels. Subscripts are estimator rank for a given performance
measure and true number of species. See Table 2 for estimator abbreviations and Table 3 for specific
performance metrics.

Estimator 25 100 500 25 100 500 25 100 500
ACE -0.229 -0.319 -0.446 0.244 0.257 0.258 0.115 0.167 0.266
Boot -0.2812 -0.4613 -0.7312 0.222 0.222 0.162 0.139 0.2612 0.5512
Chao1 -0.2510 -0.3411 -0.519 0.257 0.245 0.205 0.138 0.189 0.309
Chao2 -0.2511 -0.3410 -0.518 0.256 0.244 0.204 0.137 0.178 0.308
CY-1 -0.166 -0.185 -0.222 0.279 0.3212 0.3212 0.102 0.145 0.151
CY-2 -0.093 -0.153 -0.211 0.3110 0.3211 0.3713 0.116 0.132 0.182
ICE -0.228 -0.318 -0.445 0.245 0.258 0.259 0.114 0.166 0.255
Jack1 -0.217 -0.3712 -0.6511 0.233 0.233 0.193 0.103 0.1910 0.4511
Jack2 -0.165 -0.277 -0.5510 0.268 0.246 0.226 0.091 0.134 0.3510
Jack3 -0.124 -0.216 -0.487 0.3511 0.279 0.247 0.1310 0.111 0.297
Jack4 -0.082 -0.164 -0.434 0.5213 0.3210 0.2610 0.2712 0.133 0.254
Jack5 -0.031 -0.122 -0.393 0.8414 0.4413 0.2811 0.7114 0.2011 0.233
Mixture -0.3013 -0.051 2.2714 0.4612 6.6014 22.1814 0.3013 43.5514 496.8014
Sobs -0.3514 -0.5414 -0.7813 0.221 0.201 0.131 0.1711 0.3313 0.6313
Table S1.2 Performance of species richness estimators as a function of species abundance distribution, log-
normal (LN) and log-series (LS), averaged across all factor levels. Subscripts are estimator rank for a given
performance measure and abundance distribution. See Table 2 for estimator abbreviations and Table 3 for
specific performance metrics.

Estimator LN LS LN LS LN LS
ACE -0.187 -0.479 0.254 0.185 0.093 0.256
Boot -0.3812 -0.6013 0.307 0.183 0.2411 0.3912
Chao1 -0.2410 -0.5011 0.252 0.197 0.126 0.299
Chao2 -0.249 -0.5010 0.241 0.196 0.125 0.298
CY-1 0.021 -0.406 0.275 0.161 0.071 0.182
CY-2 0.042 -0.353 0.3110 0.2410 0.104 0.181
ICE -0.186 -0.478 0.253 0.184 0.092 0.255
Jack1 -0.2911 -0.5312 0.319 0.198 0.189 0.3210
Jack2 -0.208 -0.457 0.318 0.229 0.138 0.257
Jack3 -0.155 -0.405 0.3411 0.2611 0.137 0.233
Jack4 -0.104 -0.354 0.4312 0.3412 0.2010 0.244
Jack5 -0.063 -0.302 0.6513 0.5013 0.4213 0.3411
Mixture 1.3514 -0.071 17.9314 6.0914 323.3514 37.0814
Sobs -0.4613 -0.6514 0.306 0.172 0.3012 0.4513
Appendix S1, Reese et al., Performance of species richness estimators, Pg. 1

Table S1.3 Performance of species richness estimators as a function of the total number of individuals,
6,250 and 12,500 individuals, averaged across all factor levels. Subscripts are estimator rank for a given
performance measure and total number of individuals. See Table 2 for estimator abbreviations and Table 3
for specific performance metrics.

Estimator 6250 12500 6250 12500 6250 12500
ACE -0.347 -0.309 0.275 0.253 0.194 0.155
Boot -0.5412 -0.4413 0.274 0.276 0.3611 0.2711
Chao1 -0.4010 -0.3410 0.263 0.254 0.237 0.178
Chao2 -0.409 -0.3411 0.262 0.251 0.236 0.177
CY-1 -0.192 -0.194 0.329 0.299 0.141 0.121
CY-2 -0.161 -0.153 0.3611 0.3111 0.162 0.122
ICE -0.346 -0.308 0.286 0.252 0.193 0.154
Jack1 -0.4511 -0.3612 0.297 0.277 0.2910 0.2110
Jack2 -0.378 -0.297 0.308 0.288 0.238 0.166
Jack3 -0.315 -0.236 0.3410 0.3110 0.215 0.153
Jack4 -0.264 -0.195 0.4212 0.3912 0.259 0.199
Jack5 -0.213 -0.142 0.6013 0.5713 0.4112 0.3513
Mixture 1.3814 -0.091 18.6114 3.5014 348.1514 12.2914
Sobs -0.6013 -0.5114 0.251 0.265 0.4313 0.3312
Table S1.4 Performance of species richness estimators as a function of the spatial configuration of
individuals of a species, aggregated (Agg), hyper-dispersed (Hyp), and random (Rnd), averaged across all
factor levels. Subscripts are estimator rank for a given performance measure and spatial configuration. See
Table 2 for estimator abbreviations and Table 3 for specific performance metrics.

Estimator Agg Hyp Rnd Agg Hyp Rnd Agg Hyp Rnd
ACE -0.348 -0.317 -0.327 0.262 0.274 0.265 0.184 0.174 0.174
Boot -0.5013 -0.4812 -0.4912 0.276 0.276 0.276 0.3211 0.3111 0.3111
Chao1 -0.3810 -0.369 -0.379 0.264 0.262 0.252 0.218 0.207 0.208
Chao2 -0.389 -0.3610 -0.3710 0.251 0.261 0.251 0.217 0.208 0.207
CY-1 -0.213 -0.173 -0.182 0.299 0.319 0.309 0.131 0.131 0.121
CY-2 -0.181 -0.141 -0.151 0.3310 0.3511 0.3411 0.142 0.142 0.142
ICE -0.346 -0.316 -0.326 0.265 0.275 0.264 0.183 0.173 0.173
Jack1 -0.4211 -0.4011 -0.4111 0.287 0.297 0.287 0.2510 0.2410 0.2510
Jack2 -0.347 -0.328 -0.338 0.298 0.308 0.298 0.206 0.196 0.196
Jack3 -0.285 -0.265 -0.275 0.3311 0.3310 0.3210 0.185 0.185 0.185
Jack4 -0.234 -0.214 -0.234 0.4112 0.4112 0.4012 0.229 0.229 0.229
Jack5 -0.182 -0.172 -0.193 0.5913 0.5913 0.5913 0.3812 0.3813 0.3813
Mixture 0.4512 0.9114 0.5614 10.6914 17.0814 11.5514 114.4814 292.4314 133.7514
Sobs -0.5614 -0.5513 -0.5613 0.263 0.263 0.263 0.3813 0.3712 0.3812

Table S1.5 Performance of species richness estimators as a function of species detection probability,
averaged across all factor levels. Detection probabilities were randomly drawn from either: (1) beta
distributions with expected values of 0.5 or 0.9 or (2) beta distributions with expected values that either
increased (I) or decreased (D) with abundance after first grouping species into thirds based on ranked
abundance. Subscripts are estimator rank for a given performance measure and detection probability. See
Table 2 for estimator abbreviations and Table 3 for specific performance metrics.

Estimator 0.5 0.9 0.5 0.9 0.5 0.9
ACE -0.367 -0.309 0.274 0.243 0.204 0.155
Boot -0.5412 -0.4513 0.275 0.276 0.3611 0.2711
Chao1 -0.4110 -0.3311 0.263 0.242 0.248 0.178
Chao2 -0.419 -0.3310 0.262 0.241 0.247 0.177
CY-1 -0.222 -0.174 0.319 0.277 0.141 0.101
CY-2 -0.191 -0.142 0.3411 0.3111 0.152 0.112
ICE -0.366 -0.308 0.276 0.244 0.203 0.154
Jack1 -0.4611 -0.3612 0.297 0.278 0.2910 0.2110
Jack2 -0.388 -0.287 0.308 0.289 0.236 0.166
Jack3 -0.325 -0.236 0.3410 0.3110 0.225 0.153
Jack4 -0.284 -0.195 0.4212 0.3912 0.259 0.199
Jack5 -0.233 -0.143 0.6013 0.5713 0.4212 0.3413
Mixture 1.4114 -0.061 18.4614 4.0014 342.7114 16.0014
Sobs -0.6013 -0.5114 0.251 0.265 0.4313 0.3312

Estimator I D I D I D
ACE -0.359 -0.287 0.232 0.295 0.185 0.164
Boot -0.4913 -0.4912 0.257 0.294 0.3011 0.3211
Chao1 -0.3811 -0.3510 0.245 0.272 0.219 0.197
Chao2 -0.3810 -0.359 0.243 0.271 0.208 0.196
CY-1 -0.255 -0.112 0.246 0.3610 0.121 0.141
CY-2 -0.213 -0.081 0.2810 0.4011 0.122 0.163
ICE -0.358 -0.286 0.231 0.296 0.184 0.162
Jack1 -0.4112 -0.4011 0.268 0.307 0.2410 0.2510
Jack2 -0.337 -0.328 0.279 0.318 0.186 0.208
Jack3 -0.276 -0.265 0.3111 0.349 0.173 0.185
Jack4 -0.234 -0.214 0.3912 0.4312 0.207 0.239
Jack5 -0.182 -0.153 0.5713 0.6213 0.3512 0.4113
Mixture -0.101 1.3114 4.2214 18.5114 17.8114 344.3414
Sobs -0.5514 -0.5613 0.244 0.273 0.3613 0.3912

Table S1.6 Performance of species richness estimators as a function of survey effort, 100 (1%) and 500
(5%) cells, averaged across all factor levels. Subscripts are estimator rank for a given performance measure
and survey effort. See Table 2 for estimator abbreviations and Table 3 for specific performance metrics.

Estimator 1% 5% 1% 5% 1% 5%
ACE -0.407 -0.248 0.306 0.191 0.254 0.096
Boot -0.6212 -0.3613 0.252 0.2310 0.4411 0.1812
Chao1 -0.4710 -0.2710 0.264 0.205 0.297 0.118
Chao2 -0.479 -0.269 0.263 0.204 0.296 0.117
CY-1 -0.222 -0.155 0.3710 0.216 0.191 0.072
CY-2 -0.211 -0.092 0.4111 0.239 0.212 0.061
ICE -0.406 -0.247 0.307 0.192 0.253 0.095
Jack1 -0.5411 -0.2811 0.285 0.228 0.3710 0.139
Jack2 -0.468 -0.206 0.318 0.217 0.308 0.094
Jack3 -0.405 -0.154 0.359 0.2512 0.285 0.083
Jack4 -0.354 -0.103 0.4312 0.3413 0.319 0.1310
Jack5 -0.303 -0.061 0.6013 0.5514 0.4512 0.3114
Mixture 1.6214 -0.3312 18.9114 0.193 360.2914 0.1511
Sobs -0.6813 -0.4314 0.221 0.2411 0.5113 0.2413
Table S1.7 Performance of species richness estimators as a function of survey design, random (Random)
and linear transect (Transect), averaged across all factor levels. Subscripts are estimator rank for a given
performance measure and survey design. See Table 2 for estimator abbreviations and Table 3 for specific

Estimator Random Transect Random Transect Random Transect
ACE -0.327 -0.337 0.264 0.265 0.174 0.184
Boot -0.4912 -0.4913 0.276 0.276 0.3111 0.3211
Chao1 -0.3710 -0.379 0.252 0.263 0.208 0.218
Chao2 -0.369 -0.3710 0.251 0.261 0.207 0.217
CY-1 -0.182 -0.203 0.319 0.309 0.131 0.131
CY-2 -0.141 -0.161 0.3411 0.3311 0.142 0.142
ICE -0.326 -0.336 0.265 0.264 0.173 0.183
Jack1 -0.4111 -0.4112 0.287 0.287 0.2410 0.2510
Jack2 -0.338 -0.338 0.298 0.308 0.196 0.206
Jack3 -0.275 -0.275 0.3210 0.3310 0.185 0.185
Jack4 -0.224 -0.224 0.4112 0.4112 0.219 0.229
Jack5 -0.183 -0.182 0.5913 0.5913 0.3813 0.3813
Mixture 0.8814 0.4011 16.4114 9.5014 270.1114 90.3214
Sobs -0.5613 -0.5614 0.263 0.262 0.3812 0.3812

Appendix S2: Estimator bias, precision, and accuracy across factor levels, after accounting for estimation
issues. The tables in this appendix are performances after modifications were made to certain estimates.
Estimates less than the number observed (Sobs) were either increased to Sobs (CY-1 and CY-2) or increased to
the largest estimate across Jack1-Jack5 and Sobs (Jack2 through Jack5). Outliers, some four orders of
magnitude larger than Sobs, greatly affected the evaluation of Mixture. Relative to Sobs, CY-2 returned the next
largest estimate (approximately Sobs x 76) which we used as a proportionality threshold above which 244 Mixture
estimates were replaced with Sobs.
Table S2.1 Performance of species richness estimators as a function of the true number of species, 25, 100,
and 500 species, averaged across all factor levels. Performances that include the substitutions described above
are identified with a (s). See Table 2 for estimator abbreviations and Table 3 for specific performance metrics.

Estimator 25 100 500 25 100 500 25 100 500
ACE -0.22 -0.31 -0.44 0.24 0.25 0.25 0.11 0.16 0.26
Boot -0.28 -0.46 -0.73 0.22 0.22 0.16 0.13 0.26 0.55
Chao1 -0.25 -0.34 -0.51 0.25 0.24 0.20 0.13 0.18 0.30
Chao2 -0.25 -0.34 -0.51 0.25 0.24 0.20 0.13 0.17 0.30
CY-1 -0.16 -0.18 -0.22 0.27 0.32 0.32 0.10 0.14 0.15
CY-1 (s) -0.16 -0.18 -0.22 0.27 0.32 0.32 0.10 0.14 0.15
CY-2 -0.09 -0.15 -0.21 0.31 0.32 0.37 0.11 0.13 0.18
CY-2 (s) -0.09 -0.15 -0.21 0.31 0.32 0.37 0.10 0.13 0.18
ICE -0.22 -0.31 -0.44 0.24 0.25 0.25 0.11 0.16 0.25
Jack1 -0.21 -0.37 -0.65 0.23 0.23 0.19 0.10 0.19 0.45
Jack2 -0.16 -0.27 -0.55 0.26 0.24 0.22 0.09 0.13 0.35
Jack2 (s) -0.15 -0.27 -0.55 0.26 0.24 0.22 0.09 0.13 0.35
Jack3 -0.12 -0.21 -0.48 0.35 0.27 0.24 0.13 0.11 0.29
Jack3 (s) -0.07 -0.21 -0.48 0.31 0.26 0.24 0.10 0.11 0.29
Jack4 -0.08 -0.16 -0.43 0.52 0.32 0.26 0.27 0.13 0.25
Jack4 (s) 0.03 -0.15 -0.43 0.40 0.31 0.26 0.16 0.12 0.25
Jack5 -0.03 -0.12 -0.39 0.84 0.44 0.28 0.71 0.20 0.23
Jack5 (s) 0.19 -0.08 -0.39 0.60 0.39 0.28 0.40 0.16 0.23
Mixture -0.30 -0.05 2.27 0.46 6.60 22.18 0.30 43.55 496.80
Mixture (s) -0.30 -0.32 -0.43 0.46 0.66 0.45 0.30 0.54 0.39
Sobs -0.35 -0.54 -0.78 0.22 0.20 0.13 0.17 0.33 0.63

Table S2.2 Performance of species richness estimators as a function of species abundance distribution, log-
normal (LN) and log-series (LS), averaged across all factor levels. Performances that include the substitutions
described above are identified with a (s). See Table 2 for estimator abbreviations and Table 3 for specific

Estimator LN LS LN LS LN LS
ACE -0.18 -0.47 0.25 0.18 0.09 0.25
Boot -0.38 -0.60 0.30 0.18 0.24 0.39
Chao1 -0.24 -0.50 0.25 0.19 0.12 0.29
Chao2 -0.24 -0.50 0.24 0.19 0.12 0.29
CY-1 0.02 -0.40 0.27 0.16 0.07 0.18
CY-1 (s) 0.02 -0.40 0.27 0.16 0.07 0.18
CY-2 0.04 -0.35 0.31 0.24 0.10 0.18
CY-2 (s) 0.04 -0.34 0.31 0.24 0.10 0.18
ICE -0.18 -0.47 0.25 0.18 0.09 0.25
Jack1 -0.29 -0.53 0.31 0.19 0.18 0.32
Jack2 -0.20 -0.45 0.31 0.22 0.13 0.25
Jack2 (s) -0.19 -0.45 0.31 0.22 0.13 0.25
Jack3 -0.15 -0.40 0.34 0.26 0.13 0.23
Jack3 (s) -0.12 -0.39 0.33 0.25 0.12 0.21
Jack4 -0.10 -0.35 0.43 0.34 0.20 0.24
Jack4 (s) -0.04 -0.32 0.39 0.31 0.15 0.20
Jack5 -0.06 -0.30 0.65 0.50 0.42 0.34
Jack5 (s) 0.06 -0.24 0.53 0.42 0.29 0.23
Mixture 1.35 -0.07 17.93 6.09 323.35 37.08
Mixture (s) -0.19 -0.51 0.61 0.39 0.41 0.41
Sobs -0.46 -0.65 0.30 0.17 0.30 0.45

Table S2.3 Performance of species richness estimators as a function of the total number of individuals, 6,250
and 12,500 individuals, averaged across all factor levels. Performances that include the substitutions described
above are identified with a (s). See Table 2 for estimator abbreviations and Table 3 for specific performance
metrics.

Estimator 6250 12500 6250 12500 6250 12500
ACE -0.34 -0.30 0.27 0.25 0.19 0.15
Boot -0.54 -0.44 0.27 0.27 0.36 0.27
Chao1 -0.40 -0.34 0.26 0.25 0.23 0.17
Chao2 -0.40 -0.34 0.26 0.25 0.23 0.17
CY-1 -0.19 -0.19 0.32 0.29 0.14 0.12
CY-1 (s) -0.19 -0.19 0.32 0.29 0.14 0.12
CY-2 -0.16 -0.15 0.36 0.31 0.16 0.12
CY-2 (s) -0.16 -0.15 0.36 0.31 0.16 0.12
ICE -0.34 -0.30 0.28 0.25 0.19 0.15
Jack1 -0.45 -0.36 0.29 0.27 0.29 0.21
Jack2 -0.37 -0.29 0.30 0.28 0.23 0.16
Jack2 (s) -0.36 -0.28 0.31 0.28 0.23 0.16
Jack3 -0.31 -0.23 0.34 0.31 0.21 0.15
Jack3 (s) -0.29 -0.21 0.34 0.30 0.20 0.14
Jack4 -0.26 -0.19 0.42 0.39 0.25 0.19
Jack4 (s) -0.22 -0.14 0.39 0.36 0.20 0.15
Jack5 -0.21 -0.14 0.60 0.57 0.41 0.35
Jack5 (s) -0.13 -0.05 0.52 0.48 0.29 0.23
Mixture 1.38 -0.09 18.61 3.50 348.15 12.29
Mixture (s) -0.36 -0.34 0.56 0.50 0.45 0.37
Sobs -0.60 -0.51 0.25 0.26 0.43 0.33
Table S2.4 Performance of species richness estimators as a function of the spatial configuration of individuals of
a species, aggregated (Agg), hyper-dispersed (Hyp), and random (Rnd), averaged across all factor levels.
Performances that include the substitutions described above are identified with a (s). See Table 2 for estimator
abbreviations and Table 3 for specific performance metrics.

Estimator Agg Hyp Rnd Agg Hyp Rnd Agg Hyp Rnd
ACE -0.34 -0.31 -0.32 0.26 0.27 0.26 0.18 0.17 0.17
Boot -0.50 -0.48 -0.49 0.27 0.27 0.27 0.32 0.31 0.31
Chao1 -0.38 -0.36 -0.37 0.26 0.26 0.25 0.21 0.20 0.20
Chao2 -0.38 -0.36 -0.37 0.25 0.26 0.25 0.21 0.20 0.20
CY-1 -0.21 -0.17 -0.18 0.29 0.31 0.30 0.13 0.13 0.12
CY-1 (s) -0.21 -0.17 -0.18 0.29 0.31 0.30 0.13 0.13 0.12
CY-2 -0.18 -0.14 -0.15 0.33 0.35 0.34 0.14 0.14 0.14
CY-2 (s) -0.18 -0.14 -0.14 0.32 0.35 0.34 0.14 0.14 0.13
ICE -0.34 -0.31 -0.32 0.26 0.27 0.26 0.18 0.17 0.17
Jack1 -0.42 -0.40 -0.41 0.28 0.29 0.28 0.25 0.24 0.25
Jack2 -0.34 -0.32 -0.33 0.29 0.30 0.29 0.20 0.19 0.19
Jack2 (s) -0.33 -0.31 -0.32 0.30 0.30 0.29 0.20 0.19 0.19
Jack3 -0.28 -0.26 -0.27 0.33 0.33 0.32 0.18 0.18 0.18
Jack3 (s) -0.26 -0.24 -0.25 0.32 0.33 0.32 0.17 0.17 0.17
Jack4 -0.23 -0.21 -0.23 0.41 0.41 0.40 0.22 0.22 0.22
Jack4 (s) -0.19 -0.17 -0.18 0.38 0.38 0.37 0.18 0.18 0.17
Jack5 -0.18 -0.17 -0.19 0.59 0.59 0.59 0.38 0.38 0.38
Jack5 (s) -0.09 -0.08 -0.09 0.51 0.51 0.49 0.27 0.26 0.25
Mixture 0.45 0.91 0.56 10.69 17.08 11.55 114.48 292.43 133.75
Mixture (s) -0.36 -0.33 -0.35 0.49 0.63 0.47 0.38 0.51 0.34
Sobs -0.56 -0.55 -0.56 0.26 0.26 0.26 0.38 0.37 0.38

Table S2.5 Performance of species richness estimators as a function of species detection probability, averaged
across all factor levels. Detection probabilities were randomly drawn from either: (1) beta distributions with
expected values of 0.5 or 0.9 or (2) beta distributions with expected values that either increased (I) or decreased
(D) with abundance after first grouping species into thirds based on ranked abundance. Performances that
include the substitutions described above are identified with a (s). See Table 2 for estimator abbreviations and
Table 3 for specific performance metrics.

Estimator 0.5 0.9 0.5 0.9 0.5 0.9
ACE -0.36 -0.30 0.27 0.24 0.20 0.15
Boot -0.54 -0.45 0.27 0.27 0.36 0.27
Chao1 -0.41 -0.33 0.26 0.24 0.24 0.17
Chao2 -0.41 -0.33 0.26 0.24 0.24 0.17
CY-1 -0.22 -0.17 0.31 0.27 0.14 0.10
CY-1 (s) -0.22 -0.17 0.31 0.27 0.14 0.10
CY-2 -0.19 -0.14 0.34 0.31 0.15 0.11
CY-2 (s) -0.19 -0.14 0.34 0.31 0.15 0.11
ICE -0.36 -0.30 0.27 0.24 0.20 0.15
Jack1 -0.46 -0.36 0.29 0.27 0.29 0.21
Jack2 -0.38 -0.28 0.30 0.28 0.23 0.16
Jack2 (s) -0.37 -0.28 0.31 0.28 0.23 0.16
Jack3 -0.32 -0.23 0.34 0.31 0.22 0.15
Jack3 (s) -0.31 -0.21 0.34 0.30 0.21 0.14
Jack4 -0.28 -0.19 0.42 0.39 0.25 0.19
Jack4 (s) -0.23 -0.14 0.39 0.35 0.21 0.14
Jack5 -0.23 -0.14 0.60 0.57 0.42 0.34
Jack5 (s) -0.15 -0.05 0.52 0.47 0.29 0.22
Mixture 1.41 -0.06 18.46 4.00 342.71 16.00
Mixture (s) -0.38 -0.32 0.60 0.44 0.51 0.29
Sobs -0.60 -0.51 0.25 0.26 0.43 0.33

Estimator I D I D I D
ACE -0.35 -0.28 0.23 0.29 0.18 0.16
Boot -0.49 -0.49 0.25 0.29 0.30 0.32
Chao1 -0.38 -0.35 0.24 0.27 0.21 0.19
Chao2 -0.38 -0.35 0.24 0.27 0.20 0.19
CY-1 -0.25 -0.11 0.24 0.36 0.12 0.14
CY-1 (s) -0.25 -0.11 0.24 0.36 0.12 0.14
CY-2 -0.21 -0.08 0.28 0.40 0.12 0.16
CY-2 (s) -0.21 -0.07 0.28 0.39 0.12 0.16
ICE -0.35 -0.28 0.23 0.29 0.18 0.16
Jack1 -0.41 -0.40 0.26 0.30 0.24 0.25
Jack2 -0.33 -0.32 0.27 0.31 0.18 0.20
Jack2 (s) -0.32 -0.31 0.28 0.32 0.18 0.20
Jack3 -0.27 -0.26 0.31 0.34 0.17 0.18
Jack3 (s) -0.26 -0.24 0.30 0.34 0.16 0.17
Jack4 -0.23 -0.21 0.39 0.43 0.20 0.23
Jack4 (s) -0.19 -0.16 0.36 0.40 0.17 0.18
Jack5 -0.18 -0.15 0.57 0.62 0.35 0.41
Jack5 (s) -0.10 -0.06 0.48 0.53 0.24 0.28
Mixture -0.10 1.31 4.22 18.51 17.81 344.34
Mixture (s) -0.39 -0.30 0.41 0.65 0.32 0.51
Sobs -0.55 -0.56 0.24 0.27 0.36 0.39

Table S2.6 Performance of species richness estimators as a function of survey effort, 100 (1%) and 500 (5%)
cells, averaged across all factor levels. Performances that include the substitutions described above are
identified with a (s). See Table 2 for estimator abbreviations and Table 3 for specific performance metrics.

Estimator 1% 5% 1% 5% 1% 5%
ACE -0.40 -0.24 0.30 0.19 0.25 0.09
Boot -0.62 -0.36 0.25 0.23 0.44 0.18
Chao1 -0.47 -0.27 0.26 0.20 0.29 0.11
Chao2 -0.47 -0.26 0.26 0.20 0.29 0.11
CY-1 -0.22 -0.15 0.37 0.21 0.19 0.07
CY-1 (s) -0.22 -0.15 0.37 0.21 0.19 0.07
CY-2 -0.21 -0.09 0.41 0.23 0.21 0.06
CY-2 (s) -0.21 -0.09 0.41 0.23 0.21 0.06
ICE -0.40 -0.24 0.30 0.19 0.25 0.09
Jack1 -0.54 -0.28 0.28 0.22 0.37 0.13
Jack2 -0.46 -0.20 0.31 0.21 0.30 0.09
Jack2 (s) -0.45 -0.19 0.31 0.22 0.30 0.09
Jack3 -0.40 -0.15 0.35 0.25 0.28 0.08
Jack3 (s) -0.38 -0.12 0.34 0.24 0.26 0.07
Jack4 -0.35 -0.10 0.43 0.34 0.31 0.13
Jack4 (s) -0.31 -0.05 0.41 0.29 0.26 0.09
Jack5 -0.30 -0.06 0.60 0.55 0.45 0.31
Jack5 (s) -0.23 0.05 0.53 0.43 0.33 0.19
Mixture 1.62 -0.33 18.91 0.19 360.29 0.15
Mixture (s) -0.36 -0.33 0.73 0.19 0.67 0.15
Sobs -0.68 -0.43 0.22 0.24 0.51 0.24
Table S2.7 Performance of species richness estimators as a function of survey design, random (Random) and
linear transect (Transect), averaged across all factor levels. Performances that include the substitutions
described above are identified with a (s). See Table 2 for estimator abbreviations and Table 3 for specific

Estimator Random Transect Random Transect Random Transect
ACE -0.32 -0.33 0.26 0.26 0.17 0.18
Boot -0.49 -0.49 0.27 0.27 0.31 0.32
Chao1 -0.37 -0.37 0.25 0.26 0.20 0.21
Chao2 -0.36 -0.37 0.25 0.26 0.20 0.21
CY-1 -0.18 -0.20 0.31 0.30 0.13 0.13
CY-1 (s) -0.18 -0.20 0.31 0.30 0.13 0.13
CY-2 -0.14 -0.16 0.34 0.33 0.14 0.14
CY-2 (s) -0.14 -0.16 0.34 0.33 0.14 0.14
ICE -0.32 -0.33 0.26 0.26 0.17 0.18
Jack1 -0.41 -0.41 0.28 0.28 0.24 0.25
Jack2 -0.33 -0.33 0.29 0.30 0.19 0.20
Jack2 (s) -0.32 -0.33 0.30 0.30 0.19 0.20
Jack3 -0.27 -0.27 0.32 0.33 0.18 0.18
Jack3 (s) -0.25 -0.26 0.32 0.33 0.17 0.17
Jack4 -0.22 -0.22 0.41 0.41 0.21 0.22
Jack4 (s) -0.18 -0.18 0.37 0.38 0.17 0.18
Jack5 -0.18 -0.18 0.59 0.59 0.38 0.38
Jack5 (s) -0.09 -0.09 0.50 0.51 0.26 0.26
Mixture 0.88 0.40 16.41 9.50 270.11 90.32
Mixture (s) -0.35 -0.35 0.52 0.55 0.39 0.43
Sobs -0.56 -0.56 0.26 0.26 0.38 0.38

Appendix S3: Case study of the performance of our proposed selection framework and the asymptotic
properties of estimators.
SURVEY DATA
To evaluate the performance of our estimator selection framework and the asymptotic performances of
estimators, we used field data from two studies. Fannes et al. (2008) surveyed two sites located in lowland
rainforests for arboreal spiders. They used insecticide knockdown fogging and suspended sheets to collect
specimens across 12 samples that were split between primary and secondary forests in Kakum National
Park, Ghana (November 2005) and across five samples within the primary forests of the Luki Biosphere
Reserve, Democratic Republic of the Congo (November 2006). King & Porter (2005) compared several
methods for sampling ants across five different ecosystems in north and central Florida. From June through
September 2001, they conducted surveys at three different sites in each ecosystem via a combination of
pitfall traps, baits, litter extraction, and hand collection.
There is always uncertainty about the true number of species (Strue) in a real assemblage. However, given
the asymptotic appearance of a randomized accumulation curve from the 12 samples of Kakum National
Park, Fannes et al. (2008) considered the 11 spider species a near census. The same was not true of the
accumulation curves that resulted from randomizations of the five samples of the Luki Biosphere Reserve
(see Fannes et al. 2008, their Fig. 4) and the 1,732 samples of ants in Floridian ecosystems (King & Porter,
2005, their Fig. 1). An accumulation curve that fails to reach an asymptote indicates that data collection is
incomplete, i.e. some species have not been detected (Gotelli & Colwell, 2001; Walther & Moore, 2005).
METHODS
We used randomization techniques when evaluating performances. At every sample size, we randomized
samples without replacement, 200x for the Fannes et al. (2008) datasets and 50x for the King & Porter
(2005) dataset, same as in the original studies. We applied species richness estimators to each randomized
subset and computed bias, precision, and accuracy, where possible, and averaged over all randomizations
at each sample size. We followed Fannes et al. (2008) in assuming that Strue = 11 for evaluating performance
against their data from Kakum National Park, Ghana. Despite possibly violating the assumption that the
observed number of species (Sobs) equaled Strue, we used Strue = 8 for evaluations against the data from Luki
Biosphere Reserve. For the dataset from King & Porter (2005), we assumed that Strue = 142, which is the
number of species found in historical datasets from the region (see Deyrup 2003). Some estimators, i.e. CY-
1, CY-2, and some orders of the jackknife, could not be computed at the smallest survey sizes (see Table 2
for estimator details, including abbreviations).
To evaluate estimator performance with the field data, we compared the rate at which randomized
accumulation curves reached their apparent asymptotes, i.e., near the asymptotic value of Sobs and
estimated richness when all data were used. Reaching an asymptote faster than Sobs is an important quality
for a SR estimator (Gotelli & Colwell, 2001).
RESULTS & DISCUSSION

Selection framework
Assuming that Kakum National Park contained 11 arboreal spider species (Fannes et al., 2008), the
randomization of two surveys contained on average more than 60% of the species pool, i.e. average sample
coverage (sc) was >0.60 (Fig. S3.1). Our selection framework (Table 4) correctly identified one of the two
least biased estimators, CY-1 (actual SME = 0.08), the most precise estimator, Sobs (SD = 0.14), and the
most accurate estimators, Jack1 and Jack2 (SMSE = 0.09). When five surveys were randomized, the
average sc was 0.84. Our framework incorrectly suggested ICE for bias (SME = 0.11) when it was Chao2
(SME = -0.05), but correctly selected Sobs for precision (SD = 0.11) and Boot for accuracy (SMSE = 0.02).
The randomization of two surveys from the Luki Biosphere Reserve dataset (Fannes et al., 2008) included
>80% of the species, if it is assumed that Strue = 8. Based on that assumption, our selection framework
correctly predicted the least biased, ICE (SME = -0.01), most precise, Sobs (SD = 0.12), and most accurate,
Boot (SMSE = 0.04), estimators (Fig. S3.2). Our selection framework did not perform as well when the
average sc equaled 0.95 (four surveys), predicting that Chao2 (SME = 0.07) would be least biased when it
was actually Boot (SME = 0.03). Chao1 (SME = -0.03) was, however, correctly predicted to perform nearly
as well. The most precise estimator was correctly predicted as Sobs (SD = 0.10). The first and third most
accurate estimators, Sobs (SMSE = 0.01) and Boot (SMSE = 0.02), respectively, were predicted in the
reverse order by our selection framework, though their performances differed minimally.
When we randomized 20, 100, and 400 samples from King & Porter (2005), Jack5 produced the largest
estimates, 54.8, 87.3, and 109.1, respectively. Assuming that Strue = 142, the average sc with 20, 100, and
400 samples equaled 0.15, 0.31, and 0.46, respectively. Our framework correctly selected Jack5 to reduce

bias with 100 (SME = -0.39) and 400 (SME = -0.23) surveys, but suggested using CY-1 (SME = -0.67) rather
than Jack5 (SME = -0.61) with 20 surveys. Our framework also correctly suggested using Jack3 (SME = -
0.09) for bias reduction when 1,648 surveys were randomized (given the assumed average sc, 0.65). At the
three smaller sample sizes, Sobs, Boot, and Jack1 were correctly identified as the first, second, and third most
precise estimators, respectively. With 1,648 surveys, CY-1 was slightly more precise than Jack1 (SD = 0.01
for both).
Thus far, we have evaluated the performance of our selection framework against values assumed for Strue,
information that is generally unavailable with field data. More important would be an estimate of sc that
correctly directs estimator selection without prior knowledge of Strue. We therefore used the data from Kakum
National Park (Fannes et al., 2008) to test the performance of the sc estimated by SimAssem (Reese et al.,
2013). In building the accumulation curve, the estimated sc ranged from 0.53 to nearly 0.70 with four and 10
randomized surveys, respectively. If we again assume that Strue = 11, then the estimated sc underestimated
the empirical sc which was 0.78 and 0.98 with four and 10 surveys, respectively. With that information, our
selection framework would suggest using Jack5 (with four surveys) and either Jack3 or CY-1 (with 10
surveys) to reduce bias. Since it requires at least five surveys, Jack5 could not be computed; the next
estimator recommended for bias reduction when the estimated sc = 0.53, Jack4, produced the second
largest average estimate (12.6) when four surveys were repeatedly randomized (the average CY-1 estimate
equaled 12.9), though several other estimates were closer to 11 (see Fig. S3.1, Jack4 data are not shown).
When 10 surveys were randomly selected, the average Jack3 estimate (11.4) was closer to 11 than all but
the average Sobs (10.8) and average Chao1 (11.1) estimates; furthermore, the average CY-1 estimate (13.3)
was the second largest (the average CY-2 estimate equaled 15.0).
Asymptotic performances
Fannes et al. (2008) considered the performances of the species richness estimators unsatisfactory, largely
because none of the associated accumulation curves reached an asymptote any faster than the curve for
Sobs. In addition to recreating accumulation curves for the originally compared estimators (ACE, Boot, Chao1,
Chao2, Jack1, and Jack2), we created curves for CY-1, CY-2, ICE, and Mixture (Fig. S3.1). The estimates
from CY-2, which required at least 10 surveys, appear as a line fragment in the top right corner. While it
remained true that none of the estimators used in their study reached an asymptote any faster than Sobs, the
average CY-1 estimate was 12.1 when two surveys were randomized (Sobs = 7 at that point) which is 110%
of an assumed Strue = 11, 94% of the CY-1 estimate (12.8) based on all the data, and possible evidence of
an asymptote. The ICE estimate was even larger with two surveys (15.4), but the continual decline in
estimates at larger samples sizes implies that it was an artifact of randomization, e.g. one or more surveys
had a relatively large number of uniques that would cause a large estimate when coupled with only one other
survey. The next largest estimate at two surveys was 92% of Strue (Chao2 = 10.2). Based on all of the data,
the Jack2 estimate was <Sobs and the Jack3, Jack4, and Jack5 estimates were 7.9, 4.6, and 1.0, respectively
(curves not shown).
When the Luki survey data were randomized, Sobs never reached an asymptote, but Fannes et al. (2008)
noted a possible leveling of the Chao1 estimator. In our randomizations, none of the estimators reached an
apparent asymptote as quickly as with the Kakum data and Chao2 and CY-1 changed the least between
samples sizes of four and five (Fig. S3.2). Had there been more data with which to evaluate performance,
these two estimators possibly would have shown that they had reached their respective asymptotes. As in
the simulations with Strue = 25 (see Table S2.1), these comparisons indicate that CY-1 could require less
data to reach an asymptote than for the other estimators.
SR estimators appeared not to approach an asymptote faster than Sobs with the ant dataset of King and
Porter (2005; their Fig. 1). Though not reproduced here, none of the additional estimators performed any
better. This could indicate that a considerable number of species were yet to be encountered.
CONCLUSION
Evaluating estimator selection frameworks and asymptotic properties with field data is difficult because there
is rarely, if ever, an assurance that the evaluation is based on correct information. However, we believe that
the tests presented here demonstrate the promise of selection frameworks such as those proposed in Brose
et al. (2003) and in this study, particularly when the sample coverage is correctly estimated. Tests of our
proposed sample coverage estimator indicate that it underestimates the true sample coverage, but again,
assumptions that Sobs = Strue in the test datasets could have been false. For example, several estimators
indicated that Strue > 11 in Kakum National Park, which could be true given the difficulty in detecting all
species and evidence of the difficulty in evaluating the asymptotic properties of estimators with field data.
Thus, our study shows that additional development and testing of species richness estimator is warranted.

Figure S3.1 Species accumulation curves for 10 species richness estimators based on arboreal spider data
from Kakum National Park, Ghana. At each possible survey size (12 total), 200 surveys were randomly
drawn without replacement, species richness was estimated, and estimates were averaged across the 200
randomizations. The x-axis is the average number of individuals contained in randomized surveys (each
successive symbol on a line represents an increment of one in the number of randomized surveys).
Figure S3.2 Species accumulation curves for eight species richness estimators based on arboreal spider
data from Luki Biosphere Reserve, DR Congo. At each possible survey size (five total), 200 surveys were
randomly drawn without replacement, species richness was estimated, and estimates were averaged across
the 200 randomizations. The x-axis is the average number of individuals contained in randomized surveys
(each successive symbol on a line represents an increment of one in the number of randomized surveys). To
minimize clutter, Jack3 (the estimate based on all data equaled 10.7), Jack4 (10.8), Jack5 (10.8), and ICE
(9.2) are not displayed and, with only five surveys, there were not enough data to compute CY-2.

REFERENCES
Brose, U., Martinez, N.D. & Williams, R.J. (2003) Estimating species richness: sensitivity to sample coverage
and insensitivity to spatial patterns. Ecology, 84, 2364-2377.
Deyrup, M. (2003) An updated list of Florida ants (Hymenoptera: Formicidae). Florida Entomologist, 86, 43-
48.
Fannes, W., Bakker, D.D., Loosveldt, K. & Jocqué, R. (2008) Estimating the diversity of arboreal oonopid
spider assemblages (Araneae, Oonopidae) at Afrotropical sites. The Journal of Arachnology, 36,
322-330.
Gotelli, N.J. & Colwell, R.K. (2001) Quantifying biodiversity: procedures and pitfalls in the measurement and
comparison of species richness. Ecology Letters, 4, 379-391.
King, J.R. & Porter, S.D. (2005) Evaluation of sampling methods and species richness estimators for ants in
upland ecosystems in Florida. Environmental Entomology, 34, 1566-1578.
Reese, G.C. (2012) Simulating species assemblages and evaluating species richness estimators. PhD
thesis, Colorado State University, Fort Collins. Available at: http://hdl.handle.net/10217/68193
(accessed 23 October 2013).
Reese, G.C., Wilson, K.R. & Flather, C.H. (2013) Program SimAssem: software for simulating species
assemblages and estimating species richness. Methods in Ecology and Evolution, 4, 891-896.
Walther, B.A. & Moore, J.L. (2005) The concepts of bias, precision and accuracy, and their use in testing the
performance of species richness estimators, with a literature review of estimator performance.
Ecography, 28, 815-829.

Performance of Species Richness Estimators Across Assemblage Types and Survey Parameters Reese 2014

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Performance of Species Richness Estimators Across Assemblage Types and Survey Parameters Reese 2014

Uploaded by

Copyright:

Available Formats

Global Ecology and Biogeography, (Global Ecol. Biogeogr.

) (2014) 23, 585–594

RESEARCH Performance of species richness

Attempts to quantify species diversity often include: (1) a count

© 2014 John Wiley & Sons Ltd DOI: 10.1111/geb.12144

Table 2 Evaluated species richness

Abundance-based coverage*† ACE Chao & Lee (1992)

*Estimation by modelling heterogeneity in detection probability of species.

cept of a regression line fitted to SR versus JC, where each value 2

Table 4 The three best performing

and biased estimator can lead to a false sense of confidence when

Bias (SME) Precision (SD) Accuracy (SMSE)

Table S1, Reese et al., Performance of species richness estimators, Pg. 1

Table S2, Reese et al., Performance of species richness estimators, Pg. 1

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S1, Reese et al., Performance of species richness estimators, Pg. 1

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S1, Reese et al., Performance of species richness estimators, Pg. 2

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S1, Reese et al., Performance of species richness estimators, Pg. 3

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S1, Reese et al., Performance of species richness estimators, Pg. 4

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S2, Reese et al., Performance of species richness estimators, Pg. 1

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S2, Reese et al., Performance of species richness estimators, Pg. 2

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S2, Reese et al., Performance of species richness estimators, Pg. 3

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S2, Reese et al., Performance of species richness estimators, Pg. 4

Bias (SME) Precision (SD) Accuracy (SMSE)

Bias (SME) Precision (SD) Accuracy (SMSE)

Appendix S2, Reese et al., Performance of species richness estimators, Pg. 5

RESULTS & DISCUSSION

Appendix S3, Reese et al., Performance of species richness estimators, Pg. 1

Appendix S3, Reese et al., Performance of species richness estimators, Pg. 2

Appendix S3, Reese et al., Performance of species richness estimators, Pg. 3

Appendix S3, Reese et al., Performance of species richness estimators, Pg. 4

You might also like