See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/358731054

Using environmental niche models to elucidate drivers of the American

bullfrog invasion in California

Article  in  Biological Invasions · June 2022

DOI: 10.1007/s10530-022-02744-3

CITATIONS READS

0 244

2 authors, including:

Nicolette Nelson
University of Connecticut
7 PUBLICATIONS   9 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Nicolette Nelson on 21 February 2022.

The user has requested enhancement of the downloaded file.

Biol Invasions
https://doi.org/10.1007/s10530-022-02744-3
ORIGINAL PAPER
Using environmental niche models to elucidate drivers
of the American bullfrog invasion in California
Nicolette Nelson   · Jonah Piovia‑Scott 
Received: 25 April 2021 / Accepted: 27 January 2022
© The Author(s), under exclusive licence to Springer Nature Switzerland AG 2022
Abstract  Environmental niche models (ENMs)
are commonly used to inform management of inva-
sive species, but invasive species often violate  two
key  assumptions  of  ENMs which assert that spe-
cies’ ecological niches are stable in space and time,
and that organisms are in  equilibrium  with their
environment. The American bullfrog (Rana cates-
beiana) invasion in California provides an excel-
lent opportunity to use ENM approaches recom-
mended for overcoming these assumption violations
to inform management of a harmful invader. In  the
current study,  we used dynamic ENM approaches
and explicit comparisons between native and invaded
niches to: (1) examine how environmental drivers of
occurrence in California shifted during the invasion
process, and (2) investigate the potential for contin-
ued bullfrog expansion by explicitly assessing their
environmental equilibrium. Since their initial intro-
duction to metropolitan areas of California in the late
1800s, bullfrogs have spread throughout the state
with the most notable recent expansions in coastal
regions, the Central Valley, and the mountains of
northern California. Bullfrog occurrence in California
N. Nelson (*) · J. Piovia‑Scott 
School of Biological Sciences, Washington State
University, Vancouver, WA, USA
e-mail: nicolette.nelson@uconn.edu
N. Nelson 
Natural Resources and the Environment, University
of Connecticut, Storrs, CT, USA
was consistently associated with human activity and
higher winter temperature. Small pockets of high
human activity have sustained bullfrog populations
in otherwise unsuitable areas. The areas in California
with the highest predicted risk of increased bullfrog
abundance and future bullfrog expansion include the
southern Central Valley, the coast of northern Cali-
fornia, and mid elevations in the mountains of north-
ern California. Continued bullfrog invasion in these
areas is likely to impact sensitive native amphibian
populations.
Keywords  Environmental equilibrium · Species
distribution model · Dynamic environmental niche
model · Spatial spread · Human footprint
Introduction
In order for an invasive species to successfully spread
to new areas, the new environment must be acces-
sible and contain the biotic and abiotic conditions
required for positive population growth (Soberón and
Peterson 2005; Soberón 2007; Jiménez-Valverde et al.
2011). Invasive species are often able to disperse far-
ther and faster than native species, meaning that they
can access potentially suitable habitats quickly (With
2002; Didham et al. 2007). In addition, invasives tend
to be generalists and are often released from their
native predators and competitors, so they are usually
less limited by biotic interactions than native species
Vol.: (0123456789)
13

(Ehrlich 1989; Arim et  al. 2006; Allen et  al. 2013).
For these reasons, invasive species can often track
or adapt to changing environmental conditions bet-
ter than native species (Ehrlich 1989; Whitney and
Gabler 2008).
The relationship between environmental conditions
and invasive species spread is typically examined with
environmental niche models (ENMs), which estimate
a species’ niche from  readily available  data on eco-
logical conditions and  species occurrence (Yalcin
et al. 2017; Thornton and Peers 2019). One potential
issue with using these models is that invasive species
often violate two key assumptions of ENMs: (1) spe-
cies’ ecological niches are stable in space and time
(Gallien et  al. 2012), i.e., they occupy habitats with
similar environmental conditions in the native and
invasive portions of their range, and (2) organisms are
in  equilibrium  with their environment (Václavík and
Meentemeyer 2012), i.e., they have reached all places
with suitable habitat and are absent from all unsuita-
ble areas (Guisan and Thuiller 2005). Invasive species
rarely meet these assumptions because adaptations
acquired  during the invasion process might allow
them to occupy new environments (Broennimann
and Guisan 2008; Jiménez-Valverde et  al. 2011) and
because they may not have had enough time to reach
all suitable environments (Petitpierre et al. 2012).
Several approaches have been developed to
overcome the violation of the environmental equi-
librium assumption when building ENMs for inva-
sive species (Barbet-Massin et  al. 2018; Srivastava
et  al. 2019). For example, the predictive power of
an  ENM  is typically evaluated by determining
how well it predicts the occurrence of points ran-
domly withheld from the original dataset, but these
types of metrics (e.g.,  cross-validation) may not
be appropriate for invasive species that are not in
equilibrium with their environments (Hattab et  al.
2017). Instead, occurrences from  a  later stage of
invasion can be used to assess the predictive power
of ENMs (dynamic environmental niche modeling;
Ficetola et al. 2010). This dynamic ENM approach
can also be used to examine whether relationships
between environmental conditions and invasive spe-
cies occurrence have shifted during the invasion
process. In addition, ENMs for invasive species can
be fit with occurrence data from both the native and
invaded  portions of the range to overcome issues
with environmental disequilibrium  by capturing
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
all potentially suitable environmental conditions
(Broennimann and Guisan 2008). However, a global
description of the niche cannot capture specificities
of a regional invaded niche or explicitly examine the
degree to which invasive species are in equilibrium
with their environment (Gallien et al. 2012). Explic-
itly comparing the native and regional invaded
niches can elucidate the degree to which invasive
species are in equilibrium with their environments
and also provide insight into the process of niche
differentiation during invasion (Gallien et al. 2012).
American bullfrogs  (Rana  catesbeiana, hereaf-
ter “bullfrogs”) are considered one of the world’s
most harmful invasive species due to their rapid
global expansion and impacts  on  native species
(Lowe et  al. 2000). They are native to the eastern
half of North America but have been introduced to
other parts of the continent and the world through
anthropogenic activities. The bullfrog invasion in
California provides an excellent opportunity to
examine the invasion history and invasion stages of
a harmful species using ENM approaches recom-
mended for modeling invasive species spread. The
invasion has been well documented since the initial
introduction of bullfrogs to California in the late
1800s (McKercher and Gregoire 2021), the inva-
sive distribution is thought to still be expanding,
and their native and invaded niches may differ (Fer-
nandez and Hamilton 2015; Bissattini and Vignoli
2017). However,  areas  in the state  that are poten-
tially suitable to  bullfrogs  but not yet occupied are
unknown. Recent studies have described bullfrog
invasion  history  in the western United States but
have not used quantitative modeling to elucidate the
environmental drivers underlying their expansion
(Yap et  al. 2018; McKercher and Gregoire 2021).
Bullfrog  management would be aided by updated
information on the patterns underlying their historic
spread in California, the contribution of different
environmental factors to both contemporary and
historic bullfrog spread, and an examination of their
environmental equilibrium. In the current study, we
used dynamic ENM approaches and explicit com-
parisons between native and invaded niches to: (1)
examine how environmental drivers of occurrence
in the invaded range shifted during the invasion
process, and (2) investigate the potential for contin-
ued bullfrog expansion by explicitly assessing their
environmental equilibrium.
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…

Methods gov 2021; McKercher and Gregoire 2021; Table  1).

Background
Bullfrogs were initially introduced to California in the
late 1800s to provide sustenance for California’s rap-
idly expanding population following the Gold Rush
(Jennings and Hayes 1985). Since then, humans have
facilitated their spread by repeatedly transporting
them to new areas and by creating habitats favorable
to bullfrog persistence (Peterson et al. 2013). In Cali-
fornia, bullfrogs have expanded from metropolitan
areas to the rest of the state but appear to be absent
from high elevation habitats (McKercher and Gre-
goire 2021). Bullfrogs have contributed to the decline
of several sensitive amphibians in the state (e.g.,
Kupferberg 1997; Yap et al. 2018) and they have the
potential to impact several additional threatened and
endangered amphibians if they expand to habitats that
they do not currently occupy (Moyle 1973; Lawler
et al. 1999; Miaud et al. 2016; Adams 2017).
Bullfrog occurrence data
Bullfrog  occurrence data (species name  = 
“Litho-
bates catesbeianus” or “Rana catesbeiana”) span-
ning the continental United States were downloaded
from GBIF, BISON, and the USGS Nonindigenous
Aquatic Species Database (GBIF 2020; BISON.usgs.
Additional occurrence data from California were
obtained from the California Department of Fish and
Wildlife (Perry 2020). Observations missing geo-
graphic coordinates and temporal information were
removed from the dataset. The positional uncertainty
of species occurrence records can substantially reduce
the accuracy of ENMs (Thornton and Peers 2019), so
occurrences recorded with GPS accuracy greater than
the resolution of environmental data (6 ­km2 for his-
toric models and 5 k­ m2 for contemporary models; see
below for details) were also removed.
Environmental variables
Important factors to consider when modeling bullfrog
invasion  are  habitat availability, physiological lim-
its, and the timing of juvenile emigration.  Bullfrogs
are highly aquatic with a multiple-year tadpole stage,
so the availability of permanent waterbodies is likely
the most important factor contributing to their local
persistence (Peterson et al. 2013; Murray et al. 2015).
High mortality has been observed during sudden win-
ter freezes (Willis et al. 1956), but hibernation in deep
permanent waterbodies can buffer these effects (Stew-
art et  al. 2004; Sepulveda and Layhee 2015). Juve-
niles tend to emigrate from breeding ponds  during
warm rain events in the late summer and fall (Bury
and Whelan 1984; Morey 1988). Anthropogenic

Table 1  Variables and data sources used to determine the key drivers of historic  American bullfrog  occurrence  in California,
USA. Some predictor variables were removed from analyses due to high correlations with other predictors
Variable Mechanism of effect on bullfrog occurrence Predicted effect on Data source Resolution1
bullfrog occurrence

Bullfrog occurrence Bullfrog occurrence GBIF

BISON
USGS NAS
CDFW
Elevation Physiological limits (e.g., UV limitation) Negative PRISM 5 ­km2
Minimum winter temperature Physiological limits (winter freezing) Negative PRISM 5 k­ m2
Proximity to waterbodies Habitat availability Positive NHD
Crop cover Historic habitat availability (anthropogenic habitat Positive HYDE 6 k­ m2
modifications)
Human footprint index Recent habitat availability (anthropogenic habitat Positive Dryad 1 ­km2
modifications)
Fall precipitation Juvenile emigration Positive PRISM 5 ­km2
Maximum fall temperature Juvenile emigration Positive PRISM 5 ­km2
1
  Resolution of data before resampling for ENMs

Vol.: (0123456789)
13

activity is likely to increase the rate of bullfrog expan-
sion (i.e., Ficetola et  al. 2010) because bullfrogs are
frequently transported by humans, are often actively
farmed, and because naturalized populations often
perform well in human-altered landscapes (Peterson
et al. 2013; Herrel and van der Meijden 2014), espe-
cially in areas with abundant artificial lentic habitats
(D’Amore et al. 2009).
Elevation is another factor that could be linked to
bullfrogs’ physiological limits, habitat availability,
and movement. Topographic complexity (i.e., eleva-
tion change per unit area) and winter freezing may
limit bullfrog colonization of otherwise suitable
habitats at high elevations in the Sierra Nevada and
southern Cascades (Peterson et  al. 2013; Sepulveda
and Layhee 2015). In addition, larval survival may be
impacted by UV-B radiation, which tends to increase
in intensity with elevation (Garcia et  al. 2015),
though the hypothesis that UV-B radiation impacts
amphibian survival at high elevations has come under
scrutiny (e.g., Palen and Schindler 2010). Alterna-
tively, bullfrog habitat suitability may increase with
elevation, especially in the Sierra Nevada, due to the
prevalence of deep glacial lakes (USGS 2004) and
snowpack that provides insulation from winter freez-
ing and sustains aquatic habitats during the summer
(Rhoades et al. 2016).
The availability of bullfrog habitat was represented
by geodesic distance to the nearest permanent water-
body  (lakes, streams, swamps, marshes, reservoirs,
and canals, were included; dry lakes, intermittent
lakes, and glaciers were excluded) from the National
Hydrography Dataset (USGS 2004; Table  1); all of
the included habitat types have been associated with
bullfrogs (Bury and Whelan 1984; Morey 1988;
Conant and Collins 1991; Stebbins 2003). Man-made
habitats favorable to bullfrogs (i.e., irrigation ditches
and canals that may not have been captured in the
USGS dataset) in historic time periods (1896–1945,
1946–1980, and 1981–2000; see description of time
periods below) were represented by the percent cover
of cropland in each grid cell (i.e., Ficetola et al. 2010)
from the History Database of the Global Environ-
ment (Goldewijk et  al. 2011; Table  1). This data-
set does not represent recent conditions (after 2005)
or non-agricultural man-made habitats. Therefore,
in the most recent time period (2001–2020), man-
made habitats were represented by the Global Human
Footprint Index (Venter et  al. 2016) which includes
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
croplands as well as built environments, human popu-
lation density, electric infrastructure, pasture lands,
roads, railways, and navigable waterways. For eleva-
tion, we used a digital elevation raster downloaded
from PRISM (PRISM Climate Group 2020; Table 1).
Winter freezing was captured by monthly minimum
temperature during the typical bullfrog hibernation
period (October–March)  downloaded from PRISM
(PRISM Climate Group 2020). Climatic conditions
affecting juvenile emigration were represented by
monthly fall (July–October) precipitation and maxi-
mum temperature data downloaded from PRISM
(PRISM Climate Group 2020).
Environmental niche models
Environmental niche models (ENMs) were built to
determine the key environmental drivers of historic
bullfrog spread in California. Individual models were
built for discrete historic time periods using hypoth-
esized environmental drivers related to bullfrog habi-
tat availability, physiological limits, and the timing
of juvenile emigration. Several metrics were used
to determine which drivers contributed to bullfrog
occurrence in each time period.
Data  were binned into discrete time periods for
analysis.  To ensure that ENMs accurately reflected
shifts in historic climate conditions, cutoff points
for time periods roughly corresponded to shifts in
climatic regime associated with the Pacific Decadal
Oscillation (Newman et al. 2016). Due to low sam-
ple size and potential sampling bias in early bull-
frog records, the first two  Pacific Decadal Oscil-
lation regimes (1896–1925 and 1926–1945)  were
combined, so ENMs were built for the time peri-
ods: 1896–1945, 1946–1980, 1981–2000, and
2001–2020.  Climate and cropland data were aver-
aged over the time periods, restricted to the state
of California, and then resampled to the spatial
resolution of the variable with the coarsest resolu-
tion in each dataset (6 ­km2 for historic models and
5 ­km2 for contemporary models) using bilinear
interpolation with the projectRaster function in the
raster R package (Hijmans and van Etten 2012).
Environmental predictors were also standardized
for comparability using the normImage function
in the R package RStoolbox (Leutner et  al. 2019),
which for each pixel of a raster layer  subtracts the
mean value of the layer and divides by the standard
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…
deviation. Prior to building ENMs, bullfrog occur-
rences representing each time period were spatially
thinned to one occurrence point per grid cell of
environmental data to avoid potential issues with
oversampling (Fourcade et al. 2014).
Correlations between potential predictor vari-
ables were assessed using Spearman rank cor-
relations for predictor values extracted at bull-
frog observations plus 10,000 randomly generated
points for each time period. Relatively high cor-
relations occurred between elevation and winter
temperature (r1896-1945 =  − 0.90, r1946-1980 = − 0.89,
r1981-2000 =  − 0.88, r2001-2020 =  − 0.85), elevation
and fall temperature (r1896-1945 =  − 0.72, r1946-1980
= − 0.70, r1981-2000 =  − 0.70, r2001-2020 =  − 0.63), and
winter temperature and fall temperature (r1896-1945 =
0.72, r1946-1980 = 0.73, r1981-2000 = 0.74, r2001-2020 =
0.66). In order to avoid multicollinearity, we removed
elevation and fall temperature from our models.
In other words, ENMs were built with the follow-
ing hypothesized environmental drivers of bullfrog
spread: crop cover or human footprint, distance to the
nearest waterbody, minimum winter temperature, and
mean fall precipitation.
ENMs were fit for each time period using
the  sdm  R package (Naimi and Araujo 2016), with
10,000 pseudo-absence points randomly generated
from the  study area, the state of California. Models
were run using the generalized linear model (GLM)
and maximum likelihood methods, with the full
model structure for each time period (t):
Presence∕pseudo − absencet ∼ 𝛽0 + 𝛽1 (crop covert or human footprintt )
( )
+ 𝛽2 distance towaterbodiest + 𝛽3 min winter tempt
( )
+ 𝛽4 mean fallprecipt + Sampling effort offset + 𝜀t ,
where β represented the relationship between the
explanatory variables and the presence or absence of
bullfrogs on the scale of the link function and ε was
the binomial error term. Models also included an
offset term to account for sampling effort; the offset
is a predictor whose coefficient is assumed to be 1
(i.e., is not estimated by the model). Sampling effort
was estimated by downloading all available amphib-
ian records from GBIF (GBIF 2020) for the corre-
sponding time period, removing bullfrog records,
and converting occurrences to a grid with the same
spatial resolution as environmental data used to build
the models representing the count of non-bullfrog
amphibians in each grid cell.
Models were evaluated  during model building by
calculating the area under the curve (AUC) and sen-
sitivity with five-fold cross-validation. During cross-
validation, occurrence points were split into five
roughly equal-sized portions, and then models were
fit five times. For each model run, four portions of
the occurrence points were used as training data and
the remaining portion was used as test data (Naimi
and Araujo 2016). Sensitivity requires a threshold to
classify predicted suitability as predicted presence or
absence, which was optimized using the 10th percen-
tile threshold, defined as the suitability value below
which 10 percent of bullfrog records fell. We chose
this threshold because it does not rely on specificity
(the probability of correctly classifying absences),
which is not suitable for evaluating presence-back-
ground models where true absences are unknown.
Each models’ ability to predict future invasion was
evaluated by calculating the AUC, sensitivity, and
omission and commission rates. These metrics were
calculated using dynamic environmental niche mod-
eling, where occurrence records from the time period
in question were used as training data and occurrence
points from the subsequent time period were used
as test data (Runquist et  al. 2019). Because AUC is
heavily influenced by model parameterization deci-
sions (Lobo et  al. 2008), AUC generated by both
evaluation procedures was used to compare perfor-
mance between models rather than as an absolute
( )
[ ]
metric of model performance.
The key drivers of historic  bullfrog  occurrence
were determined by examining the AUC and cor-
relation test metrics of variable importance for each
potential environmental driver, extracted from mod-
els for each time period using the getVarImp function
in the sdm R package (Naimi and Araujo 2016). The
AUC method used a cross-validation procedure to
compare model performance when each variable was
included or excluded (Naimi and Araujo 2016). The
correlation test method used a Pearson correlation
between standard model predictions (fitted values)
Vol.: (0123456789)
13

and predictions generated from random permutation
of each variable (Thuiller et  al. 2009). In this case,
low correlation between the predictions indicated
high variable importance. The direction of environ-
mental relationships was determined by examining
the beta coefficients of each potential environmental
driver, extracted from models for each time period
using the sdm R package (Naimi and Araujo 2016).
Assessment of environmental equilibrium
In order to assess the environmental equilibrium
of  bullfrogs  in California, predictions from the
California ENM for the most recent time period
(2001–2020) were compared with predictions from
an ENM representing the native range niche for the
same time period.  The  native  range ENM was built
from  bullfrog  occurrence records and  environmen-
tal data  representing  the eastern United States,  with
the same model specifications and evaluation proce-
dures described above. The study area extent for the
native range model was derived  by adding a buffer
around the native range of bullfrogs (IUCN 2020;
Fig.  2) and increasing the buffer size until gaps in
the middle of the eastern US were filled in (110 km
buffer distance).  100,000 pseudo-absence points
were randomly generated from the study area. The
native range ENM was built with the same hypoth-
esized environmental drivers of bullfrog spread as the
California models (correlations between predictors
did not exceed 0.7): human footprint, distance to the
nearest waterbody, minimum winter temperature, and
mean fall precipitation. For the California and native
range ENMs, ensemble models were created by tak-
ing the mean of predictions from each of the two
modeling methods (GLM and maximum likelihood).
Predicted suitability from the native range ENM was
then projected onto current (2001–2020) environmen-
tal conditions in California.
The native and invaded niches occupied by bull-
frogs in California were compared following the
niche comparison framework outlined by  Gallien
et  al. (2012). Three pairs of data were compared in
geographical space in California:  (1) the observed
presences against the California model predic-
tions;  (2) the observed presences against the  native
range model predictions; and (3) the invaded against
the  native  model predictions  at the observed pres-
ences. These comparisons were used  as proxies to
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
assess: (1) how well bullfrogs currently fill their pro-
jected invaded niche in California; (2) how well bull-
frogs currently fill their projected native niche in Cal-
ifornia; and (3) how (dis-)similar are the invaded and
the native niche projections.  Niche filling was deter-
mined by  calculating the proportion of bullfrog
occurrences that  fell within areas of predicted pres-
ence from each ENM. The threshold used to bin
predicted suitability from each model into suitable
or not suitable  for bullfrogs was defined as the ­10th
percentile, the suitability value from each ensemble
model below which below which 10 percent of bull-
frog records fell. Niche (dis)similarity was assessed
by  comparing the predicted suitability from the
native niche model and the California (invaded) niche
model. Bullfrog  occurrences in California  were then
assigned to the following categories: at equilibrium
(suitable based on the native and invaded niches),
colonizing (suitable based on the native niche, unsuit-
able based on the invaded niche), adapting (unsuit-
able based on the native niche, suitable based on the
invaded niche), or unsuitable (unsuitable based on the
native and invaded niches).
Uncertainty in ENM predictions for the contem-
porary native and invaded range models was deter-
mined by calculating the model-based inconsistency
among different model runs for each ENM using
the “entropy” method in the ensemble function in
the sdm R package (Naimi and Araujo 2016), where
a value of 0 indicated minimum uncertainty (all the
models predicted the same suitability value) and
a value of 1 indicated maximum uncertainty (half
of the models predicted low or high suitability and
the other half predicted the opposite value). A mul-
tivariate environmental similarity surface (MESS)
was used to determine areas where the native niche
ENM extrapolated outside the range of values in the
training dataset, indicating potentially poor predictive
power (Elith et al. 2010). The MESS was built using
the mess function in the dismo R package (Hijmans
et al. 2011).
Results
Patterns of historic bullfrog spread
We assembled a dataset of 10,284 bullfrog occur-
rences in California (1,379 after spatial thinning).
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…
The number of American bullfrog occurrences
recorded in California has increased roughly expo-
nentially since their initial introduction to the state,
with a large increase in the number of occurrences
between 2010 and 2020 (Fig.  1). Most of the earli-
est bullfrog occurrences were recorded in the Bay
Area, Central Valley, and within the Los Angeles
metropolitan area (Fig. 2). Since then, bullfrogs have
expanded in a diffuse pattern to other parts of the
state. The areas with the most notable recent expan-
sions are most coastal regions (especially those adja-
cent to metropolitan areas), the Central Valley, and
the mountains of northern California, including parts
of the Sierra Nevada, Cascades, Klamath, and Coast
Ranges (Fig. 2).
Environmental drivers of historic bullfrog spread
ENMs  built to determine the key drivers of his-
toric bullfrog occurrence  had AUC values rang-
ing from 0.78—0.86 (Table  2). Based on the ­10th
percentile threshold, models correctly classified
bullfrog presences roughly 90% of the time (sensi-
tivity = 0.89–0.90), with little variation around the
Fig. 1  Cumulative proportion of observations used to examine historic American bullfrog spread in California
metric between different cross-validation runs for
each modeling method (Table  2). Under dynamic
evaluation, models accurately predicted future bull-
frog presences with low rates of Type I and Type II
errors (AUC = 0.78–0.82, sensitivity = 0.90, omis-
sion = 0.10, commission = 0.0010–0.0042), with little
variation around metrics between different modeling
methods (Table 3).
Based on the correlation test and area under the
curve (AUC) measures of variable importance, winter
temperature and human activity were the most con-
sistent predictors of bullfrog occurrence in California.
Winter temperature was the best predictor of bullfrog
occurrence in the time periods 1946–1980 (AUC
metric), 1981–2000 (both metrics), and 2001–2020
(AUC metric; Fig. 3), and beta coefficients showed a
positive relationship between winter temperature and
bullfrog occurrence (Fig. 3). Croplands were the best
predictor of bullfrog occurrence in the time periods
1896–1945 (both metrics) and 1946–1980 (correla-
tion test metric; Fig. 3), and bullfrog occurrence was
positively associated with croplands (Fig.  3). In the
most recent time period (2001–2020), where human
footprint was used as a metric of human activity
Vol.: (0123456789)
13

Fig. 2  Bullfrog occurrences and study areas used to exam-
ine the spatial patterns underlying historic American bullfrog
spread in California. Bullfrog occurrences representing each
time period were spatially thinned to one occurrence point
per grid cell of environmental data (6 k­m2 for 1986–1945,
instead of croplands, human footprint contributed
positively to bullfrog occurrence (correlation test
metric; Fig. 3). Fall precipitation and distance to the
nearest waterbody consistently performed poorly as
predictors of bullfrog occurrence in California (both
metrics); fall precipitation was positively associated
with bullfrog occurrence and distance to the nearest
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
1946–1980, and 1981–2000; 5 k­ m2 for 2001–2020). The native
range study area extent was derived by adding a 110 km buffer
around the native range of bullfrogs, obtained from IUCN
(IUCN 2020). Bullfrog occurrences in the native range repre-
sent the time period 2001–2020
waterbody was negatively associated with bullfrog
occurrence (Fig. 3).
Assessment of environmental equilibrium
We assembled a dataset of 34,283 bullfrog occur-
rences representing the native range of bullfrogs
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…

Table 2  Performance metrics for ENMs used to determine the key drivers of historic American bullfrog occurrence in California,
and in their native range
Time period AUC​ AUC SE Threshold Threshold SE Sensitivity Sensitivity SE

1896–1945 0.85 0.013 0.044 0.013 0.89 0

1946–1980 0.8 0.0084 0.064 0.018 0.90 0
1981–2000 0.78 0.0053 0.13 0.038 0.90 7.2 × ­10–5
2001–2020 0.86 0.004 0.088 0.013 0.90 2.4 × ­10–5
2001–2020 - native 0.79 0.0027 0.051 0.0096 0.90 4.5 × ­10–6
range

Models were evaluated using occurrence data from the time period for which the model was fit. The mean and standard error (SE)
were calculated using values from all cross-validation runs from both modeling methods (GLM and maximum likelihood) for each
time period. AUC​ area under the curve

Table 3  Performance metrics used to determine how well dynamic ENMs predicted American bullfrog occurrence in the following
time period in California
Time period AUC​ Threshold Sensitivity Omission Commission

1896–1945 0.82 0.041 0.90 0.10 0.0010

1946–1980 0.78 0.064 0.90 0.10 0.0042
1981–2000 0.82 0.13 0.90 0.10 0.0041

Models were built using bullfrog occurrences and environmental conditions for the time period listed and were tested using bullfrog
occurrence points for the subsequent time period. The value for each metric represents the mean value from two modeling methods
(GLM and maximum likelihood) for each time period. AUC​ area under the curve

(4,562 points after spatial thinning). The native range
ENM representing recent (2000–2020) conditions had
an AUC of 0.79; based on the 10th percentile thresh-
old, the native range model correctly classified bull-
frog presences 90% of the time (sensitivity = 0.90),
with little variation around the metrics between dif-
ferent cross-validation runs for each modeling method
(Table 2). Bullfrog occurrence in the native range was
strongly associated with human footprint (both met-
rics; Fig. 3). There was greater uncertainty in predic-
tions from the native niche model than the California
niche model (Fig.  4A). The areas of greatest uncer-
tainty in the California niche model predictions were
mid elevations of mountainous regions (Fig.  4A),
whereas the areas with the greatest uncertainty in the
native niche model predictions were mid to high ele-
vations of the Sierra Nevada, Klamath, and Northern
Coast mountain ranges (Fig.  4A). Based on a multi-
variate environmental similarity surface (MESS),
environmental conditions in California diverged the
most from conditions in the native range of bullfrogs
within the northern Klamath range, with additional
small areas of divergence scattered throughout the
state (Fig. 4B).
Most bullfrog occurrences in California (86% of
observations) occurred in suitable conditions based
on both the California and native range ENMs; these
observations mainly occurred along the coast of
California and in the Central Valley, throughout the
areas predicted as suitable by both models (Fig. 5).
A small proportion of observations occurred in suit-
able conditions based on the native range ENM but
not the California ENM (4% of observations); these
were mainly at mid to high elevations in the north-
ern Sierra Nevada, Cascades Range, and Klamath
Mountains. A small proportion of observations
occurred in suitable conditions based on the Cali-
fornia ENM but not the native range ENM (4% of
observations); these appeared in small patches in
the southeastern portion of the state and at mid
elevations in coastal mountains. Observations also
occurred in areas predicted to be unsuitable by both
California and native range ENMS (6%). These
were dispersed throughout the large portion of the
state that fell into this category, including high ele-
vations in the Klamath and coastal ranges, mid to
high elevations in the Sierra Nevada and southern
Cascades, and the Mojave Desert.

Vol.: (0123456789)
13

Fig. 3  Two metrics of variable importance (correlation test
metric and contribution to model AUC) and beta coefficients
estimated with ENMs of  historic and contemporary  Ameri-
can bullfrog  occurrence in California. The spatial resolution
of ENMs matched the coarsest available environmental driver
Discussion
Since their initial introduction to urbanized parts of
California in the late 1800s, bullfrogs have spread
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
for each model (6 ­km2, based on HYDE cropland data for his-
toric models, and 5 k­ m2, based on PRISM climate variables for
contemporary models). Points represent average values from
model runs for both modeling methods (GLM and maximum
likelihood) for each time period
throughout the state. In recent years they have
moved into coastal and montane areas, while the
southern deserts and the higher elevation parts of
the southern Sierra Nevada have remained sparsely
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…

Fig.  4  A Uncertainty in

predicted suitability from
ENMs describing the
invaded (California-based)
and native niches occupied
by American bullfrogs in
California during the time
period 2001–2020. High
uncertainty (values closer to
1) indicates low agreement
between model runs and
low uncertainty (values
closer to 0) indicates
high agreement between
model runs. B Multivariate
Environmental Similarity
Surface (MESS) comparing
the values of environmental
variables hypothesized to
be important drivers of
America bullfrog spread
between the eastern United
States (native range) and
California (invaded range).
Positive MESS values indi-
cate similarity in conditions
between the two ranges and
negative values indicate
dissimilarity

occupied. Our ENMs suggest that bullfrogs were
closely associated with human-modified landscapes
and relatively high winter temperature throughout
the invasion. Dynamic ENMs provided reasonable
predictions for bullfrog suitability over the course
of the invasion, suggesting that this study contrib-
utes to existing literature that shows the utility of
dynamic ENMs for characterizing the potential
expansion of invasive species. Explicitly comparing
bullfrogs’ native and invaded niches in California
provided reasonable predictions of potential bull-
frog spread in the state.
Habitat associations during the American bullfrog
invasion of California
Throughout their invasion of California, bullfrog
occurrence has consistently been associated with
higher winter temperature, which primarily occurs
at lower elevations in California. Therefore, bull-
frogs did not appear to benefit from the potential
advantages of high elevation habitats including the
prevalence of deep glacial lakes (USGS 2004) and
high snowpack that provides insulation from win-
ter freezing and sustains aquatic habitats during the

Vol.: (0123456789)
13

Fig.  5  A Habitat suitability of American bullfrogs in Califor-
nia predicted by the invaded (California-based) niche and the
native range niche during the time period 2001–2020. B Com-
parison of the native and invaded niches of American bullfrogs
in California during the time period 2001–2020. Much of the
state was predicted as highly suitable based on the native and
California niches (green). Southern deserts and portions of the
summer (Rhoades et  al. 2016). Instead, bullfrogs
appeared to be limited in their ability to occupy
cold, high elevation habitats, perhaps due to win-
ter freezing at high elevations in the Sierra Nevada
and southern Cascades (Peterson et al. 2013; Sepul-
veda and Layhee 2015). Since winter temperature
and elevation are highly correlated in California, it
is difficult to disentangle the effects of cold winters
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
Klamath mountains were predicted as highly suitable based
on the native range  niche and unsuitable based on the Cali-
fornia niche (blue). High elevations in the Sierra Nevada and
southern Cascades were predicted as unsuitable based on both
niches (brown), and a small area in northern California was
predicted to be unsuitable based on the native range niche but
highly suitable based on the California niche (tan)
from other factors associated with increased eleva-
tion. Studies in other parts of the world invaded by
bullfrogs found similar associations with elevation
(Nori et  al. 2011; Murray et  al. 2015) and winter
temperature (Johovic et  al. 2020), as did a global
study of bullfrog invasions (Ficetola et al. 2007). A
thorough examination of the impacts of topographic
complexity on regional bullfrog movement (e.g.,
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…
Van Buskirk and Jansen van Rensburg 2020) may
help shed light on the role of this factor in limit-
ing bullfrog distributions at higher elevations. The
fact that winter temperature was not an important
predictor of bullfrog suitability in the native range
model is consistent with the idea that bullfrog dis-
tribution in California is limited by a unique com-
bination of temperature, elevation, and topographic
complexity that may not exist in other portions of
the bullfrog range.
Bullfrogs were associated with humans and
human-created habitats in California. The consist-
ent association with croplands is consistent with
the idea that humans actively introduce bullfrogs in
populated areas and that bullfrogs are often asso-
ciated with human-made habitat structures (e.g.,
Ficetola et al. 2010; Murray et al. 2015). Moreover,
bullfrog observations in regions of low predicted
suitability (including the arid eastern regions of the
state) often had high human footprint values, further
highlighting the importance of human activity in
sustaining bullfrog populations in otherwise unsuit-
able areas, presumably via repeated introductions
or the creation of favorable habitats (Peterson et al.
2013). The relatively high importance of human
footprint in the native range model suggests that the
association between bullfrogs and human-modified
habitats is not restricted to California; indeed a
global analysis also found a strong positive associa-
tion with human footprint (Ficetola et al. 2007).
The limited predictive power of waterbodies in
most time periods may indicate that some important
bullfrog habitats were not accurately represented by
this predictor (e.g., cattle ponds or natural peren-
nial waterbodies used by adults for foraging), espe-
cially since those habitats likely occur on a finer
spatial scale than that captured in our environmental
data (Gahl et  al. 2009; Murray et  al. 2015). How-
ever, the beta coefficients show a consistent decline
in bullfrog habitat suitability with distance from
water, as expected. The predictor related to the tim-
ing of juvenile emigration, precipitation in the fall
(Bury and Whelan 1984; Morey 1988), was gener-
ally a poor predictor of bullfrog occurrence. Taken
together, the positive associations with cropland,
and the negative associations distance to water may
help explain the paucity of bullfrog observations in
the arid regions of eastern California.
Insights from dynamic ENMs
One goal of this study was to use dynamic environ-
mental niche modeling to overcome the violation of
the ENM assumption of environmental equilibrium.
Overall, California models used to predict future
occurrences performed similarly to models trained
on the current dataset. This indicated that dynamic
ENMs adequately captured shifts in the niches occu-
pied by bullfrogs throughout their invasion process
and that the dynamic modeling approach was suit-
able for range-shifting bullfrogs that were not likely
to be in equilibrium with their environments. The
similar performance of these ENMs also means that
the California ENM for the most recent time period
(2000–2020) may provide insight into areas where
bullfrogs are likely to expand in the future. Future
ENM-based studies of invasive species should incor-
porate dynamic modeling to address the environmen-
tal equilibrium assumption during model evaluation
and to help managers predict future invasions.
Insights from comparing invasive and native range
ENMs
Comparing the native and invaded niches, and the
frequency of invasive range occurrences in the niche
projections, can also provide insight into the inva-
sion process (Gallien et  al. 2012). There was broad
overlap between the native and invaded niches sug-
gesting that bullfrogs may be approaching environ-
mental equilibrium in California. The bulk of bullfrog
observations in California were in areas designated
as suitable habitat by both the California and native
range ENMs. These mainly occurred along the coast
and in the Central Valley, areas with a relatively
long history of bullfrog occurrences. Most of these
areas were densely occupied by bullfrogs except the
southern Central Valley and the coast of northern
California, suggesting that bullfrogs may continue
to increase in abundance in these areas in the future.
Both native and invasive range models generally
predicted low suitability in the arid, eastern parts of
the state, including the Mojave, Sonoran, and Great
Basin Deserts, as well as parts of the southern Sierra
Nevada.
Areas with high predicted suitability in the native
range model but low suitability in the invasive range
model may undergo colonization (Gallien et  al.
Vol.: (0123456789)
13

2012). These areas include large swaths of the moun-
tains of northern California, including parts of the
Coast Ranges, the Klamath Mountains, the southern
Cascades Range, and the northern Sierra Nevada
(Fig.  5). However, this prediction should be tem-
pered by uncertainty associated with extrapolating
predicted suitability into areas outside of the ENM
training dataset (Hartley et  al. 2006; Ficetola et  al.
2007), which occurred when the native range ENM
was projected onto environmental conditions in Cali-
fornia. Uncertainty from the native range model was
highest in mountainous regions of northern Califor-
nia (Fig. 4A). The MESS analysis revealed that envi-
ronmental conditions in California diverged the most
from conditions in the native range in some of the
same regions (Fig.  4B). Areas with high predicted
suitability in the invasive range model but low suit-
ability in the native range model may suggest niche
shifts during the process of invasion (Gallien et  al.
2012). These tend to be scattered in relatively small
patches throughout the state, primarily in arid, low-
elevation areas.
Management implications
The areas in California with the highest predicted risk
of future bullfrog expansion include low to mid eleva-
tion habitats with relatively high winter temperatures
and human modified habitats, such as the southern
Central Valley, the coast of northern California, and
mid elevations of the Northern and Southern Coast
ranges. High elevation areas of California and the
Mojave Desert were predicted as unsuitable based
on both the native and California niches, but bullfrog
populations have become established in these areas,
especially in pockets of suitable habitat with high
human activity. Expansion could continue in these
habitats if bullfrogs adapt to new conditions.
Competition, predation, and disease transfer
from  bullfrogs  have impacted several sensitive
amphibians in California, especially foothill yellow-
legged frogs (Rana boylii) and California red-leg-
ged frogs (Rana draytonii; Kupferberg 1997; Dou-
bledee et al. 2003). Continued bullfrog invasion has
the potential to impact these species in areas where
bullfrogs are currently rare. Additional threat-
ened and endangered amphibians species occur at
higher elevations in the mountains of California,
including Cascades frogs (Rana cascadae), Sierra
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
Nevada yellow legged frogs (R. sierrae), southern
mountain yellow legged frogs (R. muscosa), and
Yosemite toads (Anaxyrus canorus; Brown et  al.
2014, 2015; Pope et  al. 2014). The results of this
study indicate that bullfrog invasions into the high-
est elevations of the Sierra Nevada and southern
Cascades may be slowed or limited by low winter
temperatures or other factors associated with high
elevations. However, bullfrogs already overlap with
Cascades frogs at some mid-elevation sites, and our
results suggest that human activity can contribute to
the occurrence of bullfrog populations in otherwise
unsuitable areas. Furthermore, even limited habitat
overlap with at-risk native species may facilitate
pathogen transmission from bullfrogs (Peterson
and McKenzie 2014; Miaud et  al. 2016; Yap et  al.
2018). Diseases could then spread to populations
of native species that do not overlap with bullfrog
populations. In areas where bullfrogs are known or
suspected to overlap with sensitive species, careful
monitoring will be required to determine the actual
risk that bullfrogs pose to sensitive species on a
local scale. Local-scale monitoring will be espe-
cially helpful in areas with the highest uncertainty
in model predictions from this study (white areas in
Fig. 4A), mountainous regions of northern Califor-
nia and the eastern Mojave basins.
Finally, the process of building presence-back-
ground models such as ENMs includes many addi-
tional sources of uncertainty that should be carefully
considered (Beale and Lennon 2012), and many deci-
sions were made during the model-building process
that may have impacted the results. Landscape-scale
ENMs such as those developed in this study should
be paired with regional-scale presence-absence mod-
els, which require repeated systematic sampling but
may predict invasions for habitat-driven wetland spe-
cies such as bullfrogs more accurately than ENMs
(Murray et al. 2015).
Acknowledgements  MS committee members Leslie New,
Caren Goldberg, and Daniel Thornton provided guidance, feed-
back, and support throughout the duration of this study. Bull-
frog occurrence records in California were provided by Laura
Patterson and Heather Perry (California Department of Fish
and Wildlife). Sky Button provided guidance on environmen-
tal niche modeling, and Babak Naimi (University of Helsinki)
provided guidance on use of the sdm R package. The Scien-
tific Writing Association for Graduate Students (SWAGS) and
members of the Piovia-Scott lab at Washington State Univer-
sity gave feedback on writing.
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…
Authors’ contributions  Both authors contributed to the
study conception and design. NN obtained and prepared data,
selected statistical analyses to be used in the study, ran statis-
tical analyses, and produced figures and tables. NN wrote the
first draft of the manuscript; JP-S contributed substantially to
revisions of the manuscript. Both authors read and approved
the final manuscript.
Funding  This material is based upon work supported by the
National Science Foundation Graduate Research Fellowship
Program under Grant No. 1842493.
Availability of data and material  This study relies on previ-
ously published datasets that are available from the following
public domain resources: https://​www.​gbif.​org/; https://​bison.​
usgs.​gov/#​home; https://​nas.​er.​usgs.​gov/​viewer/​omap.​aspx?;
https://​prism.​orego​nstate.​edu/; https://​apps.​natio​nalmap.​gov/​
downl​oader/#/; https://​thema​sites.​pbl.​nl/​tridi​on/​en/​thema​sites/​
hyde/​downl​oad/​index-2.​html; https://​datad​ryad.​org/​stash/​datas​
et/​doi:​10.​5061/​dryad.​052q5
Code availability  The code used for all analyses in this
manuscript can be provided by the corresponding author upon
request.
Declarations  tive effects of habitat modification and species invasion
Conflict of interest  The authors have no relevant financial or
non-financial interests to disclose.
References
Adams A (2017) Decline and localized extirpation of the foot-
hill yellow-legged frog (Rana boylii) in the presence of
the fungal pathogen, Batrachochytrium dendrobatidis:
Contemporary and historical perspectives. PhD Thesis,
UC Santa Barbara
Allen CR, Nemec KT, Wardwell DA et al (2013) Predictors of
regional establishment success and spread of introduced
non-indigenous vertebrates. Glob Ecol Biogeogr 22:889–
899. https://​doi.​org/​10.​1111/​geb.​12054
Arim M, Abades SR, Neill PE et al (2006) Spread dynamics of
invasive species. Proc Natl Acad Sci 103:374–378. https://​
doi.​org/​10.​1073/​pnas.​05042​72102
Barbet-Massin M, Rome Q, Villemant C, Courchamp F
(2018) Can species distribution models really predict
the expansion of invasive species? PLoS ONE. https://​
doi.​org/​10.​1371/​journ​al.​pone.​01930​85
Beale CM, Lennon JJ (2012) Incorporating uncertainty in
predictive species distribution modelling. Philos Trans
R Soc B 367:247–258. https://​doi.​org/​10.​1098/​rstb.​
2011.​0178
BISON.usgs.gov (2021) BISON occurrence download
Bissattini AM, Vignoli L (2017) Let’s eat out, there’s cray-
fish for dinner: American bullfrog niche shifts inside and
outside native ranges and the effect of introduced crayfish.
Biol Invasions 19:2633–2646. https://​doi.​org/​10.​1007/​
s10530-​017-​1473-6
Broennimann O, Guisan A (2008) Predicting current and future
biological invasions: both native and invaded ranges mat-
ter. Biol Lett 4:585–589. https://​doi.​org/​10.​1098/​rsbl.​
2008.​0254
Brown C, Hayes MP, Green G, et al (2015) Yosemite toad con-
servation assessment. USDA Forest Service, CA Depart-
ment of Fish and Wildlife, National Park Service, US Fish
and Wildlife Service
Brown C, Hayes MP, Green G, Macfarlane D (2014) Mountain
yellow-legged frog conservation assessment for the Sierra
Nevada mountains of California, USA. USDA Forest Ser-
vice, CA Department of Fish and Wildlife, National Park
Service, US Fish and Wildlife Service
Bury RB, Whelan JA (1984) Ecology and management of the
bullfrog. US Fish and Wildlife Service, Denver Wildlife
Research Center
Conant R, Collins JP (1991) A field guide to reptiles and
amphibians: Eastern/Central North America. Houghton
Mifflin, Boston
D’Amore A, Hemingway V, Wasson K (2009) Do a threat-
ened native amphibian and its invasive congener differ in
response to human alteration of the landscape? Biol Inva-
sions 12:145. https://​doi.​org/​10.​1007/​s10530-​009-​9438-z
Didham RK, Tylianakis JM, Gemmell NJ et al (2007) Interac-
on native species decline. Trends Ecol Evol 22:489–496.
https://​doi.​org/​10.​1016/j.​tree.​2007.​07.​001
Doubledee RA, Muller EB, Nisbet RM (2003) Bullfrogs, dis-
turbance regimes, and the persistence of California red-
legged frogs. J Wildl Manag 67:424–438. https://​doi.​org/​
10.​2307/​38027​83
Ehrlich P (1989) Attributes of invaders and the invading pro-
cesses vertebrates. Biol Invasions Glob Perspect pp
315–328
Elith J, Kearney M, Phillips S (2010) The art of modelling
range-shifting species. Methods Ecol Evol 1:330–342.
https://​doi.​org/​10.​1111/j.​2041-​210X.​2010.​00036.x
Fernandez M, Hamilton H (2015) Ecological niche transfer-
ability using invasive species as a case study. PLoS ONE.
https://​doi.​org/​10.​1371/​journ​al.​pone.​01198​91
Ficetola GF, Maiorano L, Falcucci A et al (2010) Knowing the
past to predict the future: land-use change and the distri-
bution of invasive bullfrogs. Glob Change Biol 16:528–
537. https://​doi.​org/​10.​1111/j.​1365-​2486.​2009.​01957.x
Ficetola GF, Thuiller W, Miaud C (2007) Prediction and
validation of the potential global distribution of a prob-
lematic alien invasive species — the American bullfrog.
Divers Distrib 13:476–485. https://​doi.​org/​10.​1111/j.​
1472-​4642.​2007.​00377.x
Fourcade Y, Engler JO, Rödder D, Secondi J (2014) Mapping
species distributions with MAXENT using a geographi-
cally biased sample of presence data: a performance
assessment of methods for correcting sampling bias.
PLoS ONE. https://​doi.​org/​10.​1371/​journ​al.​pone.​00971​
22
Gahl MK, Calhoun AJK, Graves R (2009) Facultative use of
seasonal pools by American bullfrogs (Rana catesbeiana).
Wetlands 29:697–703. https://​doi.​org/​10.​1672/​08-​56.1
Vol.: (0123456789)
13

Gallien L, Douzet R, Pratte S et  al (2012) Invasive species
distribution models – how violating the equilibrium
assumption can create new insights. Glob Ecol Biogeogr
21:1126–1136. https://​doi.​org/​10.​1111/j.​1466-​8238.​2012.​
00768.x
Garcia TS, Rowe JC, Doyle JB (2015) A tad too high: Sen-
sitivity to UV-B radiation may limit invasion poten-
tial of American bullfrogs (Lithobates catesbeianus) in
the Pacific Northwest invasion range. Aquat Invasions
10:237–247. https://​doi.​org/​10.​3391/​ai.​2015.​10.2.​12
GBIF (2020) GBIF occurrence download - Amphibia.
Accessed 20 Oct 2020
Goldewijk KK, Beusen A, de Vos M, van Drecht G (2011) The
HYDE 3.1 spatially explicit database of human induced
land use change over the past 12,000 years. Glob Ecol
Biogeogr 20:73–86
Guisan A, Thuiller W (2005) Predicting species distribu-
tion: offering more than simple habitat models. Ecol Lett
8:993–1009
Hartley S, Harris R, Lester PJ (2006) Quantifying uncertainty
in the potential distribution of an invasive species: climate
and the Argentine ant. Ecol Lett 9:1068–1079. https://​doi.​
org/​10.​1111/j.​1461-​0248.​2006.​00954.x
Hattab T, Garzon-Lopez CX, Ewald M et al (2017) A unified
framework to model the potential and realized distribu-
tions of invasive species within the invaded range. Divers
Distrib 23:806–819. https://​doi.​org/​10.​1111/​ddi.​12566
Herrel A, van der Meijden A (2014) An analysis of the live
reptile and amphibian trade in the USA compared to the
global trade in endangered species. Herpetol J 24:103–110
Hijmans RJ, Phillips AS, Leathwick JR, Elith J (2011) Package
“dismo.” http://​cran.r-​proje​ct.​org/​web/​packa​ges/​dismo/​
index.​html
Hijmans RJ, van Etten J (2012) raster: Geographic analysis and
modeling with raster data. R Package Version 20–12
IUCN (2020) Lithobates catesbeianus. In: IUCN Red List
Threat. Species
Jennings MR, Hayes MP (1985) Pre-1900 overharvest of Cali-
fornia red-legged frogs (Rana aurora draytonii): The
inducement for bullfrog (Rana catesbeiana) introduction.
Herpetologica 41:94–103
Jiménez-Valverde A, Peterson AT, Soberón J et al (2011) Use
of niche models in invasive species risk assessments.
Biol Invasions 13:2785–2797. https://​doi.​org/​10.​1007/​
s10530-​011-​9963-4
Johovic I, Gama M, Banha F et al (2020) A potential threat to
amphibians in the European Natura 2000 network: Fore-
casting the distribution of the American bullfrog Litho-
bates catesbeianus. Biol Conserv 245:108551. https://​doi.​
org/​10.​1016/j.​biocon.​2020.​108551
Kupferberg SJ (1997) Bullfrog (Rana catesbeiana) invasion of
a California river: the role of larval competition. Ecology
78:1736–1751. https://​doi.​org/​10.​1890/​0012-​9658(1997)​
078[1736:​BRCIOA]​2.0.​CO;2
Lawler SP, Dritz D, Strange T, Holyoak M (1999) Effects of
introduced mosquitofish and bullfrogs on the threatened
California red-legged frog. Conserv Biol 13:613–622.
https://​doi.​org/​10.​1046/j.​1523-​1739.​1999.​98075.x
Leutner B, Horning N, Schwalb-Willmann J, Hijmans RJ
(2019) Tools for remote sensing data analysis. In: CRAN
Vol:. (1234567890)
13
N. Nelson, J. Piovia‑Scott
Lobo JM, Jiménez-Valverde A, Real R (2008) AUC: a mislead-
ing measure of the performance of predictive distribution
models. Glob Ecol Biogeogr 17:145–151. https://​doi.​org/​
10.​1111/j.​1466-​8238.​2007.​00358.x
Lowe S, Browne M, Boudjelas S, de Poorter M (2000) 100 of
the world’s worst invasive alien species: a selection from
the Global Invasive Species Database. The Invasive Spe-
cies Specialist Group
McKercher L, Gregoire DR (2021) Lithobates catesbeianus
(Shaw, 1802). In: US Geol. Surv. Nonindigenous Aquat.
Species Database
Miaud C, Dejean T, Savard K et  al (2016) Invasive North
American bullfrogs transmit lethal fungus Batra-
chochytrium dendrobatidis infections to native amphibian
host species. Biol Invasions 18:2299–2308. https://​doi.​
org/​10.​1007/​s10530-​016-​1161-y
Morey S (1988) American bullfrog (Lithobates catesbeianus).
California Department of Fish and Wildlife California
Interagency Wildlife Task Group, Sacramento, California
Moyle PB (1973) Effects of introduced bullfrogs, Rana cates-
beiana, on the native frogs of the San Joaquin Valley,
California. Copeia 1973:18–22. https://​doi.​org/​10.​2307/​
14423​51
Murray RG, Popescu VD, Palen WJ, Govindarajulu P (2015)
Relative performance of ecological niche and occupancy
models for predicting invasions by patchily-distributed
species. Biol Invasions 17:2691–2706. https://​doi.​org/​10.​
1007/​s10530-​015-​0906-3
Naimi B, Araujo MB (2016) sdm: a reproducible and extensi-
ble R platform for species distribution modelling. Ecogra-
phy 39:368–375. https://​doi.​org/​10.​1111/​ecog.​01881
Newman M, Alexander MA, Ault TR et al (2016) The pacific
decadal oscillation, revisited. J Clim 29:4399–4427.
https://​doi.​org/​10.​1175/​JCLI-D-​15-​0508.1
Nori J, Akmentins MS, Ghirardi R et al (2011) American bull-
frog invasion in Argentina: where should we take urgent
measures? Biodivers Conserv 20:1125–1132. https://​doi.​
org/​10.​1007/​s10531-​011-​0014-3
Palen WJ, Schindler DE (2010) Water clarity, maternal behav-
ior, and physiology combine to eliminate UV radiation
risk to amphibians in a montane landscape. Proc Natl
Acad Sci 107:9701–9706. https://​doi.​org/​10.​1073/​pnas.​
09129​70107
Perry H (2020) Bullfrog CA locations (compiled from publicly
available online datasets). California Department of Fish
and Wildlife, West Sacramento, CA
Peterson AC, McKenzie VJ (2014) Investigating differences
across host species and scales to explain the distribution
of the amphibian pathogen Batrachochytrium dendroba-
tidis. PLoS ONE. https://​doi.​org/​10.​1371/​journ​al.​pone.​
01074​41
Peterson AC, Richgels KLD, Johnson PTJ, McKenzie VJ
(2013) Investigating the dispersal routes used by an inva-
sive amphibian, Lithobates catesbeianus, in human-dom-
inated landscapes. Biol Invasions 15:2179–2191. https://​
doi.​org/​10.​1007/​s10530-​013-​0442-y
Petitpierre B, Kueffer C, Broennimann O et al (2012) Climatic
niche shifts are rare among terrestrial plant invaders. Sci-
ence 335:1344–1348. https://​doi.​org/​10.​1126/​scien​ce.​
12159​33
Using environmental niche models to elucidate drivers of the American bullfrog invasion in…
Pope K, Brown C, Hayes M, et al (2014) Cascades frog conser-
vation assessment. USDA Forest Serivice, Pacific South-
west Reserach Station
PRISM Climate Group (2020). In: Northwest Alliance Com-
put. Sci. Eng. Or. State Univ. http://​prism.​orego​nstate.​edu
Rhoades AM, Huang X, Ullrich PA, Zarzycki CM (2016)
Characterizing Sierra Nevada snowpack using variable-
resolution CESM. J Appl Meteorol Climatol 55:173–196.
https://​doi.​org/​10.​1175/​JAMC-D-​15-​0156.1
Runquist RDB, Lake T, Tiffin P, Moeller DA (2019) Species
distribution models throughout the invasion history of
Palmer amaranth predict regions at risk of future inva-
sion and reveal challenges with modeling rapidly shift-
ing geographic ranges. Sci Rep. https://​doi.​org/​10.​1038/​
s41598-​018-​38054-9
Sepulveda AJ, Layhee M (2015) Description of fall and winter
movements of the introduced American bullfrog (Litho-
bates catesbeianus) in a Montana, USA, pond. Herpetol
Conserv Biol 10:978–984
Soberón J (2007) Grinnellian and Eltonian niches and geo-
graphic distributions of species. Ecol Lett 10:1115–1123.
https://​doi.​org/​10.​1111/j.​1461-​0248.​2007.​01107.x
Soberón J, Peterson AT (2005) Interpretation of models of
fundamental ecological niches and species’ distributional
areas
Srivastava V, Lafond V, Griess VC (2019) Species distribution
models (SDM): applications, benefits and challenges in
invasive species management. CAB Rev 14:1–13
Stebbins RC (2003) A field guide to western reptiles and
amphibians, 3rd edn. Houghton Mifflin Company, Boston
Stewart ER, Reese SA, Ultsch GR (2004) The physiology of
hibernation in Canadian leopard frogs (Rana pipiens)
and bullfrogs (Rana catesbeiana). Physiol Biochem Zool
77:65–73. https://​doi.​org/​10.​1086/​378921
Thornton DH, Peers MJL (2019) Species distribution mod-
eling. In: Population ecology in practice: underused, mis-
used and abused methods. Wiley-Blackwell, London
Thuiller W, Lafourcade B, Engler R, Araújo MB (2009) BIO-
MOD – a platform for ensemble forecasting of species
distributions. Ecography 32:369–373. https://​doi.​org/​10.​
1111/j.​1600-​0587.​2008.​05742.x
View publication stats
USGS (2004) National Hydrography Dataset. In: US Dep.
Inter. US Geol Surv
Václavík T, Meentemeyer RK (2012) Equilibrium or not?
Modelling potential distribution of invasive species in dif-
ferent stages of invasion. Divers Distrib 18:73–83. https://​
doi.​org/​10.​1111/j.​1472-​4642.​2011.​00854.x
Van Buskirk J, Jansen van Rensburg A (2020) Relative impor-
tance of isolation-by-environment and other determinants
of gene flow in an alpine amphibian. Evolution 74:962–
978. https://​doi.​org/​10.​1111/​evo.​13955
Venter O, Sanderson EW, Magrach A, et al (2016) Data from:
Global terrestrial Human Footprint maps for 1993 and
2009, v2, Dryad. Dataset
Whitney KD, Gabler CA (2008) Rapid evolution in introduced
species, ‘invasive traits’ and recipient communities: chal-
lenges for predicting invasive potential. Divers Distrib
14:569–580. https://​doi.​org/​10.​1111/j.​1472-​4642.​2008.​
00473.x
Willis YL, Moyle DL, Baskett TS (1956) Emergence, breeding,
hibernation, movements and transformation of the bull-
frog, Rana catesbeiana, in Missouri. Copeia 1956:30–41.
https://​doi.​org/​10.​2307/​14392​41
With KA (2002) The landscape ecology of invasive spread.
Conserv Biol 16:1192–1203. https://​doi.​org/​10.​1046/j.​
1523-​1739.​2002.​01064.x
Yalcin S, Leroux SJ, Jonathan B (2017) Diversity and suitabil-
ity of existing methods and metrics for quantifying species
range shifts. Glob Ecol Biogeogr 26:609–624. https://​doi.​
org/​10.​1111/​geb.​12579
Yap TA, Koo MS, Ambrose RF, Vredenburg V (2018) Intro-
duced bullfrog facilitates pathogen invasion in the western
United States. PLoS ONE 13:e0188384
Publisher’s Note  Springer Nature remains neutral with regard
to jurisdictional claims in published maps and institutional
affiliations.
Vol.: (0123456789)
13