Molecular Ecology Resources (2017) 17, 120–128 doi: 10.1111/1755-0998.

12558

SPECIAL ISSUE: POPULATION GENOMICS WITH R

Developing educational resources for population genetics in
R: an open and collaborative approach
ZHIAN N. KAMVAR,* MARGARITA M. L OPEZ-URIBE,†
SIMONE COUGHLAN,‡ NIKLAUS J.
€

GR UNWALD,*§ HILMAR LAPP¶ and S T EP H A N I E M A N E L * *
*Department of Botany and Plant Pathology, Oregon State University, Corvallis, OR, USA, †Department of Entomology and
Plant Pathology, North Carolina State University, Raleigh, NC 27695, USA, ‡School of Mathematics, Statistics and Applied
Mathematics, National University of Ireland Galway, Galway, Ireland, §Horticultural Crops Research Unit, USDA Agricultural
Research Service, Corvallis, OR 97330, USA, ¶Center for Genomic and Computational Biology, Duke University, Durham, NC
27708, USA, **EPHE, PSL Research University, CNRS, UM, SupAgro, IRD, INRA, UMR 5175, CEFE F-34293, Montpellier,
France

Abstract
The R computing and statistical language community has developed a myriad of resources for conducting population
genetic analyses. However, resources for learning how to carry out population genetic analyses in R are scattered and
often incomplete, which can make acquiring this skill unnecessarily difficult and time consuming. To address this
gap, we developed an online community resource with guidance and working demonstrations for conducting popu-
lation genetic analyses in R. The resource is freely available at http://popgen.nescent.org and includes material for
both novices and advanced users of R for population genetics. To facilitate continued maintenance and growth of this
resource, we developed a toolchain, process and conventions designed to (i) minimize financial and labour costs of
upkeep; (ii) to provide a low barrier to contribution; and (iii) to ensure strong quality assurance. The toolchain
includes automatic integration testing of every change and rebuilding of the website when new vignettes or edits are
accepted. The process and conventions largely follow a common, distributed version control-based contribution
workflow, which is used to provide and manage open peer review by designated website editors. The online
resources include detailed documentation of this process, including video tutorials. We invite the community of pop-
ulation geneticists working in R to contribute to this resource, whether for a new use case of their own, or as one of
the vignettes from the ‘wish list’ we maintain, or by improving existing vignettes.
Keywords: git, GitHub, online resources, open access, open education, population genetics

Received 25 March 2016; accepted 2 May 2016

led to the rapid development of new statistical methods,

Introduction
The wealth of data resulting from the broad and afford-
able availability of modern high throughput sequencing
(HTS) and genotyping technologies have been a boon to
investigating population genetics research questions
(Luikart et al. 2003) with applications in a wide range of
fields, from medicine to epidemiology, agronomy, evolu-
tionary biology, ecology and conservation biology. As a
consequence, both the community of scientists needing
to use analytical methods in population genetics and that
of scientists developing such methods have increased
dramatically. At the same time, the expanding types of
questions made addressable by the wealth of data have
Correspondence: St
ephanie Manel, Fax: +33 (0) 4 67 61 33 36;
E-mail: stephanie.manel@cefe.cnrs.fr
which in turn have brought new input and output for-
mats with them. In this environment, even only knowing
which analytical methods are already available, and
which of them can be strung together into a functioning
workflow, can be quite challenging not only for novices
but even for more experienced users (Andrews & Luikart
2014; Benestan et al. 2016).
Even before the advent of the HTS data deluge, the
field of population genetics has relied heavily on com-
puter software for data analysis. Traditionally, such anal-
ysis often had to combine multiple software programs,
each of which uses different input formats and operating
systems (Excoffier & Heckel 2006), making it a time-con-
suming process. Since then, a consolidation on the R plat-
form for statistical computing (R Core Team 2015) has
fostered a rich, more interoperable and open source

© 2016 John Wiley & Sons Ltd

 RESOURCES FOR POPULATION GENETICS IN
ecosystem of a myriad of analytical resources for both
users and developers across many quantitative sciences
(Boettiger et al. 2015), including in population genetics.
Some of the R packages available for population genetics
analysis are featured in this special issue: ape (Paradis
et al. 2004), adegenet (Jombart 2008; Jombart & Ahmed
2011), pegas (Paradis 2010), phangorn (Schliep 2011),
poppr (Kamvar et al. 2014, 2015a) and OutFLANK (Whit-
lock & Lotterhos 2015).
The rich ecosystem of relevant R packages is obvi-
ously a blessing, but it can also be a curse. The Genetics
Task View on the Comprehensive R Archive Network
(CRAN, https://cran.r-project.org/), the repository of all R
add-on packages, mentions more than 30 packages for
different population genetic analyses ranging from pop-
ulation structure to genome-wide association studies.
Minimally, each package comes with documentation for
the methods and data structures it exposes to the user.
However, addressing a research question typically
requires combining methods from several packages, and
documentation that can guide users on selecting pack-
ages that are suitable for a given question, interoperate
well, and are well maintained is scarce. A community
mailing list where one could ask questions in this regard
exists (R-SIG-GENETICS, https://stat.ethz.ch/pipermail/r-
sig-genetics/), but it is not widely known. More impor-
tantly, although a mailing list can yield tips and answers
to ad hoc problems, it is not a go-to community resource
providing narrated example workflows, replete with
working code from loading the data to visualizing the
results, vetted by other subject experts.
To fill this gap, we created an online community web-
site of vignettes for population genetics in R, called
popgenInfo. It is freely available at http://popgen.
nescent.org. Recognizing that resources aimed to be
community-maintained often face strong challenges to
adoption and sustainability, we designed a process and
chose building block technologies for hosting, contribut-
ing and building the site aimed at overcoming these chal-
lenges while still maintaining quality assurance. In this
study, we report on the resource as a whole, describe in
detail the contribution and maintenance process and dis-
cuss our initial results with using, and in the course
refining, the process ourselves to populate the site with
an initial collection of vignettes covering basic summary
statistics and population differentiation methods, for
microsatellite, SNP and sequence-type data. In a nut-
shell, the contribution and maintenance process centres
on authoring vignettes in the text-based R Markdown for-
mat (Baumer et al. 2014); using continuous integration
for automatically testing and building the site (Wilson
et al. 2014); using Docker containers for doing so repro-
ducibly (Boettiger 2015); and using GitHub pull requests
for open peer review of all contributions (see Ram 2013).
© 2016 John Wiley & Sons Ltd
R 121
The resource we describe is aimed at both novices in R or
population genetics, for whom the site can provide
tested workflows adaptable to user’s own needs, and at
advanced R users in population genetics, who can con-
tribute their expertise by authoring, improving and peer-
reviewing vignettes. To alleviate potential barriers to
contribution, we include both detailed written documen-
tation and video tutorials.
Community resource design goals and
implementation
In designing and developing the online community
resource we report here, we sought to achieve the follow-
ing aims:
1 Long-term sustainable hosting. This required a hosting
platform for which the financial support did not come
with a built-in expiration date, precluding hosting
solutions that depend on ongoing grant funding.
2 Low technical barriers to authoring contributions. Specifi-
cally, authoring content for the website should require
few or no tools and languages that are likely to be for-
eign, or highly platform specific to users otherwise
familiar with conducting data analysis in R.
3 Provisions for strong quality assurance. Specifically, all
code examples provably work, and primers are only
published following open peer review.
4 Sustainable time commitment for website editors. In partic-
ular, manual tasks that do not depend on population
genetics or R expertise must require minimal time or
be eliminated altogether.
5 All changes and contributions are automatically tracked.
This is not only important for provenance and credit,
but also a direct consequence of preceding aims,
because it eliminates the need for contributors, editors
and peer reviewers to manually identify changes.
6 Fully reproducible and automatable software environment
for rebuilding (rendering) the website. This, too, is a direct
consequence of preceding aims. It eliminates otherwise
manual maintenance tasks, and it makes the outcome
of those tasks predictable across the possibly different
native environments used by contributors and editors.
To achieve these aims, we devised a build, deploy-
ment and contribution process with the following build-
ing blocks:
Hosting
The rendered website is hosted on GitHub Pages. This
service is provided by Github free of charge and presum-
ably funded by the revenue GitHub makes from hosting
private software source code repositories, a source that,
122 Z . N . K A M V A R E T A L .
unlike volatile grant funding, is unlikely to become obso-
lete or unavailable in the foreseeable future.
Authoring
Content for the website is authored in R Markdown for-
mat (Xie 2013; Baumer et al. 2014), which uses simple
text-based markup in which R code fragments can be
directly embedded. R Markdown can be authored in any
basic text editor. RStudio, a highly popular tool for cod-
ing R scripts, includes interactive features for navigating,
rendering and viewing of R Markdown documents (see
Gandrud 2015).
Version control
The source of the website, including all R Markdown
documents, is hosted on GitHub (http://github.com)
under Git version control. Git is a distributed version
control system, which lends itself particularly well to
community contribution. Prospective contributors can
create their own copy of the repository (called a ‘fork’),
create sets of changes against their copy (through
branching off and adding commits) and submit these
change sets (referred to as ‘pull requests’) back to the
main repository (usually called ‘upstream’), all without
having to be granted any special permissions. Using GIT
for version control, and hosting repositories on GitHub,
has become widely popular for software projects, includ-
ing for scientific software, thanks in part to the benefits
that derive from both GIT’S and GitHub’s features for col-
laborative development and open science (Ram 2013).
Blischak et al. (2016) recently published an introduction
to GIT aimed at scientist developers, and we provide a
glossary of key operations in Table S1 (Supporting infor-
mation). Because version control in general and GIT in
particular often remain lacking from the training quanti-
tative biologists receive, we provide detailed guidelines
and video tutorials for how working with GIT can be
managed directly from within R, using the GIT2R package
(Widgren & git2r Contributors 2016).
Peer review
Pull requests, the regular community contribution mech-
anism in GitHub, lend itself particularly well to the open
peer review process which we aimed to achieve. The
user interface GitHub has built around this feature visu-
ally identifies the change set being submitted for consid-
eration and allows commenting on the change set as a
whole or on individual lines of text for which changes
are being proposed. Contributors can reply to comments
or add changes that are responsive to the comments. The
full discussion thread remains openly available,
including after the change set has been accepted. Inte-
grating the contribution upon acceptance usually only
takes the click of a button by the designated maintainer
(editor). These capabilities allowed us to devise an open
peer review process that is seamless and efficient, yet
does not depend on any custom tooling.
Automatic testing and builds
We use a continuous integration testing service, Circle CI
(http://circleci.com), to automatically test every change
set submitted for consideration as a pull request. At pre-
sent, this test consists of successfully rendering the full
website from its source code, which, by virtue of render-
ing the R Markdown documents, includes executing all
embedded R code fragments. The pull request user inter-
face prominently displays the success, or failure of this
test, which thus provides immediate feedback both to
the contributor, as well as to a prospective peer reviewer
whether basic errors are present or absent. A peer
reviewer can therefore focus on evaluating the scientific
merits of the content, and the included R code. The con-
tinuous integration is configured such that upon accep-
tance of a pull request, the website is rebuilt from scratch
and automatically deployed to GitHub Pages, without
requiring any manual supervision.
Reproducible software environment
Similar to challenges with constantly evolving scientific
software in general, R packages, including those for
population genetics, sometimes change their behaviour,
data structures or interface from one version to
another, which can make the numeric results, or
whether code using them works at all, highly depen-
dent on the environment of versions and dependencies
installed on a system. To isolate the software environ-
ment needed to render the R Markdown source docu-
ments and to run the embedded R code, we use
software containerization in the form of Docker con-
tainers (Boettiger 2015). We use a Docker image
designed specifically for population genetics in R to
run all tests, and to render the website from its
sources. To support this, the Docker image includes
the tools needed to render R Markdown documents.
The specification for the Docker image is maintained
publicly under version control (http://github.com/
NESCent/popgen-docker) and can be used by anyone,
including prospective contributors, to render the vign-
ettes, or the entire website, under the exact same envi-
ronment as they would be in our automatic testing
and site rebuild (using, for example, the Docker image
continuously updated at Docker Hub at https://hub.-
docker.com/r/hlapp/rpopgen/).
© 2016 John Wiley & Sons Ltd
 RESOURCES FOR POPULATION GENETICS IN R 123

Figure 1 illustrates as a flow chart how we combine
these components into a community contribution process
that includes automatic testing, open peer review and
reproducible rebuilds of the online resource as a whole.
Vignette development
In contrast to the succinct and often terse method by
method documentation that comes with most R pack-
ages, a vignette is long-form documentation to guide
users on how multiple methods, from one or, as will be
more typical in our case, multiple packages can be com-
bined to address an analysis use case from start to end
(see Wickham 2015). Our goal was to develop vignettes
that are detailed enough for the reader to learn how to
solve a population genetics analysis problem using the
tools suggested by the vignette’s author(s).
As guidance for prospective contributors, we provide
a template vignette (http://popgen.nescent.org/TEM-
PLATE.html). To allow contributors some creative free-
dom, vignettes are not required to strictly follow a
certain structure, but they must include several key
points: (i) Description of purpose: an introduction that lets
readers know what the vignette is for; (ii) Resources: short
descriptions of the data and R packages utilized in the
vignette, to give readers context of the analyses; (iii)
Workflow: the bulk of the vignette, normally an analysis
starting from data import to inspecting or visualizing the
results; (iv) References/Acknowledgements: includes all
works cited as well as the authors and contributors of
the vignette; (v) R session information: lists the operating
system, version of R and all R packages present when the
vignette was rendered, including their versions, installa-
tion date and source. The latter is considered a best

Fig. 1 Flow chart of vignette creation and submission. Commits move from top to bottom. The left side of the flow chart (orange) repre-
sents the popgenInfo repository on the NESCent account that exists on GitHub (denoted by the cloud symbol). The right side (blue) rep-
resents an individual user’s account that can exist both on GitHub and on their own computer (denoted by the cloud and computer
exchange symbols). Each of the three shapes (square, diamond and circle) represents a separate branch of the popgenInfo repository as
indicated by the label above each. Colours within each shape represent the state of the repository at a given commit. Dashed arrows rep-
resent actions taken by a human to move information between branches or repositories. The solid arrows represent automated processes
and commits. The process of vignette submission is outlined in seven steps (blue-labelled steps are carried out by the user; black-
labelled steps are automatic). 1) The popgenInfo repository (NEScent/popgenInfo) is forked from NESCent into the user’s GitHub
account. It is at this time that the user may additionally clone the repository to their computer. 2) The user creates and moves to a new
branch from the master branch on their account (user/popgenInfo). 3) New content (e.g. a vignette or correction) is added to the new
branch with git add and changes are stored using git commit. 4) A new pull request is created from the user’s new branch on GitHub to
the NESCent/popgenInfo master branch. Once the pull request is made, all future changes to the user/popgenInfo new branch will be
tracked. The review process begins when the user declares they are ready. 5) Once all requested modifications have been made and two
maintainers find it satisfactory, the new content is pulled into the master branch of the NESCent/popgenInfo repository. 6) The website
is automatically rebuilt with continuous integration in a Docker container and deployed within minutes. 7) The user can update their
master branch by using git fetch upstream and git merge upstream/master. Icon credits: FontAwesome.

© 2016 John Wiley & Sons Ltd

124 Z . N . K A M V A R E T A L .
practice in the R community (Wickham 2015), because
packages in the R ecosystem change often, and therefore,
knowing this information is crucial when investigating
differences in behaviour between the online vignette and
running the workflow on a local system.
A simple example of a workflow vignette is the ‘Pop-
ulation Differentiation for DNA sequence data’ vignette
(http://popgen.nescent.org/PopDiffSequenceDa-
ta.html). It describes a simplified workflow on how to
estimate population differentiation using two genes (one
mitochondrial and one nuclear) from the bee species
Eulaema bombiformis. It begins with an auto-generated
table of contents (Fig. 2). After a short introduction to the
topic of population differentiation and a description of
the data, the necessary packages to run the code are
listed. The first part of the workflow demonstrates
importing data. This is important to include because data
import can be challenging for novice users. The follow-
ing section addresses the specifics of how to calculate
Fig. 2 View of a vignette on how to estimate population differentiation from sequence data.
population differentiation from DNA sequence data.
This section includes narrated snippets of R code with
rendered output (e.g. data and plots). The vignette ends
with a conclusion that summarizes the steps of the analy-
sis and discusses the biological meaning of this type of
analysis, followed by a list of vignette authors and con-
tributors, and the R session information.
It is worth noting that vignettes intentionally do not
aim to be exhaustive, neither about which packages
could be used, nor about all the alternative ways in
which the analysis question could be addressed. Instead,
it aims to provide an example of how to analyse one’s
data to answer a certain research question.
Vignette submission
Vignettes are submitted for inclusion in our site through
a four-step process, which we adapted from an open
source software contribution workflow known as
© 2016 John Wiley & Sons Ltd
 RESOURCES FOR POPULATION GENETICS IN
‘GitHub flow’ (Fig. 1; Ram 2013): (i) fork the repository
from the master repository (currently at http://github.-
com/NESCent/popgenInfo), or, if it was forked previ-
ously, update the fork’s master branch; (ii) create a new
branch from the master branch, and switch to it; (iii) add
or edit content; and (iv) create a pull request against the
master repository (the default for forked repositories).
The first step creates an exact copy of the popgenInfo
repository in the prospective vignette author’s personal
GitHub account, which is then cloned to the local com-
puter for content authoring or editing. This process only
needs to be performed once. After that, the user can
update their fork’s master branch from the master repos-
itory (typically referred to as ‘upstream’) as we have
detailed in our Contribution Guidelines (http://pop-
gen.nescent.org/CONTRIBUTING.html). In the second
step, the contributor creates a new branch (Table S1, Sup-
porting information), which we recommend to name
according to the pattern [DATE]-[SUBJECT]. By creating
a new branch, the contributor isolates themselves from
upstream changes possibly conflicting with their work in
progress (see section ‘Updating’) once they commence
step three.
To aid users who are familiar with invoking methods
in R but who are novices in version control or git, we cre-
ated a detailed tutorial on operating this git-based contri-
bution process by interacting only with R, thanks to the
git2r package (Widgren & git2r Contributors 2016),
which exposes git operations as R functions. The tutorial
is available at http://popgen.nescent.org/CONTRIBU-
TING_WITH_GIT2R.html, which includes links to
videos to walk new users through the process.1
Validation and review
As described above, after a vignette (or other contribu-
tion) is submitted through a pull request, it is first
automatically tested using a reproducible software envi-
ronment in the form of a Docker container. Once the test
results show that the site can still be successfully ren-
dered with the submitted content, the vignette then goes
through a process of open peer review in which anyone
in the community is invited to comment. Changes added
by the contributor in response to comments undergo the
same automatic testing. The open peer review is com-
pleted when a minimum of two maintainers of the
upstream repository approve. Once approved, the pull
request may be accepted, which triggers an automatic
update of the website with the new vignette in place
(Fig. 1, steps 5, 6 and 7).
1
As GitHub’s API is subject to occasional change, these videos
may appear outdated.
© 2016 John Wiley & Sons Ltd
R 125
Updating one’s fork
Contributors’ forks are in essence copies of the ‘up-
stream’ master repository (NESCent/popgenInfo)
obtained at some point in time. To minimize the chance
that a contributor’s changes conflict with those of other
contributors, all changes, including new vignettes, and
updates of existing ones must start from an up to date
copy of the master repository. This is achieved by first
fetching the changes in the master repository and then
merging them into the master branch of one’s fork.
Because in this workflow the master branch of the fork is
only ever modified by the contributor when merging in
changes from upstream, this process will not produce
conflicts when updating a fork. This is important because
the task of resolving merge conflicts sometimes requires
advanced technical knowledge. Our workflow does not
prevent merge conflicts altogether, but it allows for them
to be resolved on the upstream master repository, under
the hands of technically more experienced maintainers.
Results and discussion
We sought to create a community website for hosting
narrated, peer-vetted workflows of population genetic
analyses with demonstrably working and fully repro-
ducible code for every step.
To accomplish this, one of the main challenges we set
out to address was lowering barriers to contribution,
both technical and nontechnical, while not compromis-
ing stringent, open and transparent quality control. Only
a minority of R users are also package developers; hence,
our process for contributing needed to be intuitive
enough for those who use R for analysis, but are not
experts in R package or code development. Moreover,
many packages in the R ecosystem for population genet-
ics are the result of ongoing research and thus evolve
dynamically, which necessitates automatic testing for
quality control. The contribution model we adopted to
achieve these goals, known as ‘GitHub flow’, has been
widely used in the open source software community for
a number of years (Ram 2013).
This team of authors is comprised of both experienced
software developers, and population genetics scientists
who are frequent users of R, but not experienced in col-
laborative or reusable code development. This provided
us with a unique opportunity to continuously test and
refine our contribution process by developing the initial
set of vignettes for our website under the same process
that others would use, and circumstances similar to those
in which a community contributor would likely be.
Although the GitHub Flow model proved well suited to
ensuring that contributed workflows were of sufficient
126 Z . N . K A M V A R E T A L .
quality, we found that it presented significant challenges
for those among us not familiar with open source devel-
opment. Many of these challenges were due to a dearth
of easily discoverable, well-written examples and non-
technical documentation of each step in the process. The
frequent feedback generated by following our own pro-
cess allowed us to adapt the GitHub Flow contribution
model to our needs by splitting it into 4 concrete steps
(Fig. 1), reducing the need for technical knowledge of git
and GitHub, and allowing contributors to focus on the
content of their workflows instead.
Even though our contribution process is currently
optimized with the tools available to work on any of the
major operating systems used by population genetics sci-
entists (MS Windows, Mac OSX, Linux), some challenges
with usability by nontechnical users remain. One of these
is ensuring that contributors are working with the same
versions of R and packages as those used in the software
environment in which the website vignettes are automat-
ically tested and rendered. This software environment is
purposefully codified in the form of a Docker image
(Boettiger 2015), for which the underlying Docker file is
openly available and version tracked. In theory, this
Docker image can reproducibly provide the same soft-
ware environment (Boettiger 2015), whether to the auto-
matic testing procedure, contributors to our resource or
population genetics in R users at large. However, setting
up and running Docker on different operating systems is
still not a simple task and still differs between systems.
To address this issue, the source repository for our web-
site can itself act as an installable R package, which will
trigger installation and, if necessary, update of all its
dependencies. This then allows users to render the vign-
ettes without having to run Docker containers.
By virtue of how we designed the contribution pro-
cess, the results of following it remain available, includ-
ing its beginnings, and how it fares in the hands of both
technically well-versed and nontechnical population
genetics scientists. For example, the submission history
of the variant selection signal vignette (https://github.
com/NESCent/popgenInfo/pull/121) shows how a ser-
ies of review comments led to incremental improvement
of the vignette text, code fragments and the choice of
analytical methods, and revealed a component missing
from the Docker software environment. It concludes
with acceptance and integration after a second reviewer
approves. The population differentiation based on
microsatellite data vignette (https://github.com/NES-
Cent/popgenInfo/pull/86), submitted by one of the
technically most experienced in the team, shows how the
process lends itself to the collaborative resolution of diffi-
cult technical issues, and how authors can actively
recruit and solicit reviews from domain experts to push
the process along.
How well the contribution process can scale in the
future remains to be seen. We built technical features
into the process that allow reviewers to focus their time
on the scientific and methodological substance of con-
tributed vignettes, rather than syntactic correctness of
the code. However, the social challenges of scaling up a
peer review system of course remain, such as how to
recruit additional reviewers, and how to manage expec-
tations for turnaround times. The strides made by other
community-sustained online educational resources fac-
ing similar challenges, such as Software Carpentry (Wil-
son 2014) and Data Carpentry (Teal et al. 2015), suggest
that addressing them is possible.
We developed a number of vignettes for analyses of
population genetic data using different types of molecu-
lar markers (DNA sequences, microsatellites and SNPs).
In those vignettes, the user will find general information
on how to calculate basic statistics, such as genetic diver-
sity, Hardy–Weinberg equilibrium and F-statistics. These
vignettes provide information about topics that directly
target biological questions such as population differenti-
ation, genetic distances and detecting signals of selection.
They serve as examples of how vignettes can be con-
structed, but more importantly, we seeded the site with
vignettes on basic population genetics summary statistics
because these are often the starting points of more
detailed analyses. We hope that the demonstration of
basic analyses will lower the activation energy required
for novices to perform population genetic analyses in R.
This initial list of vignettes is by no means an exhaus-
tive one (Table 1). There are many other biological ques-
tions that could be fully addressed using analytical tools
incorporated in R packages (e.g. parentage analysis,
Table 1 List of existing vignettes (as of 26 January 2016) in the
‘Population Genetics in R’ website
Type of Marker Main Topic and List of Analyses
Microsatellites Basic statistics – observed and expected
heterozygosity, Hardy–Weinberg equilibrium
Population differentiation – Fst, AMOVA,
DAPC
SNPs Basic statistics – observed and expected
heterozygosity, Hardy–Weinberg equilibrium
Population differentiation – Fst, unsupervised
clustering
Genetic distance (individual based) – based
on Euclidean distance, based on number of loci,
based on allele differences
Detection of the signal of selection from
genome scan (population based analysis)
Sequence data Population differentiation – pairwise
population differentiation, overall F-statistics,
Analysis of MOlecular VAriance (AMOVA)
© 2016 John Wiley & Sons Ltd
 RESOURCES FOR POPULATION GENETICS IN
signals of selection, clustering, effective population size
estimation, phylogeography and landscape genetics). We
invite other population geneticists to contribute to this
effort for the advancement of the community of empiri-
cal population geneticists, and we maintain a ‘wish list’
of candidate vignettes in the issue tracker that accompa-
nies our website’s source code repository (https://
github.com/NESCent/popgenInfo/issues?q=is%3Aope-
n+is%3Aissue+label%3AWishlist).
The community website we created joins a few other
curated community resources of bioinformatics methods
and workflows. The collection of R workflows published
by Bioconductor (https://www.bioconductor.org/
help/workflows/) is perhaps most similar in purpose,
but its scope is expressly for packages in the Bioconduc-
tor repository, which does not include many packages
frequently used for population genetics. Additionally,
some of the technologies and processes around which it
is built, in particular an opaque review process, and sub-
version, a client–server system for version control, do
not lend themselves well to the contribution process we
aimed to achieve. As Ram (2013) explains in detail, a dis-
tributed version control system offers particular benefits
to community contribution, open science and a transpar-
ent open review process, for which reason we purposely
chose to design our contribution process around Git for
version control, and GitHub for pull requests. In part,
this was to enable open peer review, and in part to allow
a process for contributing and integrating changes that is
more approachable by population genetics scientists at
large who work in R, rather than primarily R package
authors. A less similar but conceptually related commu-
nity resource is the PLoS Computational Biology Topic
Pages (Wodak et al. 2012). These offer in-depth discus-
sions of various analytical subjects in bioinformatics.
However, they are much broader in scope than the gap
we sought to fill, and focus much on theoretical consider-
ations, as opposed to demonstrating specific worked
examples. Our resource could, though, conceivably take
on a complementary role to Topic Pages on population
genetics subjects. For example, a Topic Page author
could complement their Topic Pages article with worked
examples contributed to our website as tested and peer-
reviewed vignettes.
Conclusions
The statistical programming platform R has become an
essential tool for conducting population genetic analyses
in the genomics era. In recent years, a rich ecosystem of
packages for population genetic analysis has arisen,
which has allowed analyses to be carried out entirely in R
(Kamvar et al. 2015b; Krueger-Hadfield & Hoban 2016).
So that the expanding number of scientists conducting
© 2016 John Wiley & Sons Ltd
R 127
population genetics analyses can more readily come up
to speed with how best to utilize the various available
resources, we developed a community website of nar-
rated, peer-vetted and tested workflows, organized by
research use case. The site blends the rapid publishing
and openness of blogging with the peer review of tradi-
tional publishing to provide free educational material
that can be widely disseminated. Unlike articles in a tra-
ditional journal, this resource can continuously evolve
with the field. For population geneticists developing new
methods, it can be a venue to showcase their methods in
a real world, reproducible analysis.
Beyond the value of the website content itself, the
community contribution, automatic testing and repro-
ducible rebuild processes we designed may become use-
ful to other communities as templates for creating their
own open community-sustained educational resources.
Every building block we used is either freely available as
open source, or offered at a sufficiently feature-rich level
for free by ‘freemium’ platform-as-a-service providers
(here, GitHub for hosting, and Circle CI for automatic
testing and site rebuilds).
Finally, in the future, the resource we created could
be extended to enhance open education, specifically
content, tools and software that support free learning
and education practices by providing explicit exam-
ples of analyses that will not go out of date (Smith
2009). Even though the interactive platform we have
developed in this website does not replace theoretical
lectures in population genetics, it provides useful
materials to demonstrate analytical approaches in pop-
ulation genetics using R. We hope that this central
repository of materials will help demystify population
genetic analysis in R and help to advance the practice
of reproducible research.
Availability
The website can be found at http://popgen.nescent.org
and the source repository for contributors is located at
https://github.com/NESCent/popgenInfo. A snapshot
of the site’s source code, including the data sets used by
its vignettes, was archived at the time of sub- mission
and is available at http://dx.doi.org/10.5281/ zenodo.
48274
Acknowledgements
The resource reported in this study started at the Population
Genetics in R Hackathon, which was held in March 2015 at the
National Evolutionary Synthesis Center (NESCent) in Durham,
NC, with the goal of addressing interoperability, scalability and
workflow building challenges for the population genetics pack-
age ecosystem in R. The authors were participants in the
128 Z . N . K A M V A R E T A L .
hackathon and are indebted to NESCent (NSF #EF-0905606) for
hosting and supporting the event.
References
Andrews KR, Luikart G (2014) Recent novel approaches for population
genomics data analysis. Molecular Ecology, 23, 1661–1667.
Baumer B, Cetinkaya-Rundel M, Bray A, Loi L, Horton NJ (2014) R Mark-
down: integrating A Reproducible Analysis Tool into Introductory
Statistics. Technology Innovations in Statistics Education, 8, uclastat_cts_
tise_20118.
Benestan L, Ferchaud A-L, Hohenlohe P et al. (2016) Conservation geno-
mics of natural and managed populations: building a conceptual and
practical framework. Molecular Ecology, doi:10.1111/mec.13647.
Blischak JD, Davenport ER, Wilson G (2016) A Quick Introduction to Ver-
sion Control with Git and GitHub. PLoS Computational Biology, 12,
e1004668.
Boettiger C (2015) An introduction to Docker for reproducible research.
ACM SIGOPS Operating Systems Review, 49, 71–79.
Boettiger C, Chamberlain S, Hart E, Ram K (2015) Building Software,
Building Community: lessons from the rOpenSci Project. Journal of
Open Research Software, 3, 1.
Excoffier L, Heckel G (2006) Computer programs for population genetics
data analysis: a survival guide. Nature Reviews. Genetics, 7, 745–758.
Gandrud C (2015) Reproducible Research with R and R Studio. Chapman
and Hall/CRC, Boca Raton, Florida.
Jombart T (2008) adegenet: a R package for the multivariate analysis of
genetic markers. Bioinformatics, 24, 1403–1405.
Jombart T, Ahmed I (2011) adegenet 1.3-1: new tools for the analysis of
genome-wide SNP data. Bioinformatics, 27, 3070–3071.
Kamvar ZN, Tabima JF, Gr€ unwald NJ (2014) Poppr: an R package for
genetic analysis of populations with clonal, partially clonal, and/or
sexual reproduction. PeerJ, 2, e281.
Kamvar ZN, Brooks JC, Gr€ unwald NJ (2015a) Novel R tools for analysis
of genome-wide population genetic data with emphasis on clonality.
Frontiers in Genetics, 6, 208.
Kamvar ZN, Larsen MM, Kanaskie AM, Hansen EM, Gr€ unwald NJ
(2015b) Spatial and temporal analysis of populations of the sudden
oak death pathogen in Oregon forests. Phytopathology, 105, 982–989.
Krueger-Hadfield SA, Hoban SM (2016) The importance of effective
sampling for exploring the population dynamics of haploid–diploid
seaweeds. Journal of Phycology, 52, 1–9.
Luikart G, England PR, Tallmon D, Jordan S, Taberlet P (2003) The power
and promise of population genomics: from genotyping to genome
typing. Nature Reviews. Genetics, 4, 981–994.
Paradis E (2010) pegas: an R package for population genetics with an
integrated-modular approach. Bioinformatics, 26, 419–420.
Paradis E, Claude J, Strimmer K (2004) APE: Analyses of Phylogenetics
and Evolution in R language. Bioinformatics, 20, 289–290.
R Core Team (2015) R: A Language and Environment for Statistical Comput-
ing. R Foundation for Statistical Computing, Vienna, Austria.
Ram K (2013) Git can facilitate greater reproducibility and increased
transparency in science. Source Code for Biology and Medicine, 8, 7.
Schliep KP (2011) phangorn: phylogenetic analysis in R. Bioinformatics,
27, 592–593.
Smith MS (2009) Opening education. Science, 323, 89–93.
Teal TK, Cranston KA, Lapp H et al. (2015) Data carpentry: workshops to
increase data literacy for researchers. International Journal of Digital
Curation, 10, 135–143.
Whitlock MC, Lotterhos KE (2015) Reliable detection of loci responsible
for local adaptation: inference of a null model through trimming the
distribution of FST. American Naturalist, 186(Suppl 1), S24–S36.
Wickham H (2015) R Packages. O’Reilly Media, Sebastopol, California.
Widgren S et al. (2016) git2r: Provides Access to Git Repositories. R package
version 0.14.0. URL https://CRAN.R-project.org/package=git2r.
Wilson G (2014) Software Carpentry: lessons learned. F1000Research, 3, 62.
Wilson G, Aruliah DA, Brown CT et al. (2014) Best practices for scientific
computing. PLoS Biology, 12, e1001745.
Wodak SJ, Mietchen D, Collings AM, Russell RB, Bourne PE (2012) Topic
pages: PLoS Computational Biology meets Wikipedia. PLoS Computa-
tional Biology, 8, e1002446.
Xie Y (2013) Dynamic Documents with R and knitr. Chapman and Hall/
CRC, Boca Raton, Florida.
Z.K., M.L., S.C., N.G., H.L., and S.M. conceived and dis-
cussed the concept and framework for the project, Z.K.,
and H.L. implemented the underlying infrastructure,
Z.K., M.L., S.C., N.G., H.L., and S.M. wrote, reviewed,
and revised the current manuscript and vignettes.
Supporting Information
Additional Supporting Information may be found in the online
version of this article:
Table S1. Technical glossary of git and GitHub computational
terms.
© 2016 John Wiley & Sons Ltd