DOI: 10.1111/2041-210X.
12947
EDITORIAL
Accessibility, reusability, reliability: Improving the standards for
publishing code in Methods in Ecology and Evolution
Robert P. Freckleton
Department of Animal & Plant Sciences, University of Sheffield, Sheffield, UK
Correspondence
Robert P. Freckleton
Email: r.freckleton@sheffield.ac.uk
Since we launched Methods in Ecology and Evolution in 2010, a key
objective of the journal has been to improve the communication and
uptake of new methods. To achieve this, we have launched a number
of initiatives aimed at enhancing the dissemination of methods. At the
same time, over the past 7 years, there has been an almost seismic
shift in attitudes of ecologists and evolutionary biologists: it is increas-
ingly accepted that papers are not the only outputs of research; the
data and other components of the research are being recognised as
extremely significant. Consequently, many journals now have explicit
policies around data sharing.
In Methods in Ecology and Evolution, our published papers are often
the least significant component of the output: the tools that authors
produce to enable researchers to use their methods are more important
in many cases than the actual published paper. Primarily, this relates to
computer tools: code, scripts, applications, spreadsheets or webtools.
There are new and unique challenges that we face as an edito-
rial team in publishing papers that rely heavily on such outputs, most
importantly how do we ensure quality? Ecologists and evolutionary
biologists are not software engineers. When we launch our word
processor or operating system we are fully confident that it will run
flawlessly without crashing because it is produced by a team of profes-
sional programmers and extensively tested before release. Ecologists
and evolutionary biologists do not have the resources available to con-
duct such extensive testing. How then do we ensure that quality is
ensured? And in publishing papers that are so reliant on code, how
does the journal ensure that it meets acceptable standards in terms
of quality? How do we know that the tools we publish are reliable and
can safely be used by our community? The aim of the code guidelines
we are publishing is to allow us to address this issue, as well as to max-
imise the accessibility and usability of what we publish.
1 | REVIEWING CODE
When reviewing a paper, referees usually look at methodology and
are able to make a judgement on whether acceptable procedures
4  |  wileyonlinelibrary.com/journal/mee3
© 2018 The Author. Methods in Ecology and
Evolution © 2018 British Ecological Society
have been followed, and then whether the conclusions drawn are
justifiable given the limits of the methods followed. They cannot (in
all but exceptional cases) repeat the work themselves and have to
rely on work meeting accepted standards in terms of rigour of the
methodology.
The same principles are true for code: referees and editors cannot
work through code line by line. Nor can they undertake extensive test-
ing. Of course we would expect that referees will have access to code
outputs, and it is reasonable to expect that they can run the code easily.
However, the key to reviewing code is that authors should follow good
practice in terms of following protocols that minimise the chances of
major errors arising. The key elements of this are: (1) Benchmarking
data: such data are used to show how a method works from first princi-
ples and can be run through code to ensure that it is producing expected
outputs; (2) unit testing: in complex and simple applications alike, unit
testing ensures that individual components function as expected.
Although we will not mandate that authors follow a particular ap-
proach (we are publishing ‘guidelines’ after all), editors and referees
will be given discretion to recommend revision or reject if code does
not meet reasonable standards of preparation. We also hope that ref-
erees, when writing their comments, will provide specific comments
about how the standards of code can be improved.
2 | SHARING AND STORING
A unique challenge in publishing code is that much code is ‘alive’. In
many cases, developers improve their code through time or fix bugs
so that the version referred to in a published paper may be outdated.
Version control and documentation are important to ensure (1) a user
knows which version is being used; (2) what are the changes between
successive versions and how does this affect outputs?
This dynamic nature of code begs the question of how it should
be stored or shared? There are many answers to this question, and
the landscape changes through time. We will not mandate a specific
platform, but reviewers and editors can reasonably ask whether the
Methods Ecol Evol. 2018;9:4–6.
EDITORIAL
on
code is shared and archived in an appropriate place, with access to all
versions of the code.
Some kinds of code cannot be easily shared: for example, a script
that runs via commercial software would not be readily shareable. Nor
are pre-­compiled applications for which the underlying source code is
not shared. We would not normally consider such code for publication.
We are also reluctant to consider commercial applications because the
publication of this generates potential conflicts of interest for authors
as well as issues of accessibility for readers.
A final component of the ability to share code is its licensing. We
encourage authors to (1) always pick a license for their code, and (2) fa-
vour licenses that are approved by the Open Source Initiative. Choosing
a license protects author’s intellectual property, and allows users to
understand what they are allowed to do with the code.
3 | FINAL COMMENTS
As mentioned above out policy is a series of principles underpinned
by guidelines. We are not publishing a mandate, but reviewers and
editors will be given the discretion to recommend revision or rejec-
tion if they feel that code does not show evidence of good practice.
The objective in this is to ensure that papers we publish link to high
quality outputs that users can be sure has been scrutinised and meets
accepted standards of good practice.
METHODS IN ECOLOGY AND EVOLUTION:
POLI CY ON PUBLISHING CODE
The overarching objective of our policy is to ensure that high qual-
ity code is readily available to our readers, ideally open source. The
key principles we intend our policy to ensure are Quality, Usability,
Accessibility and Functionality. More specifically:
Quality Through being published in MEE, readers and users should
have confidence that peer review has been used to ensure that there
is evidence of high standards of code preparation. By necessity, this
cannot include extensive testing, but reviewers will look for evidence
of good coding practice.
Usability Code needs to be usable, so that readers are able to
implement methods readily. This includes both the implementation
and the platform. We will discourage the use of proprietary software
where possible in favour of ones that respect established standards
for software freedom (e.g. https://www.gnu.org/philosophy/free-sw.
html). The implementation should include documentation and exam-
ples with benchmarking data where possible.
Accessibility We require a version of record: this should be the exact
version of the code described in the journal and able to replicate all
results presented. However, we recognise that code evolves and en-
courage the use of platforms that permit updating with version control.
Functionality Functionality needs to be novel and add to what is
already available. Re-­implementations of existing methods are not
Methods in Ecology and Evolu
      5|
published, except where they take advantage of new methods permit-
ting significant computational gains.
We are an academic journal and we have to recognise that our
primary purpose and capability is to ensure rigorous peer review and
the integrity of the publication process. However as part of the British
Ecological Society, we have an objective to communicate ecological
science and the above principles are intended to additionally ensure
that the community is benefitting from well-­
communicated out-
puts that are accessible, as well as being designed to promote good
­research practice in the community.
WHAT DO WE PUBLISH?
Our primary code outputs are applications papers. Although not restricted
to code, a high proportion of applications describe a range of different
tools. The journal has a set of expert Associate Editors who deal with
these papers, overseeing the review process and making recommenda-
tions to the senior editors. Applications papers describe the functionality
of a tool and are intended to act as an overview of the application as well
as to act as a citeable source. Ideally the paper should succinctly describe
what the code does including information relevant for both users and
future developers. As described below, good coding practices will ensure
that code is robust and able to be rigorously reviewed.
We also publish a range of other code outputs, although not a­ lways
as the primary output of a paper. These include:
1. Scripts: many paper in MEE and other journals include scripts
that perform data analysis and may be of interest to readers
or users wishing to analyse their own data.
2. Spreadsheets: although not a common output, in some disciplines
spreadsheets are an output that are particularly valuable for end
users.
3. Web pages: some tools, particularly those underpinned by large
datasets or databases, will be available through a web interface.
There are some outputs that we are reluctant to consider:
1. Proprietary programs that do not provide access to source code.
In the absence of source code, reviewers are unable to view
details of implementation. There are also issues of compatibility
across platforms for pre-compiled applications. We of course
recognise that some applications need to be compiled before
they can be used, but in these cases, source code should be
provided.
2. We will not normally publish applications that are being developed
commercially for sale. Commercial interests generate potential
conflicts of interest and issues of accessibility, and we are unable
to publish code that is being distributed commercially. We do not
rule out, however, comparisons with commercial outputs however
we would not normally publish code that replicates commercial
packages simply because it is free.
 | Methods in Ecology and Evolu
on
6       EDITORIAL
ENSURING QUALITY THROUGH PEER REVIEW
We recognise that it is difficult for reviewers and editors to work line-­
by-­line through submitted code. In reviewing code, we therefore en-
courage the adoption of coding practices that facilitate the writing of
robust code. These include:
1. Benchmark data: these should be included and where possible
worked through from first principles as well as using code. For
example, code may implement a new statistical method: it would
be ideal to provide a worked example in the text/appendix of
a paper that is then replicated by running the code.
2. Unit testing: for more complex applications unit testing is a best-
practice approach for testing components and demonstrating that
they perform as expected.
3. Documentation: thorough documentation of code will facilitate
understanding of overall structure for reviewers and users.
4. Version control: clear history of code with revisions tracked and
documented will make clear how bugs are fixed and how subse-
quent versions of code relate to that originally submitted.
This is not an exhaustive list: the overall philosophy is that robustly
written code will be easier to review and revise in the peer review as
well as subsequently.
The above apply primarily to applications. However the same prin-
ciples apply to other code outputs: essentially code, where provided,
should be written using good practice in terms of design and testing,
and be well presented permitting peer review and eventual reuse.
WHERE SHOULD CODE BE STORED/SHARED?
There are a variety of platforms for storing and sharing code. We rec-
ommend that code is shared via such a platform. This should ideally
permit version tracking so that the version of record can be clearly
identified. Some platforms now permit a DOI to be assigned to code
and we recommend this where possible for ready identification and
tracking.
As mentioned above, some code needs to be compiled before
being run. Developers should share the source code and instructions
for compiling: our experience is that it cannot be guaranteed that
pre-­compiled applications will run on a given platform, and these may
present security concerns in any case.
All code submitted to MEE must include a fully open source soft-
ware license. This is because current copyright law prevents code
sharing by default, thus necessitating a license explicitly allowing code
sharing and reuse. Typically, the license text resides in a text file (com-
monly titled LICENSE.txt) with the code. A well-­established list of such
licenses can be found here: https://www.gnu.org/licenses/license-list.
html#SoftwareLicenses Paper submissions including code that does
not have accompanying open source licenses cannot be considered
for publication.
FINAL COMMENTS
Our primary aim is to ensure scientific integrity through high qual-
ity through peer review. This policy document highlights a series of
criteria we will be working to and to indicate in broad terms what
we expect of authors. Code outputs are heterogeneous, and it is not
possible or desirable to be completely prescriptive. We recognise
that good practice described here will greatly increase the quality of
code as well as facilitate better peer review and this will fulfil the jour-
nal’s dual aims of ensuring high quality science and communicating
ecological research. http://besjournals.onlinelibrary.wiley.com/hub/
journal/10.1111/(ISSN)2041-210X/author-guidelines.html
ORCID
Robert P. Freckleton  http://orcid.org/0000-0002-8338-864X