Publishing, Promotion

Peer-Review & Pizza
Wherein we do a deep dive into scholarly
publishing, why it makes and breaks careers, why
peer-review is so important/flawed, and how
everything is changing.
Andrew Colgoni, MSc., MLIS
Services Librarian, Thode Library


Introduction to scholarly publishing
How it works (generally)
What happens when it doesn’t work
How publishing is changing

So, why does everyone talk
about publishing so much,

• It’s how we communicate our research.

• Also including: conference presentations and posters

• It’s how we understand what others are doing

• What else is happening in your field?
• Who are the experts? Opportunities to collaborate?

• Publications are the currency of academic careers.
• Ongoing publishing required to maintain standing:

• Think: moving to new jobs & promotions, securing grants

“It is generally accepted within the university community that an
assessment by other scholars working in the same field, or
closely related fields, is the best way of determining the quality
of scholarly work. This assessment finds expression in the
acceptance of papers and manuscripts for peer-reviewed
publication, in academic awards and the approval of research
grants, in invitations to present conference papers and
university seminars… The weights to be assigned to these
different forms of assessment will vary from discipline to
discipline and from one decision to another. But the awarding of
tenure and the granting of promotion will require that an
appropriate assessment of the candidate’s scholarly
achievements has been made by carefully chosen peers and that
they have attested to the high quality of the candidate’s work.”

Index of h: has published h papers each of which has been cited in other papers at least h times.
i10-Index = the number of publications with at least 10 citations.
Much, much more:

Where do I publish?
Anywhere you can (with a caveat)
• Undergraduate journals

• Ideal for getting a publication under your belt & learning process
• Put on CV – should note that it is a student-run publication

• Highly-visible, general-science journals
• E.g., Science or Nature
• Highly “impactful” – often “worth” more

• Subject-specific journals

• Where most research is published

Publish in high quality
• Where do key authors publish?
• Content of your work
• Reputation/expectation in field
• UlrichsWeb
• The Journal Guide
• Directory of Open Access Journals (DOAJ)
• Web of Science Journal Citation Reports
• ScImago Journal and Country Rank
Robert De Bock. Magnifying Glass. [Photograph]. (2016). Retrieved October 18th, 2016

Impact Factor (IF)
• measures the ranks / ratings of a journal’s
contents compared to other journals
• used to determine relative “impact” of
journals within a field

# of citations of journal in X year / # of articles
published in journal in year X-1 and X-2
• updated yearly

So, why does everyone talk
about peer-review so much,

Why Peer Review?
Peer-reviewed work is the only form of published
research that is universally respected
• In-field colleagues
• Your employers
• Out-of-field scientists
• Research funding
• Government ministries /
• Ultimately, the public.

History of Peer Review
• Pre-1700s, editorial control was the norm, censorship rather than
• In 1752, the Royal Society of London created a “Committee on
Papers” to review and select texts for Philosophical Transactions
(maybe RS of Edinburgh in 1731?) (Fitzpatrick, 2009)
• “It was appreciated from the start that the peer review process could
not authenticate or endorse because the editors and reviewers could
not be at the scene of any crime… the journals from the beginning
threw the ultimate responsibility for the integrity of the article
squarely upon the author” (quoted in Fitzpatrick, 2009)
• Peer review as we know it now began in 1940s-1960s, when copies
became easier to make (Spier, 2002). (Consider that Watson and Crick were not



Adapted from Wiley (2016


by Editor

Sent to


by Editor

Published in
the Journal





Content of Referee Reports
Questions that the referees will be asking themselves:
• Is the work understandable and correct?
• Sufficient evidence?
• Appropriate references?
• Is it original and interesting?
• Is it well presented?
• Is the English understandable?
• Copy-editors usually deal with minor language problems
• Referees should make sure that the scientific meaning is clear

Handling reviews
• Don’t take them personally.
• Take a deep breath.
• Your job is to address each
point the reviewers make
and either make the
correction OR explain why
you don’t need to
• You get to make a case to
the Editor

Forms of Peer Review
Single Blind

Double Blind

Author does not know identity
of reviewers

Neither authors nor reviewer
know each other’s identities

• Some anonymity allows
reviewer honesty
• Potential for bias or

• Bias (potentially removed)
• Limits to anonymity

Forms of Peer Review

All parties know identities
involved, including external

• More accountability and
civility, plus incentive to do
better review
• Reluctance to fully criticize
powerful players

Changing peer-review

Open models
- Interest to reader or journal
fit not considered
- Continuous publication
- Multiple metrics (page views,
shares, comments)
- Feedback built in

ArXiv (
- Open repository for highenergy physics, math
- Pre-prints, post-prints,
unreviewed papers
- Endorsement system
- (see also:

Criticisms of Peer Review
• Peers cannot be conversant with entire literature
• Reviewers biased to results that match own expectations
• Biased against innovation
• Institutional biases (prestige)
• A few peers may not be good judges of quality
• Competitiveness in smaller areas of science
• Cannot (or is not designed to) detect concealed deceptions
(from Shatz, 2004)

When publishing goes bad.

Why do articles get retracted?
• Duplication of existing work / already published
• Permissions from authors / contributors not
• Error, manipulation, fabrication, falsification in
the science / data
• Plagiarism
• Ethics not secured

The Top 10 Retractions of
2014, from the editors of
Retraction Watch

Changing publishing

Tri-Agency Funding
(NSERC, etc.)

Government funds scientific
research with taxpayer





Library buys access to article

to Journal

Open Access
• A response to the broken system of scholarly publishing
• Canadian governmental funding (Tri-Agency: CIHR,
NSERC, SSHRC) require peer-reviewed publications be
made openly accessible within 12 months of publication.
• How?
• Publish in an open-access journal
• Pay the publisher to make the article open
• Deposit into an online repository (institutional or disciplinary)

• Publishing and communicating science is important for a
variety of reasons.
• It’s good to start early and get a publication on your CV. This
carries a lot of weight, particularly as an undergraduate.
• Peer-review as it is commonly practiced is the best we’ve got,
but expect to see more alternative approaches in the future.
• Open access is changing the existing publishing system, and
making your research open can increase its impact

• Fitzpatrick, K. (2009). Planned Obsolescence: Publishing,
Technology, and the Future of the Academy, NYU Press: New
York University, as excerpted on Mediacommonspress
• Marcus, A. and Oransky, I. (2014). The Top 10 Retractions of
2014. The Scientist.
• Shatz, D. (2004). Peer Review: A Critical Inquiry. Rowman &
• Spier, R. (August, 2002). Trends in Biotechnology, Vol. 20, Iss.
Thanks to Ali Versluis for Slides 8 & 9 & moral support from Meghan
Ecclestone and Abeer Siddiqui