Open Science

Principles, Context,
Tools and Experiences

Research Data Management

November 1, 2018
René Bekkers
VU Amsterdam
 Closing the Age
of Competitive Science
In the prehistoric era of competitive science,
researchers were like magicians: they earned a
reputation for tricks that nobody could repeat and
shared their secrets only with trusted disciples. In
the new age of open science, researchers share by
default, not only with peer reviewers and fellow
researchers, but with the public at large. I discuss
recent developments in open science and its
implications for research careers and progress in
knowledge discovery.
 Science is
a global enterprise
for the benefit of humanity

What we do
serves
a public purpose

1 November 2018 Principles 3

 Core Idea of Open Science
The transparency of open science reduces

the temptation of private profit maximization

and the collective inefficiency
in information asymmetries

inherent in competitive markets

1 November 2018 Principles 4

 Open Science is not
• Magic – a trick you try to hide from the
audience to enchant them
• Haut cuisine – a secret recipe to sell seats in
your restaurant is not science
• A TedX talk to sell your book or increase your
commercial speaker rate

1 November 2018 Principles 5

1 November 2018 Principles 6
 The Open Science Cycle

Funds

Press Ideas

Paper Design

Data

1 November 2018 Principles 7

1 November 2018 Context 8
 https://paywallthemovie.com/paywall

1 November 2018 Context 9

 https://twitter.com/arvindsatya1/status/1053910355263545344?s=03
1 November 2018 Context 10
 https://twitter.com/Protohedgehog/status/1055771618910367744?s=03
1 November 2018 Context 11
 https://www.scienceeurope.org/wp-content/uploads/2018/09/Plan_S.pdf

https://www.nature.com/articles/d41586-018-06178-7

1 November 2018 Context 12

 Similar laws mandating
open access to
publicly funded
research are in effect in
Belgium, Italy, Spain, and
Ukraine

Get unpaywall browser

extension here.
If that does not work there
is always sci-hub

https://twitter.com/EFF/status/1053631928547532800?s=03
1 November 2018 Context 13
 https://twitter.com/AndrewGYork/status/965765089373515776?s=03
1 November 2018 Context 14
1 November 2018 Context 15
 https://twitter.com/russpoldrack/status/1052561117615013895?s=03

1 November 2018 Principles 16

 https://twitter.com/ceptional/status/1045424881494441984?s=03
1 November 2018 Context 17
From To
Closed Open
Competition Cooperation
Me search Research
Rat Race Slow Science
Let me try Let us retry
Magic Science

1 November 2018 Principles 18

1 November 2018 Principles 19
1 November 2018 Context 20
 Code of Conduct Principles
1. Honesty
2. Scrupulousness
3. Transparency
4. Independence
5. Responsibility

1 November 2018 Context: Principles 21

1 November 2018 Context 22
1 November 2018 Context 23
1 November 2018 Context 24
1 November 2018 Context 25
 4 varieties of ‘replication’
When you get to the same conclusions as
the original study, we call this:

Same Analysis Different analysis

Same Data Verification Robustness
“Many analysts, One Data Set”

Different Reproduction Generalization

“Many Labs” replication
Data projects

1 November 2018 Principles 26

 Silberzahn et al. (2018) https://doi.org/10.1177%2F2515245917747646

1 November 2018 Context 27

1 November 2018 Context 28
 Magic

1 November 2018 Principles 29

 “Data”

What A A modified
happened representation representation
= reality = raw data file = analysis file

1 November 2018 Principles 30

1 November 2018 Context 31
 http://ftp.iza.org/dp11796.pdf
1 November 2018 Context 32
 Unpublished?

Published

1 November 2018 Context 33

https://twitter.com/chrisdc77/status/952908767573807105
1 November 2018 Tools 34
 Optimism

1 November 2018 Principles 35

 Solitary Animals

1 November 2018 Context 36

 Socialization

1 November 2018 Principles 37

 https://twitter.com/RoninInstitute/status/1047298872727523328
1 November 2018 Context 38
 Brian Wansink,
Marketing,
Cornell University

Diederik Stapel,
Social psychology
RUG, UvA, Tilburg

Jens Förster
Social psychology
UvA, Bochum

Mart Bax, Cultural

anthropology VU
1 November 2018 Context 39

https://peerj.com/preprints/2748.pdf
Last week of September 2018,
Wansink resigned – due to the
persistence of Nick Brown,
James Heathers, Tim van der
 Zee and others

1 November 2018 Context 40

 https://twitter.com/NecroPants_/status/1046684556156256256
1 November 2018 Context 41
 Science is
• Open – in every step of the journey

• Learning from failure – publish null findings

• Slow – do not rush, 'calm down, it will save us'

1 November 2018 Principles 42

 We are all
standing on
the shoulders
of giants

1 November 2018 43
 We are all
standing on
the shoulders
of ants

1 November 2018 44
 Slow Science

1 November 2018 Principles 45

1 November 2018 Context 46
 Why we want open science
• Because it makes better science. When others
can see our mistakes, we become more
careful to avoid them.
• Because it speeds up progress in science.
Others can build on and use your work.
• Because we are paid by public money.
• Because funders demand it.

1 November 2018 Principles 47

1 November 2018 Tools 48
1 November 2018 Tools 49
 My OSF Projects

1 November 2018 Tools 50

 Global Trust Research Consortium
• René Bekkers, Bart Sandberg, Eric Uslaner,
Zhongsheng Wu, Tom van der Meer, Arjen de
Wit, Erik van Ingen, Lisanne de Blok, M. Kent
Jennings, Bert Bakker, Dustin Gilbreath, Rati
Shubladze, Matthew Eckel, Cengiz Enrisen,
Gino Pauselli, Thessalia Merivaki, Laura
Morales, Sonja Zmerli, Jonas Linde
• Open Science Framework:
https://osf.io/qfv76/
• You are most welcome to join us!
1 November 2018 Experiences 51
 Preregistration

https://twitter.com/UCBITSS/status/1047999978419830784
1 November 2018 Tools 52
 Preregister or it didn’t happen

https://www.nature.com/articles/d41586-018-07118-1
1 November 2018 Tools 53
 Experiences sharing data
• Many new co-authors, with weak ties
(including myself in the future)
• All file drawer projects still unfinished
• It takes work to make data sharable, especially
anonymization of interview data
• Data redistribution often not permitted
• Assign a DOI to your dataset, codebook,
figure, table, and preprint to make it citeable
1 November 2018 Experiences 54
 Other experiences
• Signed reviews: gratitude from some authors
whose work I’ve reviewed, editors ignore it
• CRAP: way fewer reviews - some editors will
ask authors to share data and scripts
• Preregistration: OSF and aspredicted examples
• Preprints: never a problem – not even sharing
published pdfs through my blog,
https://renebekkers.wordpress.com/

1 November 2018 Experiences 55