MEDIA & DESIGN

Research on data journalism:

What is there to investigate?
Insights from a structured literature review

Julian Ausserhofer1,2,3, Robert Gutounig1, Michael Oppermann2,

Sarah Matiasek1,2 & Eva Goldgruber1

1: FH Joanneum University of Applied Sciences, Graz
 NODA16 Academic Pre-Conference


2: University of Vienna
 #NODA16

3: Humboldt Institute for Internet and Society, Berlin 21.04.2016, University of Helsinki
MEDIA & DESIGN

* *
*

Supported by: *Tool evaluation partners

MEDIA & DESIGN

Research Interest
MEDIA & DESIGN
Research literature on data journalism

2013: "internalist tendencies at [... the] early

stage of academic research" (Anderson, 2013, p. 1007)
↓




2015: "an explosion in data journalism-

oriented scholarship" (Fink & Anderson, 2015, p. 476)*




"rapidly growing body" of scientific studies 


(Lewis, 2015, p. 322)*







*cited via Loosen, Reimer & Schmidt (2015, p. 2)

#NODA16 @julauss 4
MEDIA & DESIGN

Research questions

How is the research literature developing?

What are the research gaps?

#NODA16 @julauss 5
MEDIA & DESIGN

Method
MEDIA & DESIGN

Structured literature review

#NODA16 @julauss 7
MEDIA & DESIGN

Why a structured literature review?

"to develop insights, critical reflections,

future research paths and research questions" 

(Massaro, Dumay & Guthrie, forthcoming)

It adopts "a replicable, scientific and

transparent process [...] that aims to
minimize bias [...]"

(Tranfield, Denyer & Smart, 2003)

#NODA16 @julauss 8
MEDIA & DESIGN
Undertaking a systematic literature review

Defining the Determining Coding data Developing Developing

questions the type of insights and future
that the studies and critique research paths
literature carrying out a through and questions
review comprehensiv analyzing
e literature
should the dataset
search
answer

Writing a literature review protocol

Adapted from Massaro et al. (forthcoming)

#NODA16 @julauss 9
MEDIADefining
& DESIGNthe Determining Coding data Developing Developing
questions the type of insights and future
that the studies and critique research
literature carrying out a through paths and
review comprehensiv analyzing questions
e literature
should the dataset
search
answer

● Empirical research on DDJ

● Social science focus, but open to other
disciplines
● Published after 1995
Included Not included
Journal articles Bachelor's and Master's theses
Book sections Press reports
Conference papers Blog posts
Reports (from industry and
research projects)
PhD theses
#NODA16 @julauss 10
MEDIADefining
& DESIGNthe Determining Coding data Developing Developing
questions the type of insights and future
that the studies and critique research
literature carrying out a through paths and
review comprehensiv analyzing questions
e literature
should the dataset
search
answer

● Preliminary search with “data-driven

journalism”

● Extracting related terms from the keyword

section of research papers
#NODA16 @julauss 11
MEDIADefining
& DESIGNthe Determining Coding data Developing Developing
questions the type of insights and future
that the studies and critique research
literature carrying out a through paths and
review comprehensiv analyzing questions
e literature
should the dataset
search
answer

Search terms No search terms

algorithmic journalism accountability journalism
computational journalism crowdsourced journalism
computer-assisted reporting dataviz
data journalism datavis
data-driven journalism ddj
data-driven reporting drone journalism
database journalism investigative journalism
datajournalism online journalism
datenjournalismus open journalism
quantitative journalism

#NODA16 @julauss 12
MEDIADefining
& DESIGNthe Determining Coding data Developing Developing
questions the type of insights and future
that the studies and critique research
literature carrying out a through paths and
review comprehensiv analyzing questions
e literature
should the dataset
search
answer

Scientific Databases
ACM Digital Sowiport

EBSCO Springer

IEEE SpringerLink

JSTOR Taylor & Francis Online

ProQuest Web of Science

Science Direct Wiley

Scopus Google Scholar

Sociological Abstracts

#NODA16 @julauss 13
MEDIADefining
& DESIGNthe Determining Coding data Developing Developing
questions the type of insights and future
that the studies and critique research
literature carrying out a through paths and
review comprehensiv analyzing questions
e literature
should the dataset
search
answer

772 search results

↓
Assessment of title, abstract & keywords
- by two independently working researchers

(Thomas et al., 2004)

↓
33 research publications

#NODA16 @julauss 14
MEDIA & DESIGNRecords identified
from scientific
databases
(n= 772)

Preliminary corpus:
Excluded after
publications included
screening
after screening of
(n= 739)
records (n= 33)

Further publications
References from Excluded after
from expert poll of
preliminary corpus screening
data journalism
(n = 1151) (n= 1148)
researchers (n= 4)

Further publications
included after screening
of references
(n = 3)

Final corpus: Publications

included in the systematic
#NODA16 review (n=40) @julauss 15
Adapted from Fecher, Friesike & Hebing (2015)
 MEDIA & DESIGN

software-assisted qualitative
content analysis
(Kaefer, Roper, & Sinha, 2015; Mayring, 2000;
Schreier, 2012; QSR International, 2015)

computational analysis of
structural aspects

(Kreibich, 2016; Lopez, 2009)

#NODA16 @julauss 16
MEDIA & DESIGN

Results
MEDIA & DESIGN

Development of the literature over time

n=40
#NODA16 @julauss 18


Publications by type and citations
MEDIA & DESIGN

n=40
bubble size = number of citations in Google Scholar
#NODA16 @julauss 19
Affiliations & collaborations
MEDIA & DESIGN

#NODA16 @julauss 20


References per year
MEDIA & DESIGN

1787-2015

n=1644

#NODA16 @julauss 21
Most-cited references
MEDIA & DESIGN

Nr. of
Publication
Citations

Meyer, P. (2002/1973). Precision journalism: A reporter’s introduction to social

15
science methods (4th ed.). Oxford: Rowman & Littlefield.

Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good:
“Computer-assisted-reporters” and “programmer-journalists” in Chicago. 15
New Media & Society, 15(6), 853–871.

Gray, J., Bounegru, L., & Chambers, L. (Eds.). (2012). The data journalism
handbook: How journalists can use data to improve the news. Sebastopol: 13
O’Reilly.

#NODA16 @julauss 22
Theoretical frames
MEDIA & DESIGN

● Science and technology studies

● Actor network theory

(Ausserhofer, 2015; De Maeyer, Libert, Domingo, Heinderyckx, & Le Cam, 2015;
Parasie & Dagiral, 2013; Parasie, 2015)


#NODA16 @julauss 23
Research designs & data collection methods
MEDIA & DESIGN

Method Nr. of studies

In-depth interviews 25

Content analysis 21

Survey 5

Short-term observation 3

Newsroom ethnography 1

Note. Content analysis includes analysis of news, databases, blogs, job ads,
visualizations, briefings, manuals, and more. Short-term observation encompasses visits
to the newsroom and participation in meetings. A newsroom ethnography is defined as a
detailed study of a newsroom over the course of several days.

#NODA16 @julauss 24
Geographical scope
MEDIA & DESIGN

Number of
Country
studies
United States 16

United Kingdom 14

Germany 5

International 3

n/a 3

Sweden 2

Switzerland 2

Norway 2

Netherlands 2

… …

#NODA16 @julauss 25
Research gaps in data journalism research
MEDIA & DESIGN

● Comparision of practices between countries 


(Appelgren & Nygren, 2014; Parasie & Dagiral, 2013)

● Long-term studies (Davenport, 2000; Knight, 2015)

● Newsroom ethnographies (Parasie & Dagiral, 2013)

● Software studies (Garrison, 1999; Lewis, 2013; Stavelin, 2013)

● Reader experience studies (Segel & Heer, 2010)

#NODA16 @julauss 26
MEDIA & DESIGN

Conclusion
Conclusion
MEDIA & DESIGN

● data journalism and its investigation has been

developing rapidly
● quality improvements in the research
● issues with the literature: few publications
refer to theory or methodology, just report
what has been investigated

#NODA16 @julauss 28
Research opportunities
MEDIA & DESIGN

● practices in small news organizations, freelancers, local

and mobile data journalism etc.
● gender
● digital methods: investigating the field through its
platforms
● theory
●…

#NODA16 @julauss 29
 MEDIA & DESIGN

Explore the literature online at:

http://literature.validproject.at

Julian Ausserhofer1,2,3,
 Robert Gutounig1,


julian.ausserhofer@fh-joanneum.at @sextus_empirico
@julauss
Michael Oppermann2,
 Sarah Matiasek1,2 &
 Eva Goldgruber1

@oppermann_m @sarahmatiasek @evagoldgruber
1: FH Joanneum University of Applied Sciences, Graz
 NODA16 Academic Pre-Conference

2: University of Vienna
 #NODA16

3: Humboldt Institute for Internet and Society, Berlin 21.04.2016, University of Helsinki
MEDIA & DESIGN

References

31
MEDIA & DESIGN
Anderson, C. W. (2013). Towards a sociology of computational and algorithmic journalism. New Media &
Society, 15(7), 1005–1021. doi: 10.1177/1461444812465137
Appelgren, E., & Nygren, G. (2014). Data journalism in Sweden: Introducing new methods and genres of
journalism into “old” organizations. Digital Journalism, 2(3), 394–405. doi:
10.1080/21670811.2014.884344
Ausserhofer, J. (2015). „Die Methode liegt im Code”: Routinen und digitale Methoden im
Datenjournalismus. In A. Maireder, J. Ausserhofer, C. Schumann, & M. Taddicken (Eds.), Digitale
Methoden in der Kommunikationswissenschaft (pp. 87–111). Berlin: digitalcommunicationresearch.de.
doi: 10.17174/dcr.v2.5
Davenport, L., Fico, F., & Detwiler, M. (2000). Computer–assisted reporting in Michigan daily newspapers:
More than a decade of adoption. Presented at the Association for Education in Journalism and Mass
Communication (AEJMC) National Convention, Phoenix, Arizona.
De Maeyer, J., Libert, M., Domingo, D., Heinderyckx, F., & Le Cam, F. (2015). Waiting for data journalism:
A qualitative assessment of the anecdotal take-up of data journalism in French-speaking Belgium.
Digital Journalism, 3(3), 432–446. doi: 10.1080/21670811.2014.976415
Fecher, B., Friesike, S., & Hebing, M. (2015). What drives academic data sharing? PLoS ONE, 10(2),
e0118053. doi: 10.1371/journal.pone.0118053
Fink, K., & Anderson, C. W. (2015). Data journalism in the United States: Beyond the “usual suspects.”
Journalism Studies, 16(4), 467–481. doi: 10.1080/1461670X.2014.939852
Garrison, B. (1999). Newspaper size as a factor in use of computer-assisted reporting. Newspaper
Research Journal, 20(3). Retrieved from http://com.miami.edu/car/baltimore1.htm
Kaefer, F., Roper, J., & Sinha, P. (2015). A software-assisted qualitative content analysis of news articles:
example and reflections. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research,
16(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/view/2123
Knight, M. (2015). Data journalism in the UK: A preliminary analysis of form and content. Journal of Media
Practice, 16(1), 55–72. doi: 10.1080/14682753.2015.1015801
Kreibich, C. (2016). scholar.py. Retrieved from https://github.com/ckreibich/scholar.py

#NODA16 @julauss 32
MEDIA & DESIGN
Lewis, S. C. (2015). Journalism in an era of big data. Digital Journalism, 3(3), 321–330. doi:
10.1080/21670811.2014.976399
Loosen, W., Reimer, J., & Schmidt, F. (2015). When data become news: A content analysis of data
journalism pieces. Presented at the The Future of Journalism 2015 Conference, Cardiff.
Lopez, P. (2009). GROBID: Combining automatic bibliographic data recognition and term extraction for
scholarship publications. In M. Agosti, J. Borbinha, S. Kapidakis, C. Papatheodorou, & G. Tsakonas
(Eds.), Research and Advanced Technology for Digital Libraries (pp. 473–474). Berlin: Springer. Retrieved
from doi: 10.1007/978-3-642-04346-8_62
Massaro, M., Dumay, J. C., & Guthrie, J. (forthcoming). On the shoulders of giants: undertaking a
structured literature review in accounting. Accounting, Auditing & Accountability Journal.
Mayring, P. (2000). Qualitative content analysis. Forum Qualitative Sozialforschung / Forum: Qualitative
Social Research, 1(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/
view/1089
Parasie, S. (2015). Data-driven Revelation? Epistemological tensions in investigative journalism in the age
of “big data.” Digital Journalism, 3(3), 364–380. doi: 10.1080/21670811.2014.976408
Parasie, S., & Dagiral, E. (2013). Data-driven journalism and the public good: “Computer-assisted-
reporters” and “programmer-journalists” in Chicago. New Media & Society, 15(6), 853–871. doi:
10.1177/1461444812463345
QSR International. (2015). Nvivo. Retrieved from http://www.qsrinternational.com/
Schreier, M. (2012). Qualitative content analysis in practice. Thousand Oaks: SAGE.
Segel, E., & Heer, J. (2010). Narrative visualization: Telling stories with data. IEEE Trans. Visualization and
Computer Graphics, 16(6), 1139–1148. doi: 10.1109/TVCG.2010.179
Stavelin, E. (2013). Computational journalism: When journalism meets programming (Dissertation).
University of Bergen, Bergen.
Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed
management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222.
doi: 10.1111/1467-8551.00375

#NODA16 @julauss 33