Ethical considerations in network

measurement papers

C. Partridge, M. Allman, CACM Oct 2015

A presentation based on the authors’

CACM contributed article
Network measurements and ethics
Network measurements
–  Typically at arm’s length from humans
–  So, not usual human-centered
model for evaluating practices
•  Review boards (IRB in the US) declare
most project “exempt” from full review

But our community finds

–  Our work increasingly has the potential to impact
humans’ well-being, yet
–  We are poorly prepared to address ethical issues
•  Beyond protecting privacy of communications content

2
Network measurements and ethics
The end result …
–  Well-meaning researchers on different pages
–  Papers not describing ethical reasoning behind set
of experiments
–  PCs that have to infer the ethical foundations of
the papers they are considering

3
 A very small sample

… PNAS is publishing an Editorial Expression of Concern

regarding the following article: … Questions have been raised
about the principles of informed consent and opportunity to opt
out in connection with the research in this paper. … June 2014

Statement from the SIGCOMM 2015 Program Committee:

The SIGCOMM 2015 PC appreciated the technical
contributions made in this paper, but found the paper
controversial because some of the experiments the authors
conducted raise ethical concerns. … August 2015

4
This article
A discussion of ethical issues with network
measurement (not human-subject) research
A straw-man proposal for an ethical
consideration section exposing ethical thinking
–  As a starting point for a discussion on ethics
–  A chance to justify/review the ethical foundations of
reported experiments

5
Background
Ethics in computing has evolved
–  But our increasing ability to extract information from
measurement data challenges key assumptions
Typical formulation distinguished between
–  Metadata – Revealing when and for how long two
parties communicated
–  Data that reveal content of the conversation with
higher expectation of privacy
But metadata is becoming content
–  e.g., Inter-packet gaps can be used to infer where
users’ fingers where on the keyword and, thus, the
letters in their password

6
Contours of harm
The authors’ focus is on
–  Ethical issues causing tangible harm to humans, not
network resources or equipment
–  Minimizing those issues, rather than eliminating them
Harm falls in an spectrum
–  E.g., how frequent do network probes have to be to
become a DOS attack? No clear lines
Sometime harm is
–  Indirect – Harm to people is a side effect, or
–  Potential – More needs to be done to cause harm
–  Researchers should still understand the ethical
implications of their experiments
7
 Collecting and using data
Active data collection can potentially inflict
direct, tangible harm
–  What about passive data collection?
–  What about using the “found data”?
E.g., Carna botnet data+
–  Leverage customers devices with guessable
passwords to gain illicit access and take
measurements that were publicly released
–  Data collection approach is a clear no-no
–  What about using the data already collected?
•  Whatever harm is already done, right?
•  Making the best of a bad situation?

8
+http://internetcensus2012.bitbucket.org/paper.html
Storing data
The measurement community encourages
–  Data preservation and data sharing – to facilitate
revisiting for validation, longitudinal analysis, …
Data sharing
–  How does a researcher determine if making the data
available is ethically OK? E.g., de-anonymization is
increasingly easy
Preserving data
–  How should it be kept? E.g., what level of encription
is “enough”?

9
The limitations of consent
A traditional way to go about this – Require
explicit consent from participants
Sometime this is possible (netalyzer, ono, ...),
if not easy
–  Logistics, language
Other is unclear who is being measured/affected
by measurements
Proxy consent is generally not allowed, although
we rely on it (e.g., measuring campus networks)
–  What if the probes “leave” campus?

10
A strawman proposal
Why an “ethical considerations” section?
–  Force authors to recognize ethical implications
–  Give explicit voice to ethical issues
•  Reviewers can directly evaluate and provide feedback on
–  Create public examples of good practices
•  Eventually leading to norms

A footnote on IRB is not enough :)

11
A strawman proposal – Guiding questions
For datasets directly collected, could the collection be
reasonably expected to cause tangible harm to people’s
well-being? If so, discuss measures taken to mitigate this

For datasets not directly collected, is there an ethical

discussion elsewhere? If so, cite, else discuss issues in
both collection and use

Using current techniques, can the data used in the study

reveal private or confidential information about
individuals? If so, discuss measures taken to prevent this

Other ethical issues specific to the work?

12
A strawman proposal – Other implications
Review forms will have to be updated
Community will need mechanisms to help
reviewers evaluate ethics
–  Guidelines, training, an ethic teleconference at the
start the reviewing process, …
PCs will need a clear philosophy to
–  Reject papers based on ethical considerations
–  Accept them with revisions
What does it mean “rejected on ethical grounds”?
–  Can you revisit? Can we mitigate the harm? Can we
prevent it in the future? …

13
Conclusions
We can’t prescribe ethical norms you can follow
Let’s instead be open about it and expose our
ethical thinking

An ongoing issue …
Workshop on Ethics in Networked Systems Research
Co-located with ACM SIGCOMM’15
http://conferences.sigcomm.org/sigcomm/2015/netethics.php

14