1
Design memo: Verifying information from the crowd
By Ushahidi ethnographer, Heather Ford
1
10 November, 2011This memo introduces the concept of verification, how it has evolved at Ushahidi andin sample deployments, alternative ways of thinking about verification and somesuggestions for further research.With over 20,000 installations of Ushahidi and Crowdmap since January, 2009,Ushahidi has been used in a number of different contexts
–
from earthquake supportin Haiti, to reports of sexism in Egypt, to election monitoring in the Sudan. In each of these cases, a map is publicized and individuals are encouraged to send reports to it.The process of verifying information reported by the crowd has taken on a variety of different forms depending on the needs and affordances of the environment and thecommunity supporting it.Ushahidi support for verification has until now been limited to a fairly simplebackend categorisation system by which administrators tag
reports as “verified” or “unverified”. But this is proving unmanageable for large quantities of data and may
not be the most effective way of portraying the nuanced levels of verification that canpractically be achieved with crowdsourced data.What research needs to be done to test verification alternatives so that Ushahidi andCrowdmap deployers are provided with due diligence tools that can advance trust intheir deployments?
Can we do this in a way that doesn’t add any new barriers to ent
ryto those who need to have their voice heard on Ushahidi? How can we ensure that thissolution is as close as possible to the needs, incentive systems and motivations of deployers and users? What is the next step for Ushahidi verification?1.
The origins
of the “verified/unverified” categories
2.
How verification works on Ushahidi3.
How others “do” verification
4.
Designing for verification5.
Further research
1.
The origins of the “verified/unverified”
categories
On the 27
th
of December 2007, President MwaiKibaki was declared the winner of the hotlycontested Kenyan presidential elections. Citingwidespread electoral irregularities, members of theopposition protested. Peaceful protests soon turnedinto targeted ethnic violence as a wave of violenceswept through the country. Kenyan lawyer,blogger and technology policy expert, Ory
1
Special thanks to Jessica Heinzelman, David Kobia, Nigel McNie, Tim McNamara,Ari Olmos, Kamal Sedra, Zein for their input on how verification is being practiced intheir projects.
verify1.To prove the truth of bypresentation of evidence ortestimony; substantiate.
2.To determine or test the truthor accuracy of, as bycomparison, investigation, orreference:
experiments that verified the hypothesis.
Heritage® Dictionary ofthe English Language, FourthEdition
2Okolloh, was in the country at the time of the violence and was providing on-the-ground citizen reporting from Nairobi through her blog, one of the main sources of information at the time
2
. She initially made a request for people to send her storiesthat were going unreported by the media (there was a government ban on local newsmedia at the time) but quickly became overwhelmed by the volume of reports.When she reluctantly left for Johannesburg with her family on January 3, Okollohmade a plea to her readers and friends to build tools to help document what washappening in the country. The Internet was still available and so Okolloh made twosuggestions to help continue the work that she had been doing before she left.
(I)t also occurs to me that we have no reliable figures of the real death tolls on the ground.Perhaps we can begin to collect information from organizations and individuals on the grounde.g. red cross, hospitals, etc. and start to build a tally online, preferably with names. Most of the people losing their lives will remain nameless, and it might be worthwhile to at leastchange that. Any volunteers/ideas?
3
One of the developers who responded to Okolloh’s post was David Kobia, a Kenyan
living in Atlanta in the U.S. who was relieved to have an opportunity to provide someassistance. Kobia, developed the platform and called it Ushahidi (
meaning “witness”
in Kiswahili) enabling people to send in reports of what was happening on the groundvia SMS or the website platform. Kobia recalls that the need for verification emergedonly two or three days after the site had been launched.
“
There was a degree of naivety when you start with five reports, but as you getinundated with 500 text messages, then you think that there needs to be someverification process in place
…
Getting verified information becomes really critical during crises like Kenya.This was really problematic because people were sending text messages tostart rumors
…
An example would be something like: "Some politician hasbeen assassinated". This could have a serious reverberating effect and so itwas important to be sensitive to the situation. You had to vet information andgo back and overlay it with mainstream media
…
We ended up verifying fewerand fewer reports and putting less up on the map.
”
4
According to Kobia, Ushahidi worked with NGOs, aid agencies and volunteers on theground to verify reports. When the violence subsided and others wanted to use thetools, the Ushahidi team ramped up development to enable others to use the opensource software in their own, independent deployments and then developedCrowdmap, a platform that enables users to set up their own deployment of Ushahidiwithout having to install it on a web server.
2
Okolloh, O. R. Y. (2009). Ushahidi, or “testimony”: Web 2.0 tools for crowdsourcing crisis
information 9.
Participatory Learning and Action
,
59
(1), 65-70.
3
Ory Okolloh, “Kenyan Pundit”, 3 January, 2008
4
Interview, 22 September 2011
3
2. How verification works on Ushahidi and Crowdmap
The way that verification currently works in Ushahidi and Crowdmap is that reportsare pushed and/or pulled into a map either by an individual typing up a report andsending it to a specific deployment via the website or by SMS and/or by organisations
and individuals’ reports being pulled into the map
by virtue of the hash tags that theycontain on Twitter. Reports are not automatically visible on the map but need to beapproved by the deployment team. After they have been approved, reports becomevisible on the public map
, and are by default marked as “
unverified
”
until they have
been checked as “
verified
”
by the deployment team. Reports that the deployersbelieve are
inaccurate are checked as “unverified”
before they leave the verificationqueue.In November last year, the ICT4Peace Foundation worked with Ushahidi to developthe Matrix plugin for the Ushahidi platform. The plugin enables deployers on theback-end to make judgements about the source reliability and information probabilityfor each report by choosing from a dropdown menu as seen below.The plugin assumes the presence of trained reporters in the field working from criteriaset by deploying organisations and consequently a separation between those reportingand those analysing the data (this is not always the case with Ushahidi deployments).The result is a matrix of reports highlighted according to the probability that they areaccurate.
Image: An example of the Matrix too in use on the site Uchaguzi TZ (fromhttp://blog.ushahidi.com/index.php/2010/11/04/analysis-plugin-ict4peace-supported-tool-for-ushahidi-deployers/ )
Ushahidi deployers using the verification functionality recognize that much of thework that they need to do happens outside the platform
–
phoning people on the
Search History:
Result 00 of 00
00 results for result for
p.
More From This User
Notes
Load more
