Welcome to Scribd, the world's digital library. Read, publish, and share books and documents. See more
Standard view
Full view
of .
Look up keyword
Like this
0 of .
Results for:
No results containing your search query
P. 1
Human Participant Research Guidelines

Human Participant Research Guidelines

Ratings: (0)|Views: 180|Likes:
Published by Crowdsourcing.org

More info:

Published by: Crowdsourcing.org on Jan 30, 2013
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
See more
See less


Revised September 2012; November 2012 
Human Participant Research Guidelines
Use of Crowdsourcing Services
Online labour markets, known as crowdsourcing, are becoming popular mechanisms forrecruiting potential research participants. Crowdsourcing is the act of outsourcing tasks to alarge group of people (a "crowd") through an open request via the internet in exchange formonetary remuneration. Crowdsourcing has become popular among social scientists as asource to recruit research participants from the general public for studies.Researchers studying the use of crowdsourcing have found that individuals who sign up tocomplete tasks tend to use the monetary remuneration they receive as a supplementary sourceof income. Some websites report the vast majority of individuals state they use the site casuallyas a way to spend time and be “at least semi-productive” (i.e., as a replacement for surfingFacebook™, etc.)These guidelines have been created to assist University of Waterloo researchers when planningonline studies using a crowdsourcing service. These guidelines detail information about themost common research ethical dilemmas researchers may face such as: contractualobligations required by the crowdsourcing service, recruitment and information-consent,inclusion/exclusion criteria, risks, use of deception or partial disclosure, appropriately estimatingthe time duration for the study, details concerning how to withdraw from the study,remuneration, privacy and confidentiality, and contacting participants. If there is new informationthat you have learned about a crowdsourcing service that would be helpful to other researchers,please contact the Office of Research Ethics so that we can include this information in theguidelines.
The most common crowdsourcing service used by University of Waterloo researchers is aservice based in the USA called Amazon Mechanical Turk. Rresearchers who are not citizens ofthe USA are not able to post a study directly to Amazon Mechanical Turk therefore many workthrough an intermediary service called Crowdflower. These two services are described in detailnext.
Amazon Mechanical Turk
This service started in 2005 mainly as a service to “crowdsource” tasks that require humanintelligence to complete (e.g., video or audio transcription). Groups, businesses, or individualswho use the crowdsourcing service Amazon Mechanical Turk are known as “requesters” andthey post tasks to be completed as HITs (Human Intelligence Tasks). “Workers” can browseamong the posted HITs and complete them for monetary remuneration set by the “requester”.“Workers” can be located anywhere in the world, but about 80% of “workers” using Amazon
Revised September 2012; November 2012 
Mechanical Turk reside in the USA or India. Remuneration provided for completing tasks can beredeemed by “workers” on Amazon.com via a gift certificate or be transferred to a bank account.University of Waterloo researchers recruit mainly USA participants.Some University of Waterloo researchers have reported that the vast majority of AmazonMechanical Turk participants complete surveys and tasks conscientiously and completely;typically about 90%. However, researchers should keep in mind that online participants oftenhave shorter periods of time and attention in which to complete a study in comparison to an in-lab participant and can be less tolerant to complete open-ended responses or complicated tasks(Buhrmester, Kwang, & Gosling, 2011).
Crowdflower, formerly known as Dolores Labs, located in San Franciso, California, was foundedto create tools to manage internet crowdsourcing. Crowdflower works with “requesters” to takelarge, data-heavy projects and break them into small tasks that are then distributed to variouson-demand workforces around the world. One of these workforces is Amazon Mechanical Turk.If needed, the Crowdflower system will aggregate the results and control for quality of the workfor the “requester”.Amazon Mechanical Turk, a USA based company, requires “requesters” to provide a USAbilling address and a USA credit card, Amazon Payments account or USA bank account to posta HIT. Therefore, it is not possible for “requesters” (i.e., researchers) from outside of the USA to
Restrictions and Prohibited Uses
It is a researcher’s responsibility to become familiar with and adhere to the policies of thecrowdsourcing service they will be using. Some prohibited activities include: collectingpersonally identifiable information and unsolicited contacting of users. HITS that violateAmazon Mechanical Turk policies include:
HITs requiring disclosure of the Worker's identity or e-mail address, either directly orindirectly
HITs requiring registration at another website or group
HITs that directly or indirectly promote a site, service, or opinion
HITs that violate the terms and conditions of an activity or website (for instanceasking Workers to vote for something)
HITs that have explicit or offensive content, for instance, nudity, but do not have theAdult Content Qualification
HITs asking Workers to solicit third parties
HITs that generate "referred" site visits or click-through traffic
HITs that ask Workers to take action to manipulate a website's behavior or results
HITs that violate intellectual property rights of any party
HITs that require Workers to download softwareIf at any time you are unsure whether the process/procedure you are proposing breaches thepolicies of the crowdsourcing service, you are encouraged to contact the service to verify thatwhat you are proposing does not breach their policies or Terms of Use.
Revised September 2012; November 2012 
post a HIT directly on Amazon Mechanical Turk. Non-USA researchers, including University ofWaterloo researchers, often work with Crowdflower to coordinate their requests and listing HITson Amazon Mechanical Turk. However, University of Waterloo researchers who are citizens ofthe USA (or have dual citizenship) can usually post studies directly to Amazon Mechanical Turk.
Data Collection Systems
Although the Crowdflower platform allows researchers to create their own surveys hosted byCrowdflower, typically University of Waterloo researchers use Crowdflower only as anintermediary to recruit participants through Amazon Mechanical Turk. Researchers tend tocreate the surveys themselves using SurveyMonkey™, Qualtrics™ or a University of Waterloodepartmental system. Crowdflower is used mainly as a third-party platform for accessing theAmazon Mechanical Turk labour market.Researchers may use programs where the computer servers are housed in the USA, such asSurvey Monkey™ or Qualtrics™, to collect a participant’s response to the study tasks orquestions. However, there are statements concerning the access of information as a result ofthe USA Patriot Act that need to be included in the information-consent letter. When preparingyour information-consent letter be sure to review the sample on the ORE website for conductingquestionnaire studies with implied consent.
Researchers are advised that when linking to a different site such as Survey Monkey™ toensure the survey page opens in a new window (or tab). If participants are to click on ahyperlink to the survey and it opens in the same window, people may be unable to navigateback to the crowdsourcing page to submit their HIT. One suggestion is to add an instruction thatreads “Please open this link in a new window.”
Listing Studies
If you are using Crowdflower to post a HIT on Amazon Mechanical Turk, you first need to createan account with Crowdflower. There is no cost associated with opening an account. Aftercreating a "job" (along with indicating the number of participants needed and the associatedremuneration) you are provided a total cost for that job by Crowdflower.Crowdflower collects a 33% fee and Amazon Mechanical Turk collects a fee of 10% on top ofwhat researchers pay to participants.
For example, if a HIT pays $0.50, Amazon MechanicalTurk collects $0.05 and Crowdflower collects 33% of $0.55 (or $0.18). The fee per participantequals $0.73. Rresearchers pay the total cost to Crowdflower who manages all remunerationboth to Amazon Mechanical Turk and participants.If you accept the cost of the job, the job is subsequently listed as a HIT on Amazon MechanicalTurk. Amazon Mechanical Turk is the checkbox that is selected by default. The job listing uses afew simple words to identify the study along with the stated remuneration. Crowdflowerautomatically inserts the first line of the “job” description as the listing name for the HIT. Be surethis line adequately and appropriately describes your study.The “time allotted” section refers to the amount of time a “worker” has from the time they click“accept HIT” to when they are allowed to submit the HIT. Sixty minutes is automatically chosenby Crowdflower when the HIT is listed. The HIT expiration date is also automatically set byCrowdflower. This is set to one week from the date the HIT is listed on Amazon Mechanical

You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->