Expert Questionnaire

Crowdsourcing platform provider engagement:

1.) Please explain how to engage with a crowd sourcing platform provider to submit a creative brief to
receive bids – engagement between client, crowdsourcing platform and members of the crowdsourcing
platform?

The engagement between Client, Crowd sourcing platform and Members

Micro tasking encompasses crowd sourcing platforms that produce highly pre-determined,
Micro tasking qualitatively identical, and homogenous contributions resulting from simple tasks. Their
main goal is the scalable and time-efficient batch processing of highly repetitive tasks.
e.g., categorizing data or writing and translating small chunks of text.
e.g., Translate Facebook, in which the task of translating the social networking software is
broken down into a myriad of simplistic translation tasks which are then performed
independently by different Facebook users as contributors.

information pooling Crowdsourcing platforms for information pooling aggregate contributions such as votes,
opinions, assessments, forecasts, or other kinds of distributed information. This information
is usually aggregated through approaches such as averaging, summation, or visualization.
e.g., particularly useful for evaluating and selecting alternatives, eliciting and validating
customer needs, forecasting, market research, or gathering location-based information.
e.g., Google Maps, which infers real-time traffic information based on the GPS-data of smart
phone users.

Broadcast search Broadcast search platforms collect contributions to solve a task in order to gain alternative
insights and solutions from people outside the organization. These contributions are highly
heterogeneous as each contribution may reflect an alternative solution to the crowd
sourced task.
e.g., Broadcast search is particularly suited for solving challenging technical, analytical,
scientific, or creative 7 problems.

e.g., Netflix Prize invited contributors to submit algorithms that forecast the preferences of
Netflix customers more precisely than Netflix’s own algorithms and offered 1 million USD
for the solution.

Open collaboration Open collaboration platforms invite contributors to team up to jointly solve a complex
problem where the solution requires the integration of distributed knowledge and the skills
of many contributors.
e.g., Open collaboration is frequently used for collaborative ideation, knowledge creation,
open source software, and other open projects.
e.g., OpenIDEO, a crowdsourcing platform of the innovation agency IDEO, contributors
collaborate openly to develop joint contributions to global problems such as maternal
health or urbanization.
a. What are the criteria that the client would look into to identify potential crowd sourcing platform
providers?

Identifying potential Crowd source Platform

Environment
•Platform: Internal (build)
vs External (available)

Management
•Budget: Small vs Large
•Crowd sourcing expert and
experience: Available vs
Non available
•Acceptance level of low
quality result risk: High vs
Low

People
•Employee for Task: Few
vs Large
•The crowd for task:
Available vs Must build

Task
• Internet vs Physical
•Interactive vs Independent
•Sensitive information vs
Non - Sensitive
•Partitioned vs Non-
partitioned
 Identifying potential Crowd source Platform
Task Properties
Existing crowd sourcing
should only be used for
internet activities
Interaction property, focusing
on the nature of the
relationship between the
organization and members
during crowd sourcing
activities
Organizations to decompose
a task into a number of small
tasks that conceal the overall
picture, thus increasing the
ability to protect privacy or
intellectual property.
Crowd should be used for
tasks that can be subdivided.
People
Choosing crowd sourcing
when an organization does
not have enough
employees to deploy the
tasks.
The nature of the target
members will influence crowd
sourcing decisions.
crowd members to have a
certain level of skill,
availability will influence the
decision to crowd source.
Skills and abilities of the
crowd as human capital in
crowd sourcing critical
success factor.
Management
Cost saving is one main
reason to choose crowd
sourcing. Project with limited
budget should be crowd
sourced.
Importance of expertise
and management in different
parts of the crowd sourcing
process, such as workflow
management, members
management, and agreement
management.
Risk and risk management as
with any project, should be
considered in crowd sourcing
activities.
Environment
The choice between internal
or external platforms plays a
role in the Crowd sourcing
decision.
The availability of a crowd
sourcing platform can
decrease the development
cost.
Different platforms include
different pools of members,
which relates to the
probability of the decision to
crowd source.
The availability of the
platform that is suitable for
the defined task is valuable in
term of the availability of
its members.
 b. What are the different engagement models between crowdsourcing platform and its registered
members? (E.g. upfront cost for freelancers to register on crowdsourcing platform to bid for the creative
advertisement brief)

Class Engagement Models Description

Task Definition Task Modularization Dividing tasks into (a multitude of ) fine-grained subtasks
Contribution Requirements Define contribution requirements that the crowd sourced
contributions must fulfill
Pretesting Pretesting tasks with a small group of contributors

Allocation Skill-Based Allocation Restricting the group of participating contributors by

Demographic-Based Allocation
Performance-Based Allocation
personal skills (e.g., languages or qualifications)
Restrict the group of participating contributors by
demographic characteristics (e.g., gender or age)
Tracking a contributor’s performance of solving tasks and
restricting the group of participating contributors by means
of past performances

Quality Assurance Manual Control Manually validating the contributions of contributors

Automated Control (Partially) automating quality assurance by mechanisms that
countercheck contributions
Peer Assessment Providing functionalities by which contributors can verify the
validity of contributions

Incentives Payments Offering financial remuneration for successfully completing a

task
Prizes Offering cash or non-cash prizes for the “best” or the “first”
contribution(s)
Reputation System Providing functionalities that signalize a contributor’s
experience, activity, and merits
Framing Framing the task so that it increases in importance for
contributors (e.g., contributing to greater good)
Feedback Providing contributors with qualitative and / or quantitative
feedback regarding their contributions
Socialization Implementing opportunities for direct communication and
interaction between contributors such as forums, chats,
social networking, or messaging

Qualification Peer Coaching Providing mechanisms with which experienced contributors

provide advice to new contributors
Tutorials Offering text- and / or video-based trainings as well as
instructions on how to solve ideal-typical tasks
Onboarding Providing sample tasks with which contributors are trained
for contributing on the crowd sourcing platform

Regulation Non-Disclosure Agreement Legal regulations in order to maintain confidentiality of

crowdsourced tasks and related information
Netiquette Establishing formal and informal rules of participation as
well as terms of use with respect to desired behaviors of
contributors
Authentication Verifying the identity of newly registered contributors
Payment model:

1.) How the typical payment process is carried out between client and the crowdsourcing platform
providers?

Brief:

Will the client be paying the entire amount to crowdsourcing platform or would the client be paying only a
management fee to the crowd sourcing platform?

S.No. Payment models between Client and Platform Adopting Platforms

1 Commission Model Creative Competition

The platform usually takes a commission out of the Payment Models Between Client and Platform
prize sum for hosting services for all participants.

2 Fee on Task basis

(10% - 20% per task) Human Intelligence Tasking (Hit)
3 Revenue Sharing Model
Crowd sourcing platform enables developers to sell Leap Motion
apps in a store and shares the revenue by a
predefined standard 70/30 model split
4 Full Payment Model Qmarkets, IdeaScale, Imaginatik, Brightidea and Spigit
Client’s total cost of ownership range from
$15,000 to $500,000 over 3 years.

b. Who decides the price for different services? Also, I have mentioned couple of scenarios below. Kindly
share your answers for the same.

-Our understanding is that, once a creative brief is uploaded into the platform then anyone elected to join
the portal can bid on the brief. However for e.g.: Does a director determine the price for a certain creative
brief or is it that the client declares their range of budget and then wait for members of the platform to
respond with the best bid)?

Client declares their range of budget and then wait for members of the platform
to respond with the best bid

ARCBAZAR Business Model

A company that has a challenge registers on the platform and sets the goal (optimal amount) and
monetary prize for the winner. The registered participants should submit their solutions in the given
timeframes. Once a deadline is met, an initiator of the competition selects the winner or winners.
Instead of paying $3,000-$4,000 for a project, the client can get the same job done for $600-$700 and
save time searching for the right employee.
 -Client has a creative brief ready and requires a crowdsourcing platform to upload the brief receive bids
on production. So in this case would the client wait to hear from different bidders for the best quoted
price than disclosing the budget? And in this scenario would not the bidder decide the price and the
crowdsourcing platform might be paid only a management fee?

Not Applicable

2.) What are the key parameters that the client could use to make the final selection from the list of
successful bids?

Key Parameters Details

Aptitude function estimates how well workers are suited for handling a
Aptitude Function task. It is used as basis for qualification and pre-selection

Rank function takes monetary aspects. A property of a sound ranking

Rank Function function is that if two workers have the same aptitude for a task then
the one with the lower bid will have a higher rank.

Create a hidden ‘‘tandem’’ task assignment comprising a worker whose

Skill Evaluation skills are known (high performer) with a high confidence and a worker
where the crowd sourcing platform has limited knowledge about its
skills (i.e., low confidence)

3.) What are the different avenues through which client could reduce cost while engaging with a
crowdsourcing platform provider? And what would be the key negotiating parameters while engaging
with a crowd source platform provider

Idea
Generatio
n
Lead Enhanced
generation efficiency

Branding
Reduced
develope
Performance Labor cost
ment
improvement
Revenue
generating and
Enhance cost saving Diverse
service labor pool

Specialized Speed to
labor pool Turn fixed market
into
variable
cost
 Performance Improvement and Revenue generation and Cost saving

Front – Office Benefit Back – Office Benefit

Marketing Technology
 Develop thousands of ad campaign ideas rather  Make un-automated functions scalable
than a few (determining the best of the choices)  Solve development problems more effectively

Sales Operations
 Reach more customers  Increase throughout
 Reach customers more quickly and cost-
effectively
 Develop sales leads
 Generate ideas for pitch books Finance
 Communicate better  Reduce costs

Product research Accounting

 Process, sift and find data to enhance  Process data in hours, instead of days
performance and gain a competitive advantage  Find errors and discrepancies faster and cheaper
by reaching thousands of skilled workers

4.) What are the industry KPIs (and benchmark values) to measure effectiveness of engagement with
crowdsourcing platform providers? Will these benchmarks vary much from an audio brief to a video brief
or other types of brief?

KPI Indicators Indicators

I. Crowd Membership -Crowd size

-Age
-Gender
-Nationality/residency
-Skill, knowledge, expertise
-Individual v corporation
-Identity
-Internality or externality
II. Crowd Platform -Cost
-Reliability
-Reach
-Capacity & storage
-Efficiency -Security
-Complexity
-Types of interaction method
-Quality of experience
III. Crowd Incentivisation -Types of incentive
-Amount of incentive
IV. Crowd Interactions & Outcomes -Tasks /Challenges created
-Interactions
-Time spent on platform
-Time to complete tasks
-Number of process cycles
-Outcomes and outputs
 -Trust measurements

Benchmark varies

Micro-tasking Micro-tasking generally a low level of granularity in involved

where problems or tasks are put to the crowd. Within the
task or problem design the process initiator describes his
specific level of expertise required to complete the task and
also the level of granularity as to the parts that comprise the
challenge. This approach in disaggregating tasks is an
important feature in innovation challenges.

5.) What are the advantages and dis advantages of engaging with crowdsourcing platform providers when
compared to traditional advertising/production agencies

Advantage Disadvantage

Retain strategic resources in-house Bridging the skills gap

Happy employees = Better employees Intellectual property/Lack of NDAs(Non-disclosure agreements)
Build scale and expertise Misalignment of expectations
free to define the amount of remuneration payments provided by the company is in no way proportionate
to the high quality of the contributions received

cost and risk advantages contribution of low quality work or even the possible lack of
variety of creative ideas, is a valuable tool for participation
the interaction between business and emerging communities
foster creativity, both at the individual and at the
organizational level
crowd voting considerably reduces the complexity of the
decision-making process Project involved to take Too much time

6.) What are the different components of a crowdsourcing platform vendor’s cost structure? (E.g. For a
vendor providing customer care call centre support service key cost structure components could be
salary/labour (around 50 -60%), infrastructure, technology, Rent/Admin, profit margin etc.)

MTurk Micro task cost structure (hourly task)

Sr.No. Cost structure component Share in cost structure (in %)

1 Labour (average annual $50000) 92%

2 Sr. No. Local tax 8% Sr.No.
Supplier Overview:

1.) Who are the key crowdsourcing platform providers existing in the market? If possible, kindly furnish them
regions wise – (North America, Latin America, Middle East & Africa, Europe, Asia Pacific)

Best crowd sourcing platforms to make idea grow

Idea Bounty Middle East & Africa
OpenIdeo North America, Europe, Asia Pacific
Innocentive North America

Best crowd sourcing platforms for graphic and product design

CrowdSpring North America

99Designs Australia
Cad Crowd North America
DesignCrowd North America, Asia Pacific

Best crowd sourcing platforms for everything in between

Micro Workers North America

Mechanical Turk or MTurk North America
Upwork North America, Asia Pacific and Europe
Source: Sculpteo

2.) With whom does following players engage for crowdsourcing?

Name Agency Name Region of engagement

Nike Zooppa & GreenXchange USA

Puma Youth Sight UK

Reebok Zooppa USA

 3.) What are the potential legal issues that the client could face while engaging with crowdsourcing platform
providers and what are the ways to mitigate the same? Additional case studies or examples will be very
insightful

Sr.No. Legal Issues Mitigate to legal Issues

1 Copyright Law Anyone interested in crowd sourcing creative designs such as logos, a tune or a computer
Legal Issues game must carefully consider copyright laws. Copyright originally vests in the person who
creates the work.

Author cannot transfer copyrights as such, but can – by way of an explicit or implicit
agreement - grant (exclusive or non-exclusive) rights to exploit a copyright.

2 License Contracts These complications of crowd sourcing a creative work may result in the crowd sourcing
company not having any rights in a given work – or even infringing the rights of a third
party from which a contributor may have stolen his or her contribution.

Therefore, essential for crowd sourcers to enter into appropriate contracts with the
contributors of their crowd sourcing projects.

4.) Kindly share few case studies of instances where leading brands (preferably based on companies like
Nike, Puma) have effectively used crowdsourcing for ad production

Case Study: 1

Bloomingdale’s, Keds allow consumers to give input in the design process

Bloomingdale’s is highlighting the idea of creativity, community and customization in a summer and fall campaign, even
if the majority of what it sells is traditional product.

The retailer promoted a collaboration with Keds and the Whitney Museum in the windows of the 59th Street flagship.
Customers could design their own custom Keds 24 hours a day through the Bloomingdale’s store windows, two of which
acted as a giant touch screen.

During store hours, three M.F.A. student artists, who won a contest judged by the three partners, painted canvases in
the store windows. Their work will be shown at a series of events through the summer. A live Webcam broadcast video
of the windows at the event blog at theoriginalsneaker.com/whitney. Visitors who “checked in” at the windows on
Foursquare got special offers, such as a two-for-one ticket to the Whitney.

Meanwhile, in the store and online, Bloomingdale’s is selling a premium Americana- and vintage-inspired collection of
Keds sneakers as well as an exclusive line designed by artist Jenny Holzer. Two more lines by artists Laura Owens and
Sarah Crowner will make their debuts in September. All the merchandise is also available on keds.com, and all profits
from the artist editions will be donated to the Whitney.

The windows proved a hit.

Source: wwd.com
Case Study: 2

Retailer ModCloth’s, “Be the Buyer program”

Retailer ModCloth, which turned over $19 million last year selling inexpensive vintage inspired clothing online, is crowd
sourcing its buying with its Be the Buyer program. Anyone can vote and comment on a sample, and the ones that garner
the most votes will be put into production and sold on the site.

Each sample is posted online for about two weeks so people can vote on it. At the end of two weeks, ModCloth looks at
the results, contacts the designer to confirm they still have the fabric, then places an order. Many of the items are
manufactured locally and arrive in two or three weeks, but some are imported and can take up to 10 weeks. Online, the
status of the sample is changed, so it says “you picked it” and “coming soon.” Customers can sign up to receive an e-mail
when the item is available for sale.

Since the experiment went up late last year, ModCloth has posted close to 300 samples online and produced 50 of
them. The online store has more than 1,800 products on its site, so Be the Buyer is a small part of the mix.

Since the program launched, along with a new checkout process, overall site traffic and conversion rates have increased.
“Users that participate in Be the Buyer spend more time on the site and spend more on the site,”

As the company learns more, it plans to involve more designers. Future iterations will let shoppers give much more
specific feedback. In the future, not all votes will be equal. A vote from a customer who has been to the site many times
and placed multiple orders will count more than a vote from a new visitor.

Source: wwd.com

Thank You
Einblicks Consultancy