Professional Documents
Culture Documents
v1.0
5 l
Latest Update: 9/17/21
56 tia
Overview
OBJECTIVE
This experiment’s goal is to understand which pages/profiles on Facebook App should be classified as a Media
Org. Your goal will be to review a series of questions that break down the components of a Media Org. After rating is
completed, we’ll use your responses to determine whether or not a page/profile has commerciality (a component of a
41 n
Media Org), whether the org was an online first or offline first entity and ultimately, whether or not it is considered a
Media Org.
WHY WE DO THIS
e
By generating this data, you’re helping us improve our machine learning models that properly identify Commerciality
and Media Orgs on Facebook. This will help us better understand and cater to the users who may want to use our
business tools, understand our more advanced features and functionality, and be notified of relevant product releases
that could benefit them. This will also help us better understand the current landscape of Media Orgs on Facebook.
32 id
As a human labeler, you’ll be identifying whether a page is a media org or not by answering the question flow below.
RATING UI OVERVIEW
Below is an overview of the rating UI, which you will use to access entities for review and submit your responses to
each job. You will access this in a tool called Single Review Tool (SRT).
nf
Below is our latest UI, link to SRT of this queue: SRT | Single Review Tool
Co
5 l
56 tia
41 ne
32 id
nf
Co
Below is an overview of the questions that you will review in this workflow, as well as the order they will appear
5 l
56 tia
41 ne
Question Flow
32 id
QUESTION 1: Translation Tool / Skip Job Verification.
This question is to understand whether or not translation tools are used or if you need to skip the job due to
harmful/sensitive content (select 1):
nf
● I will need to use translation tools to complete this job - This option is to understand if the entity displayed is in
the language you are able to label.
○ Since some countries speak more than 1 primary language, you will likely see some entities that are NOT in
your language or English. When you encounter such entities, mark the first option to this question and use
translation tools like Google translate to complete the job.
○ You will still be required to complete the job, but by using translation tools like Google translate. If you are
Co
unable to complete the job despite using these translation tools, please mark “The page link is broken and
I can NOT complete this job”
● The page link is broken/private I can NOT complete this job/Or is Sensitive Content - Choose this option if the
primary link in the job is broken or shows as private and therefore must be skipped
● None of the above, I can complete this job - This option means that there is nothing hindering you from
completing this job and will rate the job like normal.
5 l
exploitation, forced slavery, slavery, servitude, or the removal of organs.
○ Sexual and Sensitive Content including Non Consensual Intimate Images / Revenge Porn, Non Consensual
56 tia
Sexual Touching / Rape / Sexual Assault , Registered/Convicted Sex Offender , Regulated Goods , Self Harm
Sextortion, Suicidal Content
○ Terrorism which includes Credible Threats (Non Terrorism, for Terrorism please
use http://fburl.com/CTEscalation
● If you are not comfortable reviewing the content, please skip the job after escalating.
● Vendor managers should use the CO Contact Form to route job ids into CO Safety queues(CEI should never be
screenshot or replicated in any way as it only further exacerbates the issue. Please always look to use task/job
QUESTION 2: Does this account reference an actual, living individual (or group of individuals)
in its profile photo, bio, or content?
e
In this question, our goal is to identify whether or not the account references an individual. One of the requirements
for determining a media org is that the account must not be owned by an actual, living individual. To classify this, we
will indicate ‘Yes’ if you can confirm that the account references an actual individual.
32 id
Below are the possible answer choices you can select (select 1):
● Yes: The account references an actual, living individual (or group of individuals) in its profile, bio, or content.
● No: The account does NOT reference an actual, living individual (or group of individuals) in its profile, bio, or
content
nf
● An individual person.
● A group of people. When is a group considered as an individual?
Co
○ Each group member’s personality is clearly recognized in the overall brand and followers recognize each
individual as a key part of the brand. The personality of each group member can be either identified in Bio
or in the content. In Bio, it could be the reference to each team members’ names/titles/personal accounts
etc. In the content, it could be the pictures showing all team members, pictures showing each team
member or posts consistently tagging the team members’ personal accounts.
○ Examples
■ Sports Team like Manchester United
■ Couples like The Jurgys or Yummertime
■ Siblings like the McClure Twins
■ Families like The ACE Family
■ A household like Hype House LA
■ A band like The Backstreet Boys
■ A comedy troupe or duo like Mythical (Rhett & Link) or The Tenderloins
■ A friend group or squad like Dude Perfect
● Business that features an individual. For example:
○ https://www.facebook.com/lisathomasinsurance → this account indicates in its name and content that the
account belongs to an individual.
5 l
○ https://www.facebook.com/Dr.Nour.Dentist/ → this account indicates in its name, bio, and content that it
belongs to an individual.
56 tia
● An animal that is living and represents a single personality or a small group of individual animals.
Exceptions
● A memorialized person or animal, meaning that the person or animal deceased and the account is in memorial
of it. We do not expect you to side-search this — this should only be considered if it’s evident within the account.
○ For example, this account (https://www.facebook.com/TheOfficialGrumpyCat) references Grumpy Cat,
which is a personality that is no longer living.
41 n
● A fan club for an individual. The page/profile is a group/or club supporting a group or individual, but it is not
the individual’s page (e.g., this)
● Spammer/Aggregator, is not an original entity but just reposts from other accounts and other page’s content. If
the most recent 10 posts are reposts, they should not be categorized as an individual.
e
● A fictional character or persona. For example:
○ https://www.facebook.com/Karl-the-Fog-145697385477802 → this references a fictional persona and not
an actual individual.
32 id
○ https://www.facebook.com/CoComelonKids → this account is not associated with an actual individual (or
group of individuals).
○ https://www.facebook.com/Cinderella→ this account references a cartoon, but not an actual individual.
○ https://www.facebook.com/TheOfficialPrincessLeia→ this account represents a fictional character and not
an actual individual.
● A service or business that does not reference an individual person.
nf
● Profile Photo: The profile photo shows an individual, who reflects the owner of the account. This can be a photo
of the person (face, shoes, etc.), a logo representing his/her/their initials, or another reference that specifically
identifies an individual.
● Bio: The bio clearly indicates a reference to the owner of the account in the bio. For example, the bio includes a
name, handle to a specific individual’s account (IG, TikTok, etc.), or an email that is clearly tied to an individual
(e.g. includes individual’s name in address).
● Content: The contents of the profile clearly indicate that the profile belongs to an individual. For this, consistent
references to a single individual (e.g. photos of the individual) will signal that the account is associated with an
individual.
Putting it all together:
Select ‘Yes’ if the entity satisfies at least one of the following:
● The account name or handle references an actual, living individual. After reviewing the content, there’s no signals
in recent posts (up to 10 latest posts) that the account owner is deceased.
5 l
● The profile photo shows an actual, living individual. This can be a photo of an individual.
56 tia
● The bio mentions the name of an individual.
● The content reflects posts featuring the same individual more than once (up to 30 latest posts).
41 n
The objective of this question is to identify certain categories of pages that should not be considered media
organizations or not considered for media partnerships management. Gaming and Music orgs will be considered for
management by our gaming and music partnerships teams
○ This entity is an educational resource - Select this option if the page/profile belongs to an entity thats
purpose is to educate. The type of educational resource should be something that could be used in
schools or tutoring programs.
○ This entity is a Fan Club or Fan Page - Select this option if this page/profile is a group/or club supporting a
group or individual. Select all (fan owned and operated) fan pages and fan clubs.
○ This entity is a police/fire station - Select this option if the page/profile belongs to a police station or fire
Co
station.
○ This entity is a hospital/medical center - Select this option if the page/profile belongs to a hospital or other
medical center where care/treatment is provided
○ This entity is a non-profit/non-governmental organization - Select this option if the page/profile belongs to
a non-profit / non-governmental organization, such as a charity. (e.g., this)
○ This entity is another type of business - Select this option if the page/profile belongs to a for-profit brick-
and-mortar business (e.g., this) or a page/profile primarily dedicated to e-commerce (e.g., this) Select
accounts associated with non-media related business activity who only create commerce related content.
Do not select businesses that also post original media content (not directly related to selling their product)
that may be culturally relevant.
○ None of these categories apply - Select this question if none of these categories apply
5 l
Below are the possible answer choices you can select (select 1):
● Yes: The page/profile belongs to an entity that publishes media content online, offline, or is a professional sports
56 tia
league.
● No: The page/profile belongs to an entity that does not create or publish media content or is fictional.
● Can’t Tell - Select ‘can’t tell’ if and only if the page/profile does not have enough data to allow you to determine
if the entity creates or publishes content.
41 n
● An organization (not an individual) with commercial intent that publishes original or licensed media on
Facebook with the primary intention of monetizing their brand through the media they produce. Media can take
on a variety of forms from TV, Movies, Videos, Magazines, Newspapers, etc. If copyright infringement is
suspected, please escalate.
e
● Media Organizations have a business model driven by the production and monetization of content; their primary
revenue comes from the media content itself (whether garnered online or offline)
32 id
A Media Org is not:
● Media Organizations are NOT individuals. If the page looks like it belongs to or is about an individual such as a
blog or individual music, video, or content creator label ‘No’.
● Media Organizations are NOT businesses such as shops/stores who only create content related to the sale of
goods or services (e.g., this) or a page/profile primarily dedicated to e-commerce or brand (e.g., this)
nf
● Profile Photo: The profile photo shows a logo or photo that looks relevant to the media org
● Bio: The bio clearly indicates a reference to the owner of the account in the bio as a media org. For example, the
bio includes a media org name, handle to a specific business account (IG, TikTok, etc.), or an email that is clearly
tied to a business or org.
● Content: The contents of the profile clearly indicate that the profile belongs to a media org. For this, consistent
references to a media org.
● Description: The page description or category describes itself as a media organization or an entity that publishes
media content in some way.
Commercial Intent and Monetization:
You will need to examine the Name, Profile/Cover Photos, About/Bio, Featured Photos (Personal Profiles), Customized
Page Tabs (Pages), and up to the latest 30 posts within the account to look for evidence of sales or promotion activity.
● Selling: Content (e.g. selling content subscriptions, music, e-books, videos), goods (e.g. phones, watches, t-shirts,
etc.), and services (e.g. housecleaning, plumbing, etc.) can be sold, and the sales activity does not need to take
5 l
place on the FB account itself.
56 tia
● Promoting: If the entity isn't directly engaged in sales of goods or services, it could still have commerciality. Any
form of product or service promotion (e.g. profile mentions partnership with business) qualifies as commercial
activity.
○ NOTE: The product/service promotion does not need to be authentic to the account owner in order to
qualify as a promotion activity. For example, if a user says ‘Use my discount code to get X% off’, this would
still qualify as promotional activity.
A primary indicator of this is point-of-sale information. This includes, but not limited to:
41 n
● Pictures of products/services paired with text in the media (photo/video) or post captions that have promotional
language (e.g. subscribe, buy, check this out, selling, use my promo code, etc...)
● Pictures and flyers of recurring promotional events. In other words, events that are not one-time events and
reflect the consistent products/services/causes being promoted by the account owner
e
● Contact information in a picture, post, ‘About’, etc. This can be a WA number, phone number, email, IG handle
for a separate account that is used for selling products, Line ID, Kik, WeChat ID, fax number, etc. For example,
‘Inquiries at [email address]’
● Pictures that have QR codes or watermarks
32 id
● Text (anywhere on the page) asking their audience to message them for interest in the product/service
● (Pages only) Call-to-Action buttons, which provide ways for a user to interact with a business / entity. These
typically appear on the right side of a page, and could include: booking a reservation, talking with a
representative, etc. See an example below.
● (Pages only) Customized page tabs, such as menus. See further details under ‘Signals in Facebook Pages/Profiles’
nf
below.
Co
QUESTION 5: Does the media organization create primarily online or primarily offline media?
If you answered ‘Yes’ to Question 4, you will be prompted to complete this question
In this question, our goal is to identify whether or not the account was a primarily online or offline entity. A primarily
offline media organization was created or became well-known for creating traditional offline content such as TV, films,
sports, music, art or print Media such as books, newspapers or magazines.
5 l
Online first media organizations are ‘digital natives’. They were born online and gained cultural relevance through
websites, blogs, social media, gaming, podcast or other forms of ‘new media’
56 tia
Below are the possible answer choices you can select (select1):
● Primarily Online - Select ‘online’ if the entity is known for creating online content or earning fame primarily on
the internet or social media.
● Primarily Offline - Select ‘offline’ if the individual is known for creating content or earning fame primarily through
means other than the internet or social media and if they originally gained cultural relevance through off-line
media before later going online
41 n
● Can’t Tell - Select ‘can’t tell’ if and only if the page/profile does not have enough data to allow you to determine
if the media organization is known for the offline or online content/fame.
○ If a fictional character/group is owned or produced by a film studio select this option (e.g., Buzz Lightyear,
Hello Kitty, Peppa Pig, etc.)
● Sports Leagues/Conferences - e.g., NFL, NBA, Premier League, FIFA
● Traditional Print or Broadcast News - the page/profile belongs to an News organization that produced news via
traditional channels such as TV, newspapers, magazines, or radio before going online
Co
● Digital-first News - A news publisher that was born online and never distributed News via traditional channels like
TV, Radio or Print media. Examples include Buzzfeed News, Breitbart, The Daily Wire, Billy Penn
● Digital Publishers (including Aggregators) - an entity that creates and/or publishes digital content (i.e. Buzzfeed,
CollegeHumor, Tastemade, etc.), or is a Content Aggregator (a content aggregator is defined as a page/profile
whose primary purpose is to aggregate content such as news, memes, videos, etc from other sources. (e.g.,this)
○ Original Digital Content - digital publisher publishes original digital content such as their own recorded
videos, pictures, digital work.
○ Aggregator - the digital publisher aggregates digital content such as news, memes, videos, clip compilation
etc.
QUESTION 5A: What type of description would define the owner of the page/profile?
If you answered ‘Primarily Online’ to Question 5, you will be prompted to complete this question
In this question, our goal is to identify whether or not the online first account is a digital publisher or news site.
5 l
Below are the possible answer choices you can select(select 1):
56 tia
● National or Global News - Select this option if the page/profile belongs to a national or global news entity (i.e.
New York Times, Washington Post, The Economist, etc.)
● Digital Publishers (incl Aggregators)- Select this option if the page/profile belongs to an entity that creates and/or
publishes digital content (i.e. Buzzfeed, CollegeHumor, Tastemade, etc.), or is a Content Aggregator (a content
aggregator is defined as a page/profile whose primary purpose is to aggregate content such as news, memes,
videos, etc from other sources. (e.g.,this)
● This is [other] - Select this option if the page/profile belongs to an online-first media organization and the
e
QUESTION 5A1: What type of description would define the media structure of the
page/profile?
32 id
If you answered ‘Digital Publishers’ to Question 5A, you will be prompted to complete this question
In this question, our goal is to identify the digital first content as original or aggregator.
Below are the possible answer choices you can select(select 1):
nf
○ Original Digital Content - Select this option if the digital publisher publishes original digital content. At
least 50% of up to 30 posts should be original posts and not shares or reposts.
○ Aggregator - Select this option if the digital publisher aggregates digital content such as news, memes,
videos, clip compilation etc.
○ Both - Select this option if the digital publisher publishes both original digital content and aggregates
digital content.
Co
○ Can’t Tell - Select ‘can’t tell’ if and only if the page/profile does not have enough data to allow you to
determine if the entity publishes original digital content or aggregates digital content.
QUESTION 5B: What type of description would define the owner of the page/profile?
If you answered ‘Primarily Offline’ to Question 5, you will be prompted to complete this question
Below are the possible answer choices you can select(select 1):
● TV Network (Broadcast & Cable) - Select this option if the page/profile belongs to a broadcast or cable TV
network or show (e.g., Nickelodeon, ESPN, Pawn Stars, Spongebob Squarepants, etc.)
● Music Videos - Select this option if the page/profile belongs to an entity that publishes music videos (eg: Sony,
Warner)
● Traditional Music Media - Select this option if the page/profile belongs to media that showed music prior to the
internet examples include Rolling Stone, MTV.
5 l
● Recording Studio/Artist - Select this option if the page/profile records and distributes music. examples include
56 tia
Universal Music Group, Sony Music, Bad Boy Records, Hollywood Records
● Film Studio - Select this option if the page/profile belongs to an entity that creates movies or television shows
(ex: Walt Disney Pictures, Marvel, Warner Bros, DC Films, etc.)
○ If a fictional character/group is owned or produced by a film studio select this option (e.g., Buzz Lightyear,
Hello Kitty, Peppa Pig, etc.)
● Sports League - Select this option if the page/profile belongs to a sports league (e.g., NFL, NBA, etc.)
● Sports Team - Select this option if the page/profile belongs to a sports team, organization, or franchise
41 n
(e.g., this).
● News - Select this option if the page/profile belongs to an entity publishing news (e.g., Fox News, Cosmopolitan,
Fox61, Young Turks)
● This is [other] - Select this option if the page/profile belongs to an offline-first media organization and the
e
description of that organization is not contained above.
32 id
QUESTION 5B1: What type of description would define the TV Network (Broadcast & Cable)?
If you answered ‘TV Network’ to Question 5B, you will be prompted to complete this question
Below are the possible answer choices you can select (select1):
QUESTION 5B2: What type of description would define the type of News the media org is
primarily publishing?
If you answered ‘News’ to Question 5B, you will be prompted to complete this question
Below are the possible answer choices you can select (select 1) :
●
Global or National News - Select this option if the content published by the page/profile is news of interest at a
national or international scope (i.e. CNN, Telemundo Noticias, etc.)
● Lifestyle News - Select this option if the content published by the page/profile is news regarding interests,
opinions, or behaviors of individuals, celebrities, or cultures (i.e. The Dodo, Radio Canada, India Times Lifestyle
Network).
5 l
● Local News - Select this option if the content published by the page/profile is related to the coverage of events,
but the news, in a local context that would not be an interest of another locality, or otherwise be of national or
56 tia
international scope (i.e. KTOO, Gothamist, The Oregonian).
● Political News - Select this option if the content published by the page/profile is news pertaining to politics,
political figures, or political parties (Breitbart, Daily Caller, Occupy Democrats).
41 ne
32 id
nf
Co