Content Utility Rating Guidelines v1.

8 l
Release Date:

21 tia
1. Introduction
1.1 Overview/Context

44 n
This project expands our understanding of Facebook Communities through another
dimension that we call Utility. This workflow examines topics as “utilities”, which
are matched to a post within a Facebook group. Topics have standalone intent,
which is unrelated to the user’s contextual info (e.g. searcher location, time
e
generated, etc). These guidelines will provide a set of instructions on how to view,
interpret, and review content level information within a group post and match
content in posts to Utility categories in the CTX taxonomy.
49 id
What is this workflow trying to do? This project expands our understanding of
Facebook Communities through another dimension we call Utility. Currently, our
models consist of topics (like Basketball, Music, etc.) within a Facebook group. In
addition to a post’s topic, its utility is defined by what the person uses it for. The goal
of the workflow is trying to understand how useful a post is to the user within a
nf

group, and why they are looking at the post (including all content within the post).
We are trying to get a deep understanding of the user’s actions and emotions around
post interactions within a group.

What is the purpose of your rating? Labels will power a ML model that will
recognize posts/content a user may find useful and want to interact with. The model
Co

may be used to eventually recommend other posts within the user’s group,
recommend other groups the user may be interested in, and recommend other posts
in other groups the user may find interesting (and want to join the group). It may be
possible for content to have more than one utility, in such cases we want to capture
this information as well. We will only be looking at content from public groups.
1.1.2 Language Requirements

This workflow will be gathering data in the following languages:

• English
• Portuguese
• Spanish
• Arabic
 • Indonesian
• Thai
• Vietnamese
• French

8 l
• Turkish

21 tia
1.2 Understanding the Task
1.2.1 Categories and definitions

Utility & Intent is defined: The taxonomy of concepts to identify the primary
purpose of an interaction between a use and an entity.

Utility category definitions and guidelines document - please refer to the definitions

44 n
of Utilities below and any exclusions listed in this document linked.

The utilities are:

• Classmates & Alumni Utility
e o Current Classmates Utility
o School Alumni Utility
• Dating Utility
49 id
• Event Utility
• Fandom Utility
• Humor & Entertainment Utility
• Learning Utility
• Neighborhood Utility
nf

• Emotional Support Utilities

o Advice Utility
o Inspiration & Motivation Utility
o Virtual Memorial Utility
• Official Organization Utilities
o Business Organization Utility
Co

▪ Multi-Level Marketing Utility

o Charitable Organization Utility
o Cultural Organization Utility
o Government Organization Utility
o Military & Veteran Organization Utility
o School Organization Utility
o Workplace Organization Utility
• Transaction Utilities
o Buy, Sell & Trade Utility
o Crowdfunding & Fundraising Utility
o Jobs & Employment Utility
o Property & Housing Utility
▪ Housing for Rent Utility
 ▪ Property for Sale Utility

A B C D
1 Utility Utility Definition
Category Subcategory

8 l
2 Classmates & The utility used for content

21 tia
Alumni Utility pertaining to both current and
previous students, and for
parents of students affiliated with
the same educational institution.
This includes any educational
program whether it is online-only
or traditional brick-and-mortar.
This parent node should only be

44 ne used when there is not a best fit

with any of the more specific
child nodes, or the content
includes content equally for
students and alumni . EXCLUDES:
Use "Learning Utility" for
instruction, training and classes.
49 id
For content by educational
institutions to disseminate
information to students, parents
and the community, use "School
Organization Utility".
nf

3 Current The utility used to describe

Classmates interactions and activities for
Utility current students, classmates, and
parent of students affiliated with
the same educational institution.
This includes all types of
Co

activities, such as extracurricular

clubs, sports teams, and
organizations as long as the focus
is on the students themselves.
EXCLUDES: For content that
targets former students, use
"School Alumni Utility".
4 School Alumni The utility used to describe
Utility interactions and activities for
former students & past
classmates affiliated with the
same educational institution.
EXCLUDES: For content that
 targets current students, use
"Current Classmates Utility". For
instruction, training and classes,
use "Learning Utility".
5 Dating Utility The utility used for people

8 l
looking for meaningful romantic

21 tia
relationships. This can include
casual or serious dating,
marriage, long term relationships,
and other types of romantic
connections. EXCLUDES:
Activities for platonic friendships,
sexual solicitations, or
pornographic content.
6

44 n
Event Utility

e The utility used for capturing the

activities related to specific
events. This can include an event
itself, the organization and
planning of an event, and the
marketing and advertising of an
event. An event can be online or
49 id
in person, can be a single instance
or recurring, and can be official or
unofficial.
7 Fandom Utility The utility used for fans who
share a common interest in a
nf

specific named entity. Examples

of named entities include Taylor
Swift, Dragon Ball Z, The Dallas
Cowboys, Mercedes-Benz and
Star Wars. EXCLUDES: Use the
more general "Common Interest
Co

Utilities" node to describe broad-

concept topics like pop music,
anime, football, automobiles or
science-fiction movies.
8 Humor & The utility used to describe
Entertainment content that is intended to
Utility provoke laughter & provide
amusement and entertainment.
Common types of content include
memes, videos, photos, and
quotes.
9 Learning The utility used for educators &
Utility students of all types to
 collaboratively work together
through instruction, training and
classes. This can include personal,
academic, or professional
development. EXCLUDES:

8 l
Content that focuses on schools,

21 tia
students, or teachers that is not
direct instruction. Do not use this
node for content that is merely
"about" teaching or learning.
10 Neighborhood The utility used to describe the
Utility activities of current residents of a
geographic area to share
information, discuss topics, and

44 ne interact with other people within

the community. Includes specific
content such as lost and found
items or missing pets, and also
broad topics such as
recommendations and
community safety. EXCLUDES: If
49 id
the content is exclusively focused
on local buying & selling
interactions, use the "Buy, Sell &
Trade Utility" node.
11 Emotional The utility used to provide &
nf

Support receive emotional support. Types

Utilities of help often include providing
advice, exchanging personal
experiences, providing sympathy
& offering coping strategies. This
parent node should only be used
Co

when there is not a best fit with

any of the more specific child
nodes. EXCLUDES: Professional
mental health services and
counseling should not be
included.
12 Advice Utility The utility used for giving &
receiving advice, guidance or
suggestions. The content can
focus on any topic, and can range
in intensity from casual subjects
like dating to more serious issues.
EXCLUDES: This node cannot be
 used for any content concerning
medical or health-related advice.
13 Inspiration & The utility used for sharing quick
Motivation inspiration, motivation and
Utility positive feelings, typically

8 l
through quotes, images and

21 tia
memes.
14 Virtual The utility used for online
Memorial memorials to commemorate a
Utility person who has died. Content
typically includes sharing
memories, photos and videos.
EXCLUDES: Do not use this node
for other types of memorials,

44 ne
15 Official
Organization
Utilities
such as pets, places or businesses.
Use "Event Utilities" for funeral
services in real life.
The utility used to describe
official organizations created for
people with a shared purpose and
common goals. Organizations can
49 id
be thought of as groups,
associations, clubs, societies,
teams and more. Common
activities of organizations include
the exchange of information,
nf

planning activities, and

discussions between members.
Use this node for professional
networks by occupations. This
parent node should only be used
when there is not a best fit with
Co

any of the more specific child

nodes.
16 Business The utility used to describe for-
Organization profit organizations to share
Utility information with existing and
potential customers. This can
include all types of content,
including marketing &
advertising, available products,
and information about what they
do. EXCLUDES: Use "Workplace
Organization Utility" for
workplace coordination, "Buy,
 Sell & Trade Utility" where the
focus is items for sale or trade,
and "Jobs & Employment Utility"
for job postings & employment
opportunities.

8 l
17 Multi-Level The utility used to identify multi-

21 tia
Marketing level marketing scams & pyramid
Utility selling schemes.
18 Charitable The utility used to describe a
Organization charitable organization. These
Utility are often nonprofit businesses,
and their focus should be on
philanthropy, community support
and social well-being. EXCLUDES:

44 n
19
e
Cultural
Organization
Utility
For charitable transactions, use
"Crowdfunding & Fundraising
Utility".
The utility used to describe a
cultural organization, such as a
museum, library, arts
organization or historical society.
49 id
These are often nonprofit
businesses, and their focus
should be on the promotion and
preservation of arts & culture.
20 Government The utility used to describe a
nf

Organization government organization or

Utility agency. These must to be official
organizations, and are often
dedicated to dissemination of
information and policies, and the
administration of specific
Co

functions. EXCLUDES: For

military and veteran
organizations, use "Military &
Veteran Organization Utility".
21 Military & The utility used to describe
Veteran current or past members of
Organization military organizations, to share
Utility information, memories, and
provide support. EXCLUDES:
Many military branches are
considered branches of
government: use this node to
describe them, and not
 "Government Organization
Utility".
22 School The utility used by an educational
Organization institution to exchange
Utility information with students,

8 l
parents of students and their

21 tia
community. The school
organization can be online-only
or traditional brick-and-mortar.
EXCLUDES: For teaching &
learning, use "Learning Utility".
For communication between
students, use "Current Classmates
Utility".

44 n
23

e Workplace
Organization
Utility
The utility used by workers and
employees within the same
organization to coordinate & to
collaboratively manage work.
This can also include a larger
network of organizations and
companies, as long as the focus is
49 id
on internal collaboration and
workplace management.
EXCLUDES: For communication
between a business and
customers, use the "Business
nf

Organization Utility". For job

postings & employment
opportunities, use "Jobs &
Employment Utility".
24 Transaction The utility used to generally
Utilities describe all types of transactions
Co

of money, goods, opportunities or

services between people. This
parent node should only be used
when there is not a best fit with
any of the more specific child
nodes.
25 Buy, Sell & The utility used to identify
Trade Utility content where content is
primarily focused on buying,
selling or trading products and
services.
26 Crowdfunding The utility used to describe the
& Fundraising activities of raising funds and
 Utility donations for charitable
purposes, including
crowdfunding and crowdsourcing
campaigns, fundraisers, and
mutual aid. EXCLUDES: For

8 l
charitable organizations, use

21 tia
"Charitable Organization Utility".
27 Jobs & The utility used for content
Employment helping people find jobs &
Utility employment, whether it is short
or long term. This also includes
volunteer opportunities &
internships, as well as work
exchanges and time banking.

44 n
28

e Property &
Housing
Utility
The utility used for content about
buying and renting property. This
parent node should only be used
when there is not a best fit with
any of the more specific child
nodes, or the content includes
both properties for sale and for
49 id
rent.
29 Housing The utility used for people to find
for Rent rental housing & lodging by
Utility connecting tenants with rental
units. Types of housing can
nf

include houses and apartments,

but also short term lodging like
hotels, campers & rooms for rent.
EXCLUDES: Do not use for
content that includes property for
sale.
Co

30 Property The utility used for people to find

for Sale real estate for sale by connecting
Utility buyers with properties. Types of
property include houses,
apartments, and parcels of land.
EXCLUDES: Do not use for
content that includes property for
rent.

1.2.2 The Rating UI in SRT

 8 l
21 tia
44 ne
There will be hover over definitions (‘i’ button) next to each utility option to help
49 id
raters carefully assign the correct utility.

1.2.3 The Rating Process

1. Review the group post and content included: Review the images as well as
peripheral texts such as captions, author name and overlay text for additional
nf

context about the post. Review any media (pictures, videos) and/or products
included in the post.
2. Determine Ratability: Evaluate the entire post and its content decide
whether or not you can rate the job.
3. Choose Rate or Skip: You must Skip if:
a. The post or any content/media within the post is mostly in a foreign
Co

language that makes it impossible to understand the utility.

b. The post or any content included is sensitive
c. The photos do not load, or have extremely low quality (i.e., too blurry
or too dark) that you cannot understand the content as a whole
enough to rate.
4. Rate: Identify the utility label(s) that clearly match the purpose of the group
content.
5. Click Submit.
1.2.4 Identifying the utility label
1. Identify the most relevant utilities of the provided group content using the
search typeahead that lists all the available utility labels.
 2. The full list of utility labels is available in the tool through a search typeahead
that allows multiple utility labels to be selected. The initial labels shown in
the typeahead (with no user input) will be randomly generated for each job.
3. Use the following order to get the most relevant utility:
a. Look at the Group Level Information (title/description/cover photo)

8 l
and the given Content within the Post to find the matching utility

21 tia
label.
i. If the group-level information is not helpful, then still just use
the given post’s content to identify the utility of the group.
4. If you can only identify a utility for the given content (after looking at both
group level information and post content) that’s not part of the current
labels, you may indicate this by selecting the corresponding option. In this
case, there is no need to select any label from the typeahead.

44 n
1.3 Reasons to Skip a Job
1.3.1 Foreign Language
e
When the post contains foreign language, you should use Google Translate or the
“See Translation” button to better understand the posts’ or media within the post’s
49 id
context. If there is only a small portion of content in another language and you’re
able to gather enough signal to complete the task, proceed to rate the job.

Only choose Skip for foreign language if the foreign language text cannot be easily
translated, or if you cannot gather enough signals to rate.
nf

1.3.2 Sensitive Content

One reason to skip a job is if the post or media within the post contains Sensitive
Content. This could include:
• Child Exploitation and/or Child Nudity - Escalate (directions below)
• Self Injury and Suicidal Content - Escalate (directions below)
Co

• Credible Threats, Violence or Calls to Violence - Escalate (directions below)

• Adult Sexual Content and/or Nudity
• Hate Speech
• Acts of Terrorism
• Human Trafficking
• Bullying and Harassment
• Unlawful Sale of Animals: Consider the type of animal being sold as well as
the seller. Sale of animals by private individuals (rather than by businesses)
is considered sensitive
• Violent, gruesome, or repulsive content, intended to shock users, for example
dismembered body parts or visible internal organs (unless in a medical
setting).
 • Personally Identifiable Information (PII): This includes names, residential
addresses, and phone numbers of private individuals. This does NOT include
business addresses and phone numbers, or business names (e.g., “Becky
Smith Travel Agency” or “Law Offices of Perry R. Clark”).
• If you recognize the user

8 l
Note: Instructions for Immediate Escalation:

21 tia
To immediately escalate content with credible threats of self-injury or suicidal content,
or Child Exploitation and/or Child Nudity, email outsourcing-suicidelertesc@fb.com.

Internal: Submit contact form

External:
Send an email to your Team Lead and include both the violation reason and job
permalink. Your Team Lead will identify whether or not the comment or media in fact

44 n
violates. If so, they will submit the job using the link above.

e
2. Understanding the Labels
49 id
Step 0: Can you label this job?

Review the Group Level Information and provided Post Content to decide
whether you can label this job or not. The options below are meant to be exhaustive
and each will have different actions. Some actions lead to the next question, whereas
some will finish the job.
nf

1. Yes. I can label this job. (Action: Go to next question)

There is enough information for the post content to make a high quality
decision about the job. Most jobs will have this answer.
2. Yes. But GI Review. (Action: Go to next question)
There is enough information for the content to make a high quality decision
about the job. However, the group or content is sensitive or violating and
Co

should be sent to GI for review.

3. No - Group/Post Mismatch. (Action: Stop and job complete)
The group level and post are a mismatch. This happens when the topic or
utility is conflicting from the admin intent and member actions.
4. No. Non-Informative (Action: Stop and job complete)
a. The content is not clear or there is not enough information to decide
what the utility of the content is.
b. This does not include content from groups where people share and
interact with a wide range of topics without a central theme, this
would be the Broad Discussion Utility.
5. No. Content Language Mismatch (Action: Stop and job complete)
The content is in a different language than the specified queue.
 6. No. Rendering Issue. (Action: Stop and job complete)
The content is experiencing rendering issues (e.x., content does not load to
be able to rate).

8 l
Step 1: Select the Utility for this group (Action: go to next step)

21 tia
Select the most relevant utilities for the content from the options in the search
typeahead (Action: Go to next question)
• The utility of the content should be evident from the post text, or media
included (photos, videos).
• The full list of utility labels is available through a search typeahead that
allows a single utility label to be selected.
• Examples of utilities from content-level information:

44 ne o Buy, Sell & Trade Utility

▪ Group Name: Sorsogon Buy And Sell Online
▪ Group Description: A one stop shop for your needs
▪ Post/Content:
49 id
nf
Co
 ▪Comment: The post text clarifies that the post is a for selling
ramen kits
o Fan Utility
▪ Group Name: Los Angeles Clippers Fans

8 l
▪ Group Description: A place to talk about our ❤️ LA Clippers

21 tia
▪ Post/Content:

44 ne
49 id
nf

•Comment: Post text/video clarifies that this post is

about shared interest for a specific entity (Los angeles
Co

clippers)
o Organization Promotion Utility
▪ Group Name: Forum for latest updates and discussions of ICAI
▪ Group Description: This is a group open for all those who are
either aspiring to become or have become a Chartered
Accountant and company secretary. Everyone is invited to
share his thought on the latest activities of ICAI and other
Financial updates.
▪ Post/Content:
 8 l
21 tia
44 ne
49 id
nf

▪ Comment: Post content suggests that the post is about

discussion/updates about the organization
o Broad Discussion Utility
Co

▪ Group Name: 4TheCulture5900

▪ Group Description: We made this Group for all of our rappers
fans Fam you can post anything but no fake stuff.
▪ Post/Content:
 8 l
21 tia
44 ne
49 id
nf

o Neighborhood Utility
▪ Group Name: Our Sunnyvale
▪ Group Description:Please, write in this group info ONLY about
Sunnyvale.Events in Sunnyvale. Stores in Sunnyvale.Learning
groups in Sunnyvale. Advertising ONLY ONES A MONTH
Co

▪ Post/Content:
 8 l
21 tia
44 ne
49 id
nf

o Professional Networking Utility

▪ Group Name: Filmmaking Hub of Philadelphia
Co

▪ Group Description: This group is for Philadelphia based

filmmakers and film collaborators to network, share, and
support the local film community. Feel free to add other
filmmakers, post your work, screenings, jobs, ask for advice,
and offer services.
▪ Post/Content:
 8 l
21 tia
44 ne
49 id
nf

o Property Rental Utility

▪ Group Name: Witbank ROOMS&HOUSES to rent
▪ Group Description: The aim of dis group is to help each other
seeking rooms and houses to rent all over Witbank
▪ Post/Content:
Co
 8 l
21 tia
44 ne
49 id
3. Understanding the Context
nf

Please follow this process when rating Relevance:

1. Perform side searches of the query topic to understand the intent. We’ve
included a side search button for Google, Facebook, and Google Images for
each job, or you may use a side search engine of your choice.
Co

2. Review the result photo carefully to identify what it is about.

3. Rate relevance for a topic based on how well the topic represents the content
of the post