IG Reels: NLP Labeling

Overview
The IG Reels team wants to ensure that we are recommending Reels that users can
understand. In this workflow, raters will be asked to watch an Instagram Reel, and rate for
whether certain aspects of the Reel (caption, audio) are in a specific language (or language
neutral).

Guidelines

Escalations
In the course of performing your work, you may encounter certain violating content that must be
escalated for possible further action. Please follow these guidelines for escalating content:
Escalate content related to: 1. Escalate the Job ID (incl. SRT link)
● Terrorism immediately to your manager
● Potentially viral mass killing 2. If you are not comfortable reviewing
● Child exploitation or solicitation the content, skip the job after
● Self-injury or suicidal content escalating
● Child abuse 3. Vendor managers should use the
● Human trafficking PDO contact form to escalate Job IDs

To immediately escalate content with credible Violating content should never be screenshot
threats of self-injury or suicidal content, email or replicated in any way as it only
outsourcing-suicidelertesc@fb.com. exacerbates the issue. Please only use
task/Job ID for issue identification.

Tooling & Workflow

[Screenshot of workflow here]
1. Is the audio in the language you are labeling for?

Yes All or the majority of the speech and singing

audio is in the language you are labeling for.

Some The audio features an equal amount of the

language you are labeling for, and other
languages.

None None or very little of the audio features the

language you are labeling for.

There is no speech or singing There are no comprehensible words in the

audio, or there is gibberish / no audio.
 2. Is the caption in the language you are labeling for?

Yes All or a majority of the caption is in the

language you are labeling for.

Some The caption features an equal amount of the

language you are labeling for, and other
languages.

None None or very little of the caption features the

language you are labeling for.

There is no caption The Reel does not have a caption.

3. Is the text in the media in the language you are labeling for?
a. We are evaluating for the text that is part of the Reel, and cannot be turned on /
off (versus Instagram provided annotations, which can be turned on / off).
b. If the text is only numbers, we can go ahead and label “Yes”

Yes All or a majority of the text is in the language

you are labeling for.

Some The text features an equal amount of the

language you are labeling for, and other
languages.

None None or very little of the text features the

language you are labeling for.

No text in media The Reel does not feature native text.

4. Is the media language neutral?

a. Please be objective as possible when considering “appreciation” - another way to
think about it is whether the user would understand the intent of the Reel with
understanding the language
i. Ex 1) A Reel with a foreign language song in the background, but the
focus is of a cute kitten playing → Even if you personally don’t care about
kitten content, you would mark “Yes” (language neutral) as someone else
might
ii. Ex 2) A Reel with foreign language narration in the background, but the
focus is of a cute kitten playing → As with above, someone else might
appreciate the kitten content, but the narration might be adding value
(meme/humor or otherwise), so you’d mark “Borderline”
Yes The media either does not feature a
language, or it can be appreciated regardless
of whether a user understands the language.

Borderline The media can be appreciated if a user

doesn’t understand the language, but
understanding the language would add value
to their experience.

No The media cannot be appreciated without a

user understanding the language.