Professional Documents
Culture Documents
1 Guidelines
Last updated 2022-08-26
I. Rater instructions
Note: In these instructions, document will be used to refer to any text, image, or video that you may be
evaluating
Objective
The labeling task is to determine if the core purpose of a document falls into one or more restricted categories
and categorize the document into the best-fit category(s) that apply.
Use all of the information provided within the document to determine the core purpose of the document and
each correct category that describes the core purpose.
Overview
• The restricted categories are organized in a hierarchy, with some sub-categories appearing as
“children” of “parent” categories.
o Category = Parent Category
o Sub-category = Child category
• The parent will not be selectable in the SRT UI, and the user will see the child categories for that parent
when expanding each parent category using the arrow. Note: some categories consist of only one
term and do not have child categories. These single-term categories will be selectable.
The user will be presented with one document at a time and asked to select the most specific, but still
inclusive, child category or multiple child categories for each document.
Error Buttons:
There are four types of issues you might come across when evaluating documents: Page Load Error, Missing
Content, Wrong Language, and Sensitive.
The following table describes what each of these errors might look like, as well as what you should do if you
come across them while categorizing.
Rating Instructions
1. Evaluate all components of the document, including watching the full play time of videos.
2. Determine the core purpose of the document or video.
3. Do a side search if necessary: some documents may have an unclear purpose, product or service and
further context may be needed. Please do a side search in this case. Wikipedia is a good place to start,
but sometimes a broader search will be necessary.
4. Select if the user is able to make a decision on this job
a. If yes, move forward to select the most relevant restricted category(s) - child categories
b. If not, select the most appropriate error category (see Error category definitions above in
Materials)
5. Determine if the core purpose falls into any restricted category(s)
a. Yes, categorize the restricted category(s) to the best-fit child-category/child-categories (select
all relevant categories)
b. No, select the “None Apply” option. This option should only be selected if the user has
thoroughly evaluated all images, text, and/or hashtags and found that the document does not
relate to any restricted categories.
c. IMPORTANT the user must select at least one child category OR “None Apply” per job. If
neither None Apply nor child categories are selected, then the job will not be counted.
6. Indicate if you used the text, image/video, or text and image/video in the document to categorize
into child category or categories. If you did not select a child category and selected “None Apply”,
select N/A.
7. Select submit to move on to the next job
In some documents, a possible restricted category topic can be mentioned, but is not the core purpose. In
these cases, only apply labels related to the main topic of the document.
On the right side of the screen, there will be the interactive section where the task is completed and
submitted. If the content that is provided is not clear or there is confusion, perform a brief side search, if
necessary, using a search engine or Wikipedia to gain a better understanding.
Avoid Over-Labeling
Please ensure that the labeling does not include more labels than necessary to categorize the document.
Examples:
V3.1 has removed several categories. Frequently used categories: Couples & relationships, Married and Single
are removed. Less common categories: Grandparents, Family name, Screen name, and Title are also removed.
Content types
For each of the restricted categories, examples are provided whether a document falls within the categories
based on the types of content it represents. The content could feature the following types of examples:
• Products and services related to the category
• Businesses, organizations, associations, & entertainment (including establishments, places)
• Education & schooling associated with the category
• Attributes related to the category
• Culture, topics & causes related to the category
• Job titles or professions within organizations affiliated with the category
• Public figures associated with the category
Definitions
1. Age: Identifies content explicitly restricted to a specific age-group.
• Children: content explicitly related to (serving or purchased for) babies or children under 13. For
example:
o Image, video, or posts (including any captions) in which a child or children are mostly the core
purpose, or primary focus of the content.
▪ Note: Where a post is focused on parental status, but also shows an image of a child,
please co-label with Children and Parents.
o Services explicitly for babies or children (e.g., youth sports leagues, elementary schools,
pediatric dentists)
o Products explicitly for babies or children (e.g., children’s literature, children and baby clothing,
baby equipment, children’s toys), NOT toys enjoyed or collected by a range of age groups
o NOTE: this label includes products and services for babies, infants, or toddlers
• Adolescent: content explicitly related to (serving or purchased for) adolescents aged 13-17 (inclusive).
For example:
o Image, video, or posts (including any captions) in which teens or what appear to be older
children are mostly the core purpose, or primary focus of the content. This must also be
supported by captions and hashtags (e.g., #teens, #U16s, etc.)
o Services explicitly for adolescents (e.g., middle and high schools, high school football, teen
counseling services)
o Products explicitly for adolescents (e.g., teen literature or magazines)
• Skin color: user-specific data, content, products, or services indicating an individual’s natural skin color,
including user-provided adjectives (e.g., skin lighteners, “fair skin,” “melanin-rich skin,” “olive skin” and
products or services directed at a specific skin color)
• Fingerprints: user-specific data indicating an individual's unique fingerprint pattern data
• Hair color: user-specific data indicating an individual's unique natural hair color
• Height: user-specific data indicating an individual's unique height, including user-provided adjectives
(e.g., tall, short, petite, etc.)
• Iris scan: user-specific data gathered through mathematical pattern-recognition techniques on video
images of one or both irises to reveal individually unique iris patterns
3. Faith & spiritual belief: identifies content closely related to faith, religion, and religious
belief, including the practiced absence of such belief and deeply held philosophical beliefs.
For example:
• Content related to membership or affiliation with a specific faith-based or philosophical group
o user-declared faith or religion
o businesses, organizations, or entertainment related to a religious affiliation (e.g., church,
mosque, temple, Jewish Community Center, Christian rock)
o products or services related to a religious affiliation (e.g., Scripture finder websites, prayer
journals, Christian booksellers)
o culture, topics, or causes related to a religious affiliation (e.g., Christian homeschooling, Islamic
fashion, film or literature with an explicitly religious purpose)
o education related to a religious affiliation (e.g., Yale Divinity School, Yeshiva University)
o job titles related to a religious affiliation (e.g., pastor, minister, guru, rabbi)
o public figures related to a religious affiliation (e.g., Jesus, Joel Osteen, Allah, Sai Baba)
o attributes related to a religious affiliation (e.g., atheism, Christianity, Sikhism)
o holidays based on religious belief, even if secularized (e.g., Halloween, Easter, Ramadan,
Hannukah)
▪ Does not include widespread, secularized holidays referenced for sale or promotion
purposes (e.g., Easter sale, Christmas discounts, etc.)
• Content, products, and services related to philosophical, spiritual, occult, or metaphysical beliefs
7. Health: Identifies content relating to physical and mental health conditions, treatments,
organizations
• Mental health
o Self-harm & suicide prevention: content related to suicide or other types of self-harm,
including user experiences, treatment programs, specialists, or awareness campaigns.
• General health care
o Preventative care & health programs: content, products and services related to medical
health generally, but not tied to a specific medical condition or non-medical healthy lifestyle
choices. Includes:
▪ general medical topics and practitioners (e.g., pharmacies, family practitioners,
pediatricians, preventative or general dentistry, primary care, lung health)
▪ occupational health and safety (e.g., work safety gear, hardhats, HazMat suits)
▪ medical education and supplies (e.g., nursing schools, CPR mannequins, medical models)
▪ health insurance and health administration (Medicare, appointment services)
▪ cosmetic surgery not related to a medical condition (breast augmentation, face lifts,
medical esthetic services)
▪ medical education (e.g., nursing programs) or job postings
o Sexual & reproductive health care: content related to general sexual & reproductive health
care excluding medical conditions (e.g., gynecology, prostate examinations, birth control, etc.)
o Vaccines & vaccine status: content related to vaccines and vaccination, including vaccination
services and clinics, vaccination awareness campaigns, user-declared vaccination opinions or
status, etc.
▪ Note: User vaccination opinions should also be labeled with Politics > Political opinion
• Health data: Identifies user-specific data indicating unique personal health data
o Blood type: user-specific data indicating unique personal classification of blood type
o Genetic data: user-specific data indicating unique personal genetic data, including genetic
mapping, declared genetic disorders, DNA sequencing, chromosomal data, or genetic markers.
Note: may be co-labeled with Ancestry in the case of user-specific genetic/DNA data related to
ancestry
o Personal health information: user-specific personal health measurements, biosignals (e.g., data
from electrocardiogram or electroencephalogram), biological cycle data (e.g., sleep data,
ovulation data), reading & level data, including health & fitness data from programs, devices,
software, and wearables (e.g., heart rate/bpm; calories burned, oxygen levels, blood glucose
levels, etc.)
• Medical condition
o Accessibility settings: user-specific data indicating unique personal control settings intended to
increase access to technology, as related to physical or neurological ability
o Addiction: content related to current or past addiction to a substance, thing, or activity. For
example:
CONFIDENTIAL TO APPEN – DO NOT FORWARD
▪ user-declared status as an addict, recovering from addiction, or involved in supporting
someone suffering from addiction
▪ products and services related to addiction (e.g., Al-Anon, smoking cessation programs)
▪ topics and causes related to addiction (e.g., gambling addiction, addiction as a disease)
o Body dysmorphia & eating disorder: content related to current or past vulnerability regarding
body image vulnerabilities or disordered eating. For example:
▪ user-declared status as suffering or having suffered from body dysmorphia or eating
disorder
▪ services relating to eating disorder treatment (e.g., National Eating Disorders
Association; Eating Disorders Helpline)
▪ content promoting eating disorder (e.g., pro-ana/pro-mia; ‘thinspo’)
o Disability: Identifies content related explicitly to disability or specific disabilities, either mental
or physical. For example:
▪ user-declared status as having a disability
▪ attributes related to specific disabilities (e.g., “disabled,” “people with disabilities,”
“learning disabilities”)
▪ products and services related to specific disabilities (e.g., Paralympics)
▪ topics and causes related to disabilities (i.e., Disability Pride Month)
Note: if the words “disabled” or “disability” are not used, there should be clear link
between the product/service and disability status (e.g., Paralympics)
o Illness & injury: content related to specific health conditions (both physical and mental health
conditions), injuries, treatments for health conditions other than pregnancy/childbirth,
addiction, disability, or eating disorder. For example:
▪ user-declared illness or injury
▪ names of specific health conditions (e.g., “cancer,” “asthma,” “HIV/AIDS,” “ADHD,”
“diabetes”)
▪ services and products tied to specific health conditions (e.g., chemotherapy, Beltone
hearing aids, Certified diabetes educator, glasses/lenses, wheelchairs)
• Note: words such as “treatment,” “therapy,” “care,” or “medicine/medical”
often require this label
▪ information on injury, disease, or disease risk, including medical history, medical
opinions, diagnosis and clinical treatment
▪ medical services related to a medical condition (e.g., cardiothoracic surgeon, DaVita
dialysis centers), excluding preventative health or wellness services
▪ advocacy groups and support groups for survivors (e.g., American Cancer Society, Find a
Cure for Cystic Fibrosis, Alzheimer’s Association of America)
o Prescription or over-the-counter (OTC) medication: medications intended for the treatment of
specific medical conditions and may or may not require a prescription. For example:
▪ user-declared use of prescription or OTC medication
▪ products and services related to medications (e.g., Nyquil, acetaminophen, Ambien,
insulin needles, pill cases, herbal remedies)
▪ topics related to medications (e.g., PrEP, medical cannabis)
o Pregnancy & childbirth: Identifies content, products, and services related to pregnancy, birth,
and postpartum experiences, including abortion.
▪ Abortion: Identifies content, products, or services related to the termination of a
pregnancy. For example:
CONFIDENTIAL TO APPEN – DO NOT FORWARD
• user-declared experience with terminating a pregnancy or discussion of abortion
• abortion service providers or resources
• prescription or OTC medications related to terminating a pregnancy
▪ Pregnancy, birth & postpartum: Identifies content, products, and services related to
pregnancy and childbirth. Includes:
• user-declared status as pregnant or postpartum
• medical services related to pregnancy and childbirth (e.g., fertility therapy,
obstetrics, lactation specialists)
• products related to pregnancy and childbirth (e.g., pregnancy tests, breast milk
pumps, belly balm, nursing pillows, etc.)
• services related to pregnancy and childbirth (e.g., birthing classes, doulas,
midwifery, etc.)
• topics related to pregnancy and childbirth (e.g., natural childbirth, home birth,
breast feeding, etc.)
Note: When applying this label, do not co-label with gender, medical condition, or
age.
• Wellness
o Fitness & self-care: Identifies content related to general, non-medical health maintenance and
healthy lifestyle. For example:
▪ products related to healthy lifestyle (e.g., fitness trackers, hydration trackers, free
weights)
▪ services related to healthy lifestyle (e.g., gym memberships, yoga classes, hot stone
massage, reiki)
▪ Does NOT include bath and beauty treatments (e.g., shower gels, hand creams, facial
products or treatments, etc.)
o Personal fitness information: user-declared data indicating unique personal calculated or
achievement-related fitness data (e.g., 'ran a 5k,' 'benchpress personal best,' etc.)
▪ EXCLUDES raw data or measurements (e.g., heart rate/bpm, oxygen levels, blood
glucose levels)
o Vitamins & supplements: Identifies pills, capsules, powders, or liquids intended to supplement
diet and provide nutrients, not for treating a specific medical condition. For example:
▪ user-declared use of vitamins or supplements
▪ products and services related to vitamins/supplements such as Airborne, creatine
products, multivitamins, collagen protein powders, herbal supplements to ‘support’
behaviors or body parts (e.g., ‘Turmeric joint support’ or ‘Focus’ supplements)
o Weight loss: Identifies content intended to promote weight loss. For example:
▪ user-declared content about losing weight, discussion of prior weight loss, or weight loss
progress
▪ topics related to weight loss (e.g., keto diet, calorie counting, intermittent fasting)
▪ products and services related to weight loss (e.g., liposuction, macro counting/tracking
apps, Noom, meal-replacement shakes and bars)
▪ words and phrases such as ‘slimming’, ‘fat loss', 'burn fat', 'melt away fat', 'stay trim' are
clues that an ad relates to weight loss.
▪ Does NOT include fitness equipment (e.g., fitness trackers, free weights, yoga mats)
which are not necessarily weight loss-related - use ‘Fitness & self-care’ label.
Topics related to ethnic, ethnolinguistic, ethnoreligious groups should be labeled with this category.
Topics related to non-ethnic cultures (such as national cultures) should not. Topics related to racial
identity should be labeled with Ethnicity.
14. Politics: content related to political affiliation or belief, voting activity and/or related to
political issues or government services
• Government services: content related to non-political government agencies and services. For example:
o agencies such as the Department of Motor Vehicles or local Parks & Recreation
Note: Does NOT include broad political locations such as municipalities, counties, cities, states, or
countries
• Political affiliation: content related to specific political parties, politicians, political philosophies or
other content affiliated with a political party or political movement. For example:
o User-declared political affiliation, including intent to vote for a particular candidate
Side research note: News sources with an explicit political leaning, state-owned news agencies, and
any news organizations affiliated with state propaganda should receive this category. When news
sources are closely tied to political affiliation, Wikipedia searches will often note the “political
alignment” of a news source in structured data on the right-hand side or in a content header. Any time
a political alignment is clearly stated, the topic should receive this category.
• Political candidate: content promoting a specific political candidate for public office
o Ads could be for local political positions (e.g., city council/councilman, mayor, alderman) or
state/national positions (e.g., representatives, assemblymen, senators, governors, presidents,
etc.)
o Where a document shows a politician as a part of a larger political movement, unrelated to a
specific election, Political affiliation is the correct categorization.
• Political issues: Social or political issues subject to political debate or lobbying, but without statement
of a specific stance or explicit political affiliation. Any topic related to a political or social issue that
might be subject of lobbying, a key issue in an election, or the subject of community organizing should
receive this category, if not tied to a specific politician or political party (which would indicate political
affiliation) or position (which would indicate political opinion). For example:
o topics such as “gun control,” “Israeli-Palestinian conflict”, “Environmentalism,” “criminal-justice
reform,” or “animal rights”
o documentary film or non-fiction books related to a political issue without statement of a
specific stance
• Political opinion: Political opinions or beliefs not related to specific political parties or membership.
References to “activism” and “political leaning” might indicate a topic is related to political opinion. In
general, “pro-xx” or “anti-xx” are strong indicators of political opinion. For example:
o user-declared opinions on a political topic (e.g., reproductive rights, gun control, immigration,
etc.)
o topics such as “pro-gun control,” “anti-regulation,” or “Support Israel,” “pro-Second
Amendment rights”
o documentary film or non-fiction books with a clear political opinion (e.g., No Safe
Spaces, Citizenfour)
o DOES NOT include firearm ownership, training, or hunting that is not explicitly about a political
opinion related to gun control or gun rights
• Voting activity: Content related to voting, voting choices, or beliefs about voting rights. For example:
o user-declared status as having voted or promoting voting (NOTE: may be co-labeled with
political affiliation or opinion)
o topics or attributes such as “get out the vote” or “I voted!”
15. Regulated products & services: Identifies content related to legally restricted or
culturally restricted products/services and/or products/services linked to damaging health
consequences.
• Alcoholic beverages: Identifies content related to beverages containing alcohol. For example:
o Products and services related to alcoholic beverages (e.g., Barefoot Pinot Grigio, Hendrick’s Gin,
wine clubs)
o Non-alcoholic beverages intended to mimic alcoholic beverages (non-alcoholic spirits, non-
alcoholic wine and beer, cocktail mixes, ‘virgin’ cocktails)
o Topics related to alcoholic beverages (e.g., oenology, distilling, designated driver)
o Businesses or entertainment related to alcoholic beverages (e.g., bars, wineries, brewpubs)
Excludes: incidental images of alcohol, such as in the background of an image, that are not part of
the core purpose
• High sugar, fat, and salt foods: Identifies advertised products (e.g., with commercial intent, not user
posts) related to processed foods linked to detrimental health outcomes, generally characterized by
high sugar, fat, and/or salt content.
o Ice cream, including milkshakes, frozen coffee drinks (e.g, Frappucino, frappé) gelato, frozen
yogurt, and popsicles (e.g., Klondike bars, Ben & Jerry’s, McFlurry)
o Savory snacks, such as chips/crisps, crackers, or rice snacks (e.g., Pringles, Ritz crackers, Hot
Cheetos).
o Sweet snacks, such as prepared cookies, prepared cakes, pastries, sweetened breakfast cereals
and toaster pastries (e.g., Oreo, croissants, Froot Loops, Pop Tarts, bakery treats, etc.). Excludes
mixes and ingredients.
o Instant & fast foods, such as instant/ powdered soups and noodles (e.g., Cup-o-Soup, instant
ramen), pre-prepared (e.g., not homemade, ingredients, or recipes) fast food pizza,
hamburger/hot dog, nugget, and/or fries.
16. Sex life: Identifies content related to expressions of sexual practices or sexual activity,
including relationships and dating.
• Adult products & services:
o user-declared or provided content related to sexual practices, nudity, etc.
o products and services related to sexual practices (e.g., sex work, sex toys, adult film or
magazines)
o culture or topics related to sexuality (e.g., Kama Sutra, human sexuality, sexuality studies,
gender studies, polyamory)
o public figures related to sexual expression or sex life (e.g., adult film actors)
• Dating: content related to the process of dating and forming relationships. For example:
o dating services or apps
o Does NOT include reality or game shows about dating (e.g., Love Island, Married at First Sight,
The Bachelor, etc.)
• Sexual orientation: Identifies content related to patterns of sexual and/or romantic attraction. For
example:
o user-declared sexual orientation
o organizations or associations related to sexual orientation (e.g., The Trevor Project, GLAAD,
American Institute of Bisexuality)
o attributes related to sexual orientation (e.g., “gay,” “bisexual,” “asexual,” “heterosexual”)
o topics and causes related to sexual orientation (e.g., gay pride, marriage equality)
o public figures related to LGBTQIA+ activism (e.g., Harvey Milk), but *not* public figures who
identify as LGBTIA+ without taking an explicitly activist role.
o documentary or non-fiction materials related to sexual orientation, but NOT fictional film or
literature featuring characters or plot related to sexual orientation.
• Sexual partners: user-declared information related to sexual partners, including identity, gender, etc.
17. Socioeconomic Status: Identifies content related to socioeconomic status, especially as it
relates to vulnerability based on factors related to income/assets, education level, and
employment.
• Education level
Ad elements examples:
Note: Not all ads will follow the same consistent format.
[optional] “scroll image” arrow for when more than one image is present in the ad and “more” button to
expand long captions
Examples of Webpages
o URL: Check the URL for specifics about the product or service search. In this case, we can see
that protein powder is the focus of the search
• Some webpages show professional profiles. To the best of your ability, label the page for the main
purpose of the professional service offered by the professional profile or contact. For example:
o A listing of orthopedic surgeons in a geographic area should be labeled will ‘Illness & injury’
because the main purpose of these professional listings is promoting services treating a specific
medical issue.
o A profile of a gynecologist, showing their photo, education, professional experience, and
specialties should be labeled with ‘Sexual & reproductive health’ because the main purpose is
promoting services relating to this specific branch of health care.
• News and research content
News and research content should be classified similarly to other webpage content. However, because
webpage content has such a huge range of news articles and research papers, here are some specific
guidelines for these scenarios.
o Health-related news: Health news and research can range from clear, consumer-facing advice
(“7 Ways to Lower Your Blood Pressure”) to highly-specific professional research (“Factor Levels
with Platelet Count in Colorectal Cancer: Clinical Evidence?”). Regardless of the type or style,
label health news for its main purpose and topic. For example:
▪ If it is about a medical condition, label with Illness & injury, even if the article seems
highly specific or research-based.
▪ If an article is about fitness, label as Fitness & self-care
o Political news: Like medical news and research, political news can range from clear political
opinions to complex scholarly research. To the best of your ability, evaluate the headline and
topic of the news article to determine if it has to do with
▪ a political issue (neutral or unbiased discussion)
▪ a political opinion (taking a side, but not explicitly discussing political party membership
or specific left-wing/right-wing affiliation)
▪ political affiliation (specifically related to one political party or faction)
o Crime & tragedy news: News about crime is not considered sensitive. News stories discussing
crimes committed by specific individuals do not relate to the sensitive category of ‘Criminal
record.’
Keywords
Some users will be asked to evaluate a keyword using the sensitive category labels.
• Read the keyword and decide whether it relates to any of the sensitive category labels. Be sure to
review the labels, as necessary.
• None apply label: use when the meaning of the keyword is clear, but it does not relate to any sensitive
category. For example:
o Keyword: Fun
o Keyword: Industry
• This keyword is not useful label: use when
a. there could be multiple meanings of a word that could affect its sensitivity. For example:
i. Keyword: Afghan (blanket? person from Afghanistan? hound?)
ii. Keyword: Diet (eating patterns? weight loss plan?)
iii. Keyword: Separation (relationship status? separation anxiety?)
iv. Keyword: Dates (fruit? dating?)
b. the context or meaning of the keyword is completely unclear and no assessment can be made.
For example:
Solution: Here, the keyword, ‘trending,’ is very general. It doesn’t have a clear connection with any sensitive
category in any context.
Examples of apps
Sometimes an app might not be available in your location. You’ll use other information in the UI to label the
app.
5. Make a label decision: correct label: Health > Medical conditions > Prescription/OTC drugs
6. Indicate if you were able to reach a decision on this job: correct response: Yes
7. Indicate how you reached your decision: in this case, the correct selection would be Text from UI and
URLs because we were able to gather information about the app from both links, the app category,
and the text description.
Concept workflow
Sample decisions
Description: An Italian soda is a soft drink made from carbonated water and simple syrup usually flavored
Decision: High fat, sugar, and salt foods
2. Concept: grasshopper
3. Concept: beer_mug
Description: none
Decision: Alcoholic beverages (Rationale: no side search necessary - this is clearly related to a sensitive data
category)
4. Concept: fios
Description: none
Decision: None apply (Rationale: a side search shows that '“Verizon Fios, also marketed as Fios by Verizon, is a
bundled Internet access, telephone, and television service that operates over a fiber-optic communications
network with over 6.5 million customers in nine U.S. states.”)
5. Concept: raw_hashtag_ascidian
Description: none
Decision: None apply (Rationale: a side search for the concept portion of the hashtag, ‘ascidian’ shows
that “Ascidiacea, commonly known as the ascidians, tunicates, and sea squirts, is a polyphyletic class in the
subphylum Tunicata of sac-like marine invertebrate filter feeders.”)
Content can be text, image, or video. In this case, you will be categorizing Facebook Page Posts with text only.
Any personal information (e.g., names, phone numbers, etc.) will be removed from the post in the following
format: <redacted_data_type>
• Consider only the core purpose of the Facebook page post. What kind of business, product, or service
is this post intended to promote?
• In isolating the core purpose of the post, avoid labeling extra text that does not specifically relate to
the core purpose (e.g., labeling all duties in a job post for a nursing position)
Approach
In evaluating the image or video documents, you should ask yourself two central questions:
1. What is the core purpose of this image or video?
2. Does the user reveal any sensitive information about themself in the image or video? Are restricted products or
services shown?
There may be tangential mentions of restricted topics, but if they do not indicate a core purpose about a restricted topic they should
not be labeled.
Considerations:
• Evaluate each job fully by watching the full video, examining all images in a series, and reading the full caption.
• Any text and image/video should be evaluated together. Restricted topics could appear in either.
• Any user-provided emojis may be considered for context.
Workflow
1. Evaluate the job by examining all images and reading the full caption text.
For this job, we have plenty of information to make a decision. The images show a range of family pictures from several decades. The
core purpose of the text is clearly to mourn and remember a loved one. The caption reveals that the writer has suffered the death of a
beloved family member.
The restricted topic Tragedy & hardship > Personal loss is the correct label.
While several types of family relationship are mentioned, we don’t exactly know the nature of the family relationship, so no ‘Family
relationships’ category should be selected.
In this job, the image provided a small amount of context, but the core purpose was revealed in the caption. ‘Text’ would be the most
appropriate selection to report what part of the document was most helpful in categorization.