Guidelines

This hit app asks judges to select channels that have mostly content. Judges should visit the channel home
page, inspect the recent videos, and decide whether channel is a defect. Channel should be treated as
defect channel if more than 30-40% of the videos on channel home page are defect videos.

Update: 5.26.2022: Please see the section on RED and GREY CONTENT. For some defects, you may have
to additionally say whether the content is RED or GREY.

Update: 4.29.2022: Please see the additional examples here

Updates to guidelines:

- Don’t mark geo-restricted videos as “Inactive”. Try to judge them based on the video title.
- Stale news should be marked as a defect (i.e. video making official announcement about Bill
Gates Divorce in 2022, video in 2022 showing USA’s presidential election results from 2020).
- Usage of profanity/sexual words: Videos that use words like “asshole, f**k, D**k, p***y, c**t,
“sexy”, “masturbate” etc should be treated as defect.

url: https://www.youtube.com/watch?v=z6YzhQt0v28 this video is not

(the uploader has not made the video available “IN YOUR COUNTRY”) playable in India,
but it can be
assumed that it’s
playable in USA.
Further, based on
metadata like
channel name
(official Saturday
Night Live channel,
subscriber count
12 mn, view count
180K and likes 3K)
it can be inferred
that this is a good
video. Don’t mark
such cases as video
didn’t play.

Newly added defect segments (4/20/2022)

 - Stale news should be marked as a defect (i.e. video making official announcement about Bill
Gates Divorce in 2022, video in 2022 showing USA’s presidential election results from 2020).
- Usage of profanity/sexual words: Videos that use words like “asshole, f**k, D**k, p***y, c**t,
“sexy”, “masturbate” etc or use some visual representations, emojis, gestures to communicate
such words same should be treated as defect.

Low authority examples:

Newly added defect segments:

- Drugs/Alcohol/Tobacco: Channels that are dedicated to promote or show usage of drugs

(marijuana / cocaine etc) or alcohol or tobacco should be treated as defects
- Violence/gore/blood in Gaming videos: Channels showing such videos should be treated as
defect. But for channels showing ordinary gaming videos (Minecraft/fortnite/Maria etc) need not
be treated as defect.
- Low Authority sources: E.g. Low authority news sources, sources indulging in name-calling, insults
etc. can be classified as defect.
- Minor news: normally perform at least one of the functions listed below:
 •regional news/local news, not a story about well-known individuals
•Not US national news, international news, or news about international
politicians, celebrities.
o National news: The reporting of events around the country through
traditional and new media formats. Organizations report the facts
without bias.
o International news: International news refers to news about countries
other than the one where you live and news that involves multiple
nations.
- Clickbait: Content that has misleading titles/manufactured or fake thumbnails that are
intentionally used to attract users to click.
Usually we can consider label as "Clickbait" when the title shows these words.

shocking unbelievable be shocked flawlessly

startled stunned least desirable most devastating
incredible implausible no longer exist the best and worst
make your jaw drop you just won’t believe you’ve never heard of you'll never believe
can you believe can you guess you won’t believe you’ll never guess

Other clarifications:

- For Sports Clubs channels like Barcelona, Manchester United etc. mostly they will have a mix of
advertising and other very popular viral videos. Hence, you are advised not to treat them as
defect, unless such a channel is entirely filled with advertising/promotional videos.
- Magazine channels / publications and major publishing companies (e.g. Buzzfeed, Vogue,
Cosmoplitan, Autocar, NYtimes, BBC, Guardian etc) should not be treated as a defect.
- For movie studios that upload trailers (e.g. Disney, Star Wars, Pixar Animation, Marvel Comics) –
such sources shouldn’t be labeled as defect.

For this hitapp, “Foreign” is to be interpreted as

- Content that’s not in English language

- Content not relevant to United States users (e.g. minor local news from Kenya, local news
from Indian cities, or minor area-based teams etc.). Exception: If the video is in English and
it’s about internationally popular entities (e.g. European football, internationally famous
musicians) or internationally relevant news (e.g. Covid spike in some other countries, major
events of international significance outside USA), Travel destinations outside USA, or a
General Topic like science/economics/geography/world politics that could be equally relevant
to users from any country– such videos need not be labeled as defect.

Types of defects:

Type Examples (videos) Example (channels)

Adult, Racy, or
potentially
indicating on
thumbnail
(including exercise https://www.youtube.com/c/SexyHotGirlsMus
videos that have ic/videos
less emphasis on
exercise and
contain suggestive
poses)

Shocking or
frightening
thumbnail, people
potentially feel
sick, https://www.youtube.com/channel/UCBnbnH
frightening/scary 7DGXT9yBBVFbZeIwQ/videos
strong negative
emotions.

Guns/weapons-
oriented content.

https://www.youtube.com/user/hickok45/vide
os
Foreign language
or not market-
preferable
content for USA
market (eg Hindi Foreign market
songs, local https://www.youtube.com/channel/UCV0q8V
Nigerian news 7lWrEex5pfL7yhYVg/videos
etc).

Advertising/prom
otional videos
(videos uploaded
by
companies/brands
/ promoting their https://www.youtube.com/c/Honda/videos
own products)

Pimple popping
videos, and other
such icky stuff

https://www.youtube.com/user/DrSandraLee

Violent content
1. Use or promote
violence
2. Content that https://www.youtube.com/c/grimpew/videos
support violent
extremism,
terrorist
organizations,
gangs, cartels, or
criminal
organizations
3. Traffic accidents
that cause
casualties or
vehicle
deformation or
damage.
• Cars
collide/pla
ne
crash/Ship
wreck/trai
n
derailmen
t
• Car/bike/T
rain
Crashes
with Non-
motorized
vehicle
• vehicle
spontaneo
us
combustio
n

Close up Views of https://www.youtube.com/user/prehistoricpet

reptiles and stv
insects

Educational videos Education about sex

about surgery, https://www.youtube.com/channel/UCIRoo9L
scientific aspects sz1DOXRIvEo-UARQ
of sex and sexual
organs etc.
Prank videos
showing
unpleasant
pranks/suggestive
/sexual etc. https://www.youtube.com/c/jackvale/videos

Minor news:
normally perform
at least one of the
functions listed
below:
• regional
news/local
news, not
a story
about
well-
known
individuals
• Not US
national
news,
internatio
nal news,
or news
about
internatio
nal
politicians,
celebrities
.

Clickbait: Content
that has
misleading
titles/manufacture
d or fake
thumbnails that
are intentionally
used to attract
users to click.
If there are any content that makes the judges feel uncomfortable but is not listed in the rules above,
please select “other detrimental content”.

How to judge:

- Visit the channel home page/videos page

- Usually, a quick glance at the video title/thumbnails should tell you whether the majority
of videos from channel are defect or not. Occasionally you may want to watch a few seconds
of the video.
- Channel should be treated as defect channel if more than 30-40% of the videos on channel
home page are defect videos
-

To become a judge for this hit app, users need to pass the qualification test. Users can use the preview
questions as practice.

Note: Sometimes channels that are otherwise good, may have some few defect videos. E.g. BBC Earth is
a good channel, with mostly good videos, with the odd bad video in between (about snakes).

Why these guidelines:

Our final goal is to provide video recommendations to Bing users without knowing their interests. This
part is key. It’s different from search, where the users explicitly indicate their interests via search queries.
In the recommendation scenario (where videos are implicitly shown to users), we need to be more careful
about what kind of videos we show, and past research indicates that above types of content
(adult/racy/pranks/surgery/reptiles/gruesome/foreign language/boring advertisements) are not suitable
to show as recommended videos, unless we know that the user is specifically looking for them. For the
purposes of this hitapp, you can assume that the user is not interested in any of the above “defect” topics.

Additional examples

video url title dia reason

gn
 osi
s
https://www.facebook.com/viralhog/vi Alligator Snaps at Swimmer de close up of
deos/alligator-snaps-at- fec scary animals
swimmer/459924482112246/ t
https://www.youtube.com/watch?v=Zn Court Cam: Son Commits Vicious de frightening and
T-B0uckqk Crime to Collect Life Insurance fec violent
Money From Parents | A&E t
https://www.youtube.com/watch?v=m Ambulance Called After Car Flips 8 de frightening and
BChrMMA7CI Times In Horror Crash I Street fec violent content
Outlaws: Memphis t
https://www.youtube.com/watch?v=ov Court Cam: Murderer Is de frightening and
qy6ZpRMRQ Emotionless When Describing fec violent content
Killing Woman | A&E t
https://www.youtube.com/watch?v=A5 He Went Missing.. They Found de frigtening,
Oc2T__lNE Him In a Snake.. fec violent,
t disgusting
content
https://www.youtube.com/watch?v=0s LeBron James yells I m a mother f de profanity, use
hviZ7gGk4 king problem b-tch at Kings bench fec of bad language
after this... t
https://www.youtube.com/watch?v=O3 Serena and Venus Williams half de use of clickbait,
RN8sA8Kcw sister goes NUCLEAR on their dad fec extreme
Richard & Will Smith (TRUTH t exaggeration
EXP0SED)
https://www.youtube.com/watch?v=irT She Did THIS For $200... de profanity, bad
yrmxZfMY fec language is
t used in the
video
https://www.youtube.com/watch?v=Fw Court Cam: Woman Strips in de adult and racy
UroHMknro Miami Courtroom | A&E fec
t
https://www.youtube.com/watch?v=DT Man Protects His Wife From de frightening and
Y9FaQbUuc THREE Home Invaders | I Survived fec violent
a Crime | A&E t
https://www.youtube.com/watch?v=Nx Everything on the internet is fake de annoying
uPiyfUCQg #shorts fec pranks
t
https://www.youtube.com/watch?v=aZ Will Smith Admits To Vomiting de upsetting
GfGlk8ZiM After Sex With Women fec content
t
https://www.youtube.com/watch?v=W Crazy Dashcam Fails - Bad Drivers de frightening and
UoIgQI2wPA and Road Mayhem Compilation | fec violent
FailArmy t
https://www.youtube.com/watch?v=AV UFO FROM ANOTHER SOLAR de profanity, bad
mHdZ-YE9Q SYSTEM - What the F*** Was fec language in
t video titel
 That?! | The Proof Is Out There |
#Shorts
https://www.youtube.com/watch?v=uQ Court Cam: Man SPITS on Judge, de frightening and
SokD4kD4Q Then Starts Jail Riot | A&E fec violent
t
https://www.youtube.com/watch?v=dS Driver Stretchered Away After de frightening and
xvyXHnEFc Horror Crash Knocks Him fec violent
Unconscious I Street Outlaw t
https://www.youtube.com/watch?v=sP The Walking Dead Scene That de frightening and
9eo1qhg8o People Think Went Too Far fec violent
t
https://www.youtube.com/watch?v=tN 90 Day Fianc : Angela FLASHES de frightening and
obleleEcE Everyone During Tell-All Special fec violent
(Exclusive) t
https://www.youtube.com/watch?v=mi Neighborhood Wars: Neighbor de frightening and
l1FJGBq3Q ATTACKS Car With Hammer Over fec violent
Parking Space | A&E t
https://www.youtube.com/watch?v=lp Man Goes WILD with Road Rage de frightening and
T3GsVSyak After Car Accident | Neighborhood fec violent
Wars | A&E t
https://www.youtube.com/watch?v=f_ Baby Burned By Mother? | Steve de frightening and
P1tL1mXnw Wilkos fec violent
t
https://www.youtube.com/watch?v=nzi WRAPPING MYSELF AS AN EARLY de adult and racy
WpIMYGkE CHRISTMAS PRESENT FOR MY fec
BOYFRIEND! t
https://www.youtube.com/watch?v=4 Louisiana Alligator Cook! #shorts de close up of
WF9bAvR_50 fec scary animals,
t disgusting,
scary video
https://www.youtube.com/watch?v=j9 GUERO10k KILLED HIS OPP S de frightening and
NPq-sondg GRANDPARENTS, FACING DEATH fec violent
PENALTY t
https://www.youtube.com/watch?v=Cx Meghan is now a living walking de defamatory
gxWfNqvdw Hollywood Joke, and a listers are fec content, low
avoiding her! Part 1 t authority
source
https://www.youtube.com/watch?v=0D Hunters SHOCKED - They Just de low authority
3lJ2JYFZ0 Stumbled Upon Bigfoot! | fec content
Mountain Monsters t
https://www.youtube.com/watch?v=m 5-Minute Crafts - Handyman tips no
1_1qdneBfM and hacks that work extremely de
well! 🧑 ?🔧 fec
t
https://www.youtube.com/watch?v=N Dog Decides to Rescue Girl Playing no
Q7z7-msq0U in Ocean de
 fec
t
https://www.youtube.com/watch?v=fM Armored Warfare - Riders on the no
NpuhHv2PU Storm Trailer de
fec
t
https://www.youtube.com/watch?v=1l When Achraf Hakimi pranked his no
XwHlyI4ks wife 😅 | #shorts de
fec
t
https://www.youtube.com/watch?v=UR Boyfriend Goals After a Long Day no
M9aGz8PG0 😍 #shorts de
fec
t
https://www.youtube.com/watch?v=kE Ted Cruz makes dumbest move de bad
faj97Q96U humanly possible at press fec language/defa
conference t matory content
https://www.facebook.com/skysportsf1 Sky Sports F1 - New rules on tyres in no
/videos/new-rules-on-tyres-in- France de
france/211424690708816/ fec
t
https://www.youtube.com/watch?v=X4Cw56 What I Eat In A Day On The Starch no
gd4e8 Solution For Maximum Weight Loss / de
Low Calorie Density fec
t
https://www.facebook.com/5Minute.Crafts.Gi 5-Minute Crafts GIRLY - Cool and no
rly/videos/cool-and-cheap-bushcraft-diy- cheap bushcraft DIY ideas you can de
ideas-you-can-make- make yourself fec
yourself/105098031533026/
t
https://www.youtube.com/watch?v=KKvW_z2 20 Avatar Mysteries That Need no
q7-U Explaining de
fec
t
https://www.youtube.com/watch?v=NjO32b- Axis prototype coaster front seat on- no
qzlQ ride 4K POV @60fps S&S Worldwide de
facility fec
t
https://www.youtube.com/watch?v=9wGXD4I Chess game are Real in The Queen no
Tt4w Gambit de
fec
t

RED and GREY CONTENT

IN SHORT,
 - RED content = SHOCKING AND DETRIMENTAL content totally unsuitable for showing to
anyone on video FEEDs.
- GREY content = DEFECTIVE content that may be okay to show to a limited extent.

Adult and Racy: Automatically consider it as RED content.

Frightening and Violent:

- Mark this content as

o RED content IF video shows gruesome/shocking video footage of death/extreme
violence/very serious injuries etc
o GREY content IF video shows none of the above, e.g. news report about
stabbing/murder without footage of the stabbing/murder, news report and footage
about bomb explosions from Russia Ukraine war without footage of people getting hurt,
footage of volcanoes, floods, natural disasters that don’t show people getting seriously
hurt etc.

Icky stuff:

- Mark this content as

o RED content IF video shows gruesome content like ear wax removal, pimple popping,
surgeries etc.
o GREY content otherwise

Close up views of scary animals

- Mark this content as

o RED content if video shows gruesome details of reptiles (snakes/crocodiles/alligators) or
insects killing/feeding etc.
o GREY content otherwise

Extremely Low authority sources for topics like Politics/Covid/QAnon/any conspiracy theories/ etc

- Mark conspiracy theories about Covid/politics/QANON or any other subject as RED CONTENT
- Mark Low authority sources on crime news as RED CONTENT

IN SHORT,

- RED content = SHOCKING AND DETRIMENTAL content totally unsuitable for showing on video
FEEDs.
- GREY content = DEFECTIVE content that may be okay to show to a limited extent.
Video title Area Classification
❤ How to do Sexual Foreplay & Adult & racy RED
Undo Bra ❤ - YouTube
T0yFun ~Adult Toy Reviews~ Adult & racy RED
The Ultimate Rabbit ~ Soft
Touch TPU Silicone Rabit
w/Perfectly Ears - YouTube
KISSING HER MELONS 😘 TO Adult & Racy RED
SEE HOW SHE REACTS 🥵💦
*she didn’t last 1 minute* -
YouTube
Eating bugs and Carolina Icky stuff RED
reapers VOMIT WARNING! -
YouTube
PUKE Fruit Challenge! GROSS! - Icky stuff RED
YouTube
CCTV shows murderer laughing Frightening and Violent RED
with friend shortly before killing
him - YouTube
CCTV Footage: Man Stabbed to Frightening and Violent RED
Death in Knife Attack at Noida,
UP - India TV - YouTube
VIDEO NOW: Police pursuit Frightening and Violent GREY
ends in multi-car crash on I-195
- YouTube
Russian tank explodes hurling Frightening and violent GREY
turret 250ft into the air after
Ukrainian missile strike -
YouTube
Russian ‘tank graveyard’ after Frightening and Violent GREY
52 vehicles blown up in one
battle as Putin forced into
retreat - YouTube
Police: Ohio teen caught having Adult & Racy RED
sex with wiener dog again -
YouTube
Car Crash Compilation 2021 #11 Frightening and Violent GREY (doesn’t show serious
- YouTube injuries)
The Inc*st Dad Who Went On A Frightening and Violent RED
K*LLING Spree After Being
Taken From Lover/Daughter -
YouTube
The Girl Whose Jealousy Led To Frightening and Violent RED
Her T*rturing Her FRIEND -
YouTube
Drone footage shows destroyed
Azovstal steel plant in Mariupol
- YouTube
Woke female college student
has an epic meltdown when a
pic of a slave found on lamppost
at campus. - YouTube
Benny Johnson - CREEPY JOE Is
At It AGAIN | Facebook
Benny Johnson - Hillary gets
CAUGHT orchestrating Russia-
Hoax after SHAMELESSLY
peddling MASSIVE lie in public
Remember this? | Facebook
19 kids killed, suspect dead
after Texas elementary school
shooting - YouTube
CCTV Footage Of Car Accident
On Nizambad Hyderabad
National Highway - YouTube
Drone video shows two-car
rollover crash on Seattle's Ship
Canal Bridge | FOX 13 News -
YouTube
Deadly 70-car pileup in Iowa
caught on camera - YouTube
5 Biggest Tsunami Caught On
Camera - YouTube
Frightening and Violent GREY (news report)
Low Authority Content on RED
politics
Low Authority Content on RED
politics
Low authority content on RED
politics
Frightening and Violent GREY (news report and doesn’t
show footage of
death/injuries)
Frightening and Violent RED (shows death/injuries)
Frightening and Violent GREY (doesn’t show
death/injuries)
Frightening and Violent GREY (doesn’t show
death/injuries)
Frightening and Violent GREY (Doesn’t show
death/injuries)