Page Usefulness Quick Guide

Page 1: General Information

This Quick Guide will help you work on the Page Usefulness HIT App without disqualification,
and potentially earn a bonus on this task.
Introduction:
In Bing images, we fill the image search results with images from many web pages.
Sometimes, users will click on the image and are taken to the page where the image is
found. We want to ensure this page provides the User with additional value on both the
Search Query and the Image Clicked.
In this HitApp you will be given a User Query (their search), an Image, and a Source Page
(the page where the image is from).
Here are some questions you will be answering:
• What is the User looking for in this query? What is their Intent?
• Was it even worth it for the User to click this link and see this page?
• Did the User learn something new, or did this page not add value to their experience?
• Considering both the query and the image that they clicked, will the User be satisfied
or disappointed with this page?
Here is some context for the situation this HitApp Represents:
1. A User issued a query in the Bing ‘Images’ tab as shown on the first image below.
2. User clicked on one of the images from the search results.
3. The User clicked on the image and was brought to the website the image was from.

Query: {pelmeni recipe} was issued on IMAGES tab in a search engine and returned image results.
Result images to
query of {pelmeni..}
1: Searched

2: Clicked on Image

3: User Brought To
Source Page What You’ll Find in this Document:
1. General Information
Your task will be to see if 2. HitApp Layout / What to Judge on Source Page
3. Identifying User Intents and Judging Process
this Source Page provides 4. Breakdowns for Each Rating
value to the User 5. Additional Rules and Examples
 Page Usefulness Quick Guide
Page 2: HitApp Layout / What to Judge on Page
HitApp Layout:
1. Research Bar
2. User Query Info
3. Image Box / Link
4. Source Page Box / Link
5. Guidelines (click on arrow
to expand, collapse box)
6. Hotkey Instructions

What Content to Judge On Page:

DON’T JUDGE
When judging Page
Usefulness, we should
judge main content only
and ignore the following:

• Navigation areas,
menus
• Ads
• Banners and other
decorative elements
• “Related content”, “see
also”, “more from…”,
JUDGE main “additional Images”
content sections
only • Any additional
suggested content

You do not need to find

the image on the page for
it to be useful (there is a
separate HitApp that
handles this). You only
need to ensure that the
page is giving the user
additional, useful
information to the Query
and Image.

DON’T JUDGE
 Page Usefulness Quick Guide
Page 3: Identifying User Intents / Judging Process
Determining the User’s Intent From Their Search Query
When a User searches for a query on a Search Engine, they have a goal in mind. We call this goal an “Intent”. Queries can
have multiple possible intents (such as “Apple” meaning the fruit or the company).
In this HitApp, we split up these Intents into three main types:
• Image Intent: The user is only looking for images to view, download, etc. They do not need more information.
• Informational Intent: The user is looking for some kind of text / information. It is clear from the query that images alone
will not satisfy the user.
• Mixed Intent: Both Image and Informational intents could satisfy this query, depending on the intent of the user.
Desktop Wallpapers,
Informational Grilled fish recipe,
Disney Coloring Pages,
Image Intent Mixed Intent Chad Long Lawyer Seattle,
Christmas Clipart, Intent 2018 Toyota Corolla for sale
Frog images free
Picasso Paintings,
red bumpy rash,
Tom Hanks,
Yellow Shirt Size SM

Judging Process:
STEP 1: Before submitting a rating, determine if the rating is:
“Broken”, “Cannot Access Content”, “Detrimental”, or “Foreign” applies.
Start Ensure you click on the direct links to the Source Page/Image if they aren’t
loading correctly in the HitApp.

No No The page is written in a No

The page contains The live page link doesn’t load or link
pornography, nudity, language that is not
redirects to the home page of the site.
gore and/or offensive the target language, or
The page requires login or subscription
text. English.
to view part of or all the content.

Yes Yes
Yes

Cannot Access Foreign

Detrimental
Content/Broken

None of the above-mentioned scenarios applies

STEP 2: Check if you can judge the usefulness of

The page contains
highly specialized Yes
Cannot
information, and I am
not an expert in this judge
field.
I can
judge it!
Page is unexpected, it
doesn’t provide any
extra information; link
is not worth clicking.
Yes
page. If the page contains highly specialized
information, you can either select “cannot judge” or
keep on judging depending on how familiar you are
with the topic.
STEP 3: Judge the Page Usefulness!
Extra information on
page is very limited,
tangential to the Page provides extra
No No
image and query, or is information and adds
hard to find; link might value to the User – it is
be interesting for worth seeing!
some Users.
Yes
Yes

Partially Very Satisfied

Disappointed
Satisfied
Please, find some examples and more detail for the above-mentioned labels on the Page 4.
 Page Usefulness Quick Guide
Page 4: Rating Breakdowns
PAGE IS BROKEN
Page link does not load Page link does not load when opening in a separate tab in the browser
Page link redirects to a Page may be redirected to the homepage of the same website, a page
different page stating that this page does not exist or a different website
DETRIMENTAL (more on Page 5)
Any images, videos or ads containing pornography, nudity (bare nipples, sexual organs),
gore (dead/mutilated bodies of people) and/or offensive text (hate speech, extreme swear words,
etc.). Generally, it wouldn’t be allowed on TV News in the United States. Any images or source pages
containing cases of child endangerment/exploitation should be reported.
CANNOT ACCESS CONTENT/BROKEN
The page link does not load, or page requires a User to create a login or subscription to view part or
all the content. If the user can view the content by clicking past the prompt, do not select this option.
PAGE IS FOREIGN
The page is written NOT in the Market Language or English. English is never foreign. Do not take the
location where the query was issued into account.
CAN'T JUDGE
Page contains highly The usefulness of the page is difficult to establish or requires
specialized information to specialized knowledge. For example, query for a medical terms,
Judge specialized math/physics questions, coding examples, etc.
DISAPPOINTED / LOW
Page is Page has image gallery related to
{nightmare before christmas clip
unexpected/irrelevant given Christmas, but not to "Nightmare
art}
this query & image. before Christmas" movie.
Page does not provide any {Abercrombie and fitch women Page contains just the image, and
extra information winter coats} provides no additional info
PARTIALLY USEFUL / PARTIAL
Page has image of Abraham Lincoln
Relevant info is very limited {Abraham Lincoln}
to download but no additional info
Relevant info is hard to get {1970's flight attendant Slide deck where relevant
to uniforms} information is on slide 15
Page reviews several Staples office
Info is tangential to the
{staples office chair 23234} chairs, but it does not mention the
query 23234 model at all.
VERY USEFUL / FULL
Page provides additional info Page talks about a 1967 Chevy Camaro
Mixed Intent: and provides a lot of extra information
related to both the Image
{67 camaro} in addition to the image.
and Query
Page provides collection of Image Intent: Page has images of Princess
images queried {Princess Coloring Pages} coloring pages.
Page Usefulness Quick Guide
Page 5: Additional Considerations / Examples
“Downgrading by one”: “Downgrading” the rating of a Hit is the act of changing the final rating of a hit to one level lower (Very
Satisfied to Partially Satisfied, or Partially Satisfied to Disappointed). If a situation below says to downgrade, work the Hit as normal
and then downgrade your score at the end.

The Source Page is Only Useful to the Image OR Query

In this HitApp, the ideal Source Page will provide useful information to both the Query AND the Image
provided in a Hit. Some Hits have a Query and Image that are about two different topics, and the source
page only addresses one- such as the example on the right. The Query is asking about motorcycle helmets,
but the Image is of a knitted winter hat. A source page that is a useful page about winter hats Completely
Satisfies the Image clicked but the rating is downgraded by one to Partially Satisfied as it doesn’t address
the Query (motorcycle helmet). The same would be true if the source page only had information on
motorcycle helmets (and not winter hats).
Products Out of Stock or No Longer Available
Some useful source pages will have products for sale that are relevant to the query but out-of-stock or
unavailable for purchase.
• Product appears to be temporarily unavailable: judge as if it was available.
• Product is permanently unavailable (expired auction, discontinued, ‘unknown future availability’):
downgrade by one.
Snippet Answers
A snippet is a small clipping of a larger article/webpage. We would prefer to show a user an entire
page/article instead of a snippet. In cases where only a snippet is shown when there was a more
complete version of the page, the final rating should be downgraded by one. The image to the
right is an example of an incomplete Snippet answer that should be downgraded by one.
Foreign Currencies
When judging source pages with products for sale, check the product price currency against the
user's location. If there is a mismatch between the product currency and the User’s country,
the final rating should be downgraded by one.
For Example, in the Hit on the Right:
The User’s country is Thailand and the currency on the source page is in Pounds (£). Pounds are
not used in Thailand, so, even if there is useful information on the store page, the final rating
should be downgraded by one. This also applies to US Dollars if they are not the currency of the
User’s location.
Subscriptions, CAPTCHAs, Pop-ups
Handle any cases where content is blocked on the page in the ways below:
• Content Requires User to Login or Subscribe: Use the Cannot Access Content rating if you cannot view part or
all of the content.
• CAPTCHAS/Challenge-Respond Test: Complete challenge and work the hit as normal. (No downgrade needed)
• Pop-ups/Coupons/Newsletter Offers/ ‘Cookies’: If these can be canceled/exited/accepted to view the content,
work the hit as normal. (No downgrade needed)
Additional notes on Detrimental rating
Acceptable on pages (not detrimental):
• Graphic imagery as a result of medical queries (surgeries, symptoms, etc.)
• Images with people in garments that completely cover genitals
• Images with nudity that is pixelated/blurred
• Well-known artwork showing nudity
Not Acceptable on pages (detrimental):
• Images of explicitly sexual acts or sex toys, even if they are pixelated/blurred Swimsuits Well-Known Art
• Images with any visible genitalia (bare nipples, sexual organs).
Pixelated (non-sexual)
This includes medical-related images and images with see-through garments.
nudity
Pages Dedicated to Image Download
The sole purpose of some image Source Pages (stock image pages like
GettyImages, Flickr, iStock Photo, etc.) is to provide users with the ability
to download an image in higher quality (high-resolution, non-
watermarked). If a Page allows a user to download/purchase a higher-
quality version of the image in the Hit but provides no additional useful
information, judge the Hit based off the user's query intent.
For Example, in the Hit on the right:
If User query has an image intent, the User would be Very Satisfied as it
provides them with a high-quality image for purchase/download. Image intent query
If User query has an informational intent, the User would be
Disappointed as the page doesn’t have any additional information. Informational intent query
If User query has a mixed intent, the final rating should be at least Mixed intent query
Partially Satisfied, as there is the possibility of both intent types.