Image Relevance Quick Guide
In this HitApp we are determining the relevance of search queries and search results. All the
queries were issued on the Images Tab; hence the results are images.
Search query: terms used for search on web, for example {cats}
Search result(s): for the purposes of
this HitApp any potentially related
image(s) returned by the search
Quality of the image is evaluated in another HitApp, please, do not downgrade based on
quality!
Start
Image contains pornography, Yes
offensive text or other
unacceptable content
No
Yes
Image is not topically related
to the query
No
Image has some topical Yes
connection to the query, but
there is a mismatch
No
Image is a good result, but
queried topic is not the only
Yes
/ main one, or there is a
distraction from topic
No
Relevant
Images Tab
Search relevance: how good is the image in
the result considering the search query.
Let’s help the internet searchers find the
best images on web!
Please note: if you cannot
understand the query even after
some research, please, use a
checkbox “I cannot understand to
query” to skip a hit.
Detrimental
Detrimental: you wouldn’t expect to
see this on TV or in a newspaper
Not Related: no topical connection
Not Related with the query and won’t be
interesting
Related: could be of interest to the
searcher but is not what they asked
Related for. It can be a feature, location or
any other mismatch.
Relevant but Imperfect:
• There is a distraction, and the
Relevant but query is not the focus of the
Imperfect image, rather a secondary topic
• Not a common version, for
example, a toy or an artwork
• There is a foreign text in the image
Relevant: exactly what the searcher
was looking for.
Image Relevance Quick Guide

UHRS has a bunch of helpful functions that can help you make a good judgment. Please, find
some of them below.

You can research the query in Bing or

You can see the query and some context Google search, or even translate it using
including where the query was issued. the links on top or CTRL + [key letter]

In case images are not

loading, please make
sure to use the links to
research the result

Use the checkbox in case you don’t understand the query,

buttons in the tool or the hot keys on the keyboard as
described on the very bottom of the page and Submit button
to make your judgment.

Query Result Comment

Examples and exceptions for RELEVANT
Query is not asking for a not photoshopped image
[clearly manipulated
{heart-shaped and having a natural image satisfying the query is a
artistic image with cloud
cloud} great luck. It is what the searcher was looking for,
shaped as a heart]
hence RELEVANT.
[someone’s photo from a
book or magazine]

It is an image quality problem which is evaluated

{samurai} in another HitApp. If this image is Relevant to the
query – please, do not downgrade it. RELEVANT.

Even though pink grasshopper is not the most

common, it is still okay to select RELEVANT. The
{grasshopper}
searcher has not specified exact color, so any color
grasshopper should be fine.
Image Relevance Quick Guide

Query Result Comment

Examples and exceptions for RELEVANT but IMPERFECT
RELEVANT result would ideally be focused on the
query. Since the focus is on tourists, this one is
[correct destination, but RELEVANT BUT IMPERFECT.
{famous
the focus of image is
destination}
clearly on tourists] Please note, for example {Paris} -> [image of Eiffel
tower] is RELEVANT, because Eiffel tower is a great
generic representation of Paris for many searchers.

The pathway is clearly the focus of the image. Since

{sea} the query is a secondary topic in the picture, it is
RELEVANT BUT IMPERFECT.

Cultural Background: In the U.S. ‘festivals’ are

mostly music festivals, e.g., pictures of concerts
with a prominent stage and lots of people would be
{festival} RELEVANT. In Chinese ‘festival’ tends to be more
about cultural festivals, for example pictures of
issued in China lanterns for mid-fall.

Considering the background, this result for a query

issued in China is RELEVANT BUT IMPERFECT.

Examples and exceptions for RELATED

The result is the queried object but with different
{couch in Moscow} [couch sold in New York] location. Since it might be still of interest to the
searcher, RELATED
The result has distant topical connection with the
[perfume with the name
{flower name} query, and can still be of interest to the searcher,
of this flower]
hence RELATED
Nor exactly what the searcher was looking for. Not
{3 pack tshirts size
[3 yellow t-shirts] correct, but close and still might be of interest,
XL Pink/Purple}
hence RELATED

Examples and exceptions for NOT RELATED

[lime pits used for old
style tanning process A bit too far from the query meaning, so it is NOT
{vegetable tanning} itself, but not the new, RELATED. Though since there still is some topical
organic vegetable connection, it can be RELATED as well.
tanning]

Thank you for helping make search results better!